From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x236.google.com (mail-oi0-x236.google.com [IPv6:2607:f8b0:4003:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 083BF1A1DF9 for ; Tue, 30 Aug 2016 08:17:32 -0700 (PDT) Received: by mail-oi0-x236.google.com with SMTP id l203so30306343oib.1 for ; Tue, 30 Aug 2016 08:17:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=C8WOWjWN9Oz+jOAfaTj6xxzOnuLPci94LczQXNViSCY=; b=J8HNh1Q3ZpuIl1WZ5ov9J6SjHItN3gQGLgrs4B3oWtGyAJAYTKtM+t7bXdAwURxwJg n88aUb8Avb/3dFpuKyGjZpTC2gt5QyxdytMOuaxT3wi0B6WrreHuKLK3svow3XpdjhSd L0gzyl7RkaG7K3O/jGEJ0gtN6gFAWjT8TRb55Jb6UgjfbMbra4vbMu/NO2qpv8B49UqL VZLXoX6vXbhWP7zATjzAU3/RNlY+gcaBI1wm9Sx9oCG21ti4Hc+KSAdD/fjwPEnDP8Fr 8QfTjRbvJVkAJjbHQuiDb63fLlkjoHfb0FToMvpUJXjqedLKW6QjeEM7qfVmokSZeKZW bM+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=C8WOWjWN9Oz+jOAfaTj6xxzOnuLPci94LczQXNViSCY=; b=i3CXZCwYy9ysDjicY9pvFw5xbh6o7ujE8RHByyDyBqjepSgEZA4YeDbcvlkIg49MkY 2vGx/QXbHcOeo02lLDW2sRM6xSkEyjsT1O1x5OrTIg5Wm4NZhZawmeOYXsEQ+SNxDP+6 kcRfN0h2jAtVB7uADSwTb+/X2Oe970sJqU6YIqzG2Zjq37ov1NBcJGzitmZQ3Ir9PCP1 vz9tJ6GSVUa80zsSfk/OpsXsnhcDcuWrJIzlLHCIANZChqTqkPtRvglpOvASuPzgdUI1 7D47/UzcPSMnGxpqTFSz4PHjlg8nYdSj+6snkZ1/92pAZH5hyBfeqZRtp/L3NnD8WPS0 LqUQ== X-Gm-Message-State: AE9vXwOlINeNH4O+OVKlJxMAVo8Kvgp5s5JZRDqv8dtHT1j8pxZX2nqn2s7uoRxJBtEDJI7bGfhtVk25RGfFZw== X-Received: by 10.202.4.149 with SMTP id 143mr4197646oie.91.1472570251083; Tue, 30 Aug 2016 08:17:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.50.86 with HTTP; Tue, 30 Aug 2016 08:17:30 -0700 (PDT) In-Reply-To: <91B24DF7-4F53-4DEF-A600-755A69368B79@apple.com> References: <91B24DF7-4F53-4DEF-A600-755A69368B79@apple.com> From: valerij zaporogeci Date: Tue, 30 Aug 2016 18:17:30 +0300 Message-ID: To: Andrew Fish Cc: Michael Zimmermann , edk2-devel Subject: Re: Crc32 X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Aug 2016 15:17:32 -0000 Content-Type: text/plain; charset=UTF-8 well, that bit in the 32th (bit 33) term of the polynomial is "implied", but it should be used, otherwise it would not give a 32 bit remainder (crc). if to count 04c11db7 as a polynomial, its non-zero msbit is bit 26, so with it, it would be crc25. Why results don't match, I suspect it's becuase that pre- and post- processing involved. Tianocore algorithm inverses bits at the beginning and in the end. Pure definition of CRC doesn't require this. It would be good to know the exact specification on the algorithm and its nuances. And of course in the pure case, calculating CRC on the polynomial itself should give 0 (since polynomial itself is multiple of itself). It's the easiest way to check. So CRC of 104c11db7 = 0 In the the above calculator (in the Michael's response), neither CRC(104c11db7) nor CRC(04c11db7) gives 0. (4E24FFBE and B79B7E7A respectively). So if it is a right CRC32, then it's not a pure CRC mathematically. and additionally processed. How? and where it is possible to learn that? 2016-08-30 17:09 GMT+03:00, Andrew Fish : > >> On Aug 30, 2016, at 6:32 AM, valerij zaporogeci >> wrote: >> >> Thank you. >> But the polynomial value for CCITT crc32 is 104c11db7, not its tail >> 04c11db7. > > The spec does not say it is a CCITT, it states it uses the CCITT algorithm > with a seed polynomial of 0x04c11db7. > > >> And this whole mess with all those online calculators was the exact >> reson I asked here. Because they mostly give WRONG result. As your >> example. It's easy to check it's wrong. Because for example leading >> zeroes in the message MUST not change the Crc32. >> Here we have: >> for 01 -> CD39D477 >> for 00 01 -> D71E16AC >> and so on. >> Also, it's obvious that Crc32(1) would be a polynomial without the >> most significant non-zero bit, thus crc32 from 1 by the polynomial >> 104c11db7 would be 04c11b7. >> The only online calculator so far giving right results is here: >> https://ghsi.de/CRC/index.php?Polynom=100000100110000010001110110110111&Message=0104c11db7 >> >> I wrote function following the mathematics of the Crc and it gives >> results exactly like in the above calculator. >> It's fun to imagine what happens with such an incompatibilty for >> example in case of GPT header. >> > > As far as I can tell this is used in a lot of real world things: HDLC, ANSI > X3.66, ITU-T V.42, Ethernet, Serial ATA, MPEG-2, PKZIP, Gzip, Bzip2, > PNG,[36] many others[1] > > I think Ken added this to the spec in the last millennium. Ken's background > was networking and Windows X86 HAL so I'm assuming he picked the same CRC as > Ethernet? > > It is always possible we could have botched the math some place along the > way and not noticed as we are self consistent so everything appears to > work. > > Thanks, > > Andrew Fish > > [1] https://en.wikipedia.org/wiki/Cyclic_redundancy_check > > >> 2016-08-30 14:08 GMT+03:00, Michael Zimmermann >> : >>> well as u already said, the spec says it uses 'a standard CCITT32 CRC >>> algorithm with a seed polynomial value of 0x04c11db7' >>> >>> this is the implementation which confirms it: >>> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/RuntimeDxe/Crc32.c >>> >>> after testing it it indeed produces CCITT32 results like this online >>> generator: >>> http://g6auc.me.uk/CRC32/index.html >>> >>> Thanks >>> Michael >>> >>> On Tue, Aug 30, 2016 at 2:54 AM, valerij zaporogeci >>> >>> wrote: >>> >>>> Hi, all. >>>> Yet another dumb question from me. >>>> UEFI specification has Crc32 calculation service and uses Crc32 in >>>> several places. but it only humbly mentions in the note somewhere in >>>> the description of System Table about what exact one it wants. Namely >>>> it states that the polynomial seed is 04c11db7. And that's all. >>>> My question is - does really the specification means the 33-bit >>>> polynomial >>>> 104c11db7? And is the algorithm just a plain remainder calculation >>>> without any additional pre/post processing of it? So that for example >>>> Crc32 of the 4-byte sequence b16b00b5 would be >>>> 8c1f0a7c? >>>> Thank you. >>>> _______________________________________________ >>>> edk2-devel mailing list >>>> edk2-devel@lists.01.org >>>> https://lists.01.org/mailman/listinfo/edk2-devel >>>> >>> >> _______________________________________________ >> edk2-devel mailing list >> edk2-devel@lists.01.org >> https://lists.01.org/mailman/listinfo/edk2-devel > >