CRC32 Document?

1

Hi,

I’m converting an old clarion project to Java. The program uses an internal crc32 function.

I have not been able to find the doc on this function.

module ( ‘core clarion lib’ )
CRC32 ( long , unsigned , ULONG ), ULONG ,RAW,NAME( ‘Cla$crc32’ )
end

The third parameter is labeled a seed. Anyone know the inner workings of this class? I have to match it in a Java program and don’t want to guess.

usage is like this
crc ulong

crc = CRC32(address(bs),x,seed)
crc = band(crc, 0FFFFFh)

I guess the seed is the starting crc value. Good or bad guess?

Anyone know?

2

Tried a simple CRC of ABCD. Does not match with a 0 seed between java and clarion.

Java
public static void main(String args) {
CRC32 crc32 = new CRC32();
String test = “ABCD”;
crc32.update(test.getBytes(StandardCharsets.US_ASCII),0,4);
System.out.println(crc32.getValue());

    crc32.reset();
    crc32.update(65); //A
    crc32.update(66); //B
    crc32.update(67); //C
    crc32.update(68); //D
    System.out.println(crc32.getValue());

both of these produce the same result 3675725989

Clarion
bb cstring (100 )
CODE
bb = ‘ABCD’
message('ABCD 0 '& CRC32(address(bb),len(bb),0)) ! 4199808953

3

Screen Shot 2021-09-21 at 12.02.14 PM
Screen Shot 2021-09-21 at 12.02.14 PM2114×1164 213 KB

This site has the crc32 value of ABCD matching the java calculation.

What is the algorithm behind the Clarion crc?

4

I started to play, to search for a seed
this code finds one - and then quits
I haven’t tested it against other inputs

I suggest altering the code to search all seeds
and show all seeds that return the desired answer

  PROGRAM


  MAP
    MODULE('RTL')
      CRC32( LONG, UNSIGNED, ULONG), ULONG,RAW,NAME('Cla$crc32')
    END
  END

bb CSTRING(100)
Seed ULONG 

AddrBB        LONG,AUTO 
LenBB         UNSIGNED,AUTO
CorrectAnswer EQUATE(3675725989)

Window WINDOW('Caption'),AT(,,395,224),GRAY,AUTO,FONT('Segoe UI',9)
		BUTTON('&OK'),AT(291,201,41,14),USE(?OkButton),DEFAULT
		BUTTON('&Cancel'),AT(340,201,42,14),USE(?CancelButton),STD(STD:Close)
		STRING(@N20),AT(23,27,266),USE(Seed),FONT(,24)
	END

EVENT:User:CheckMore EQUATE(EVENT:user)
  CODE
  bb = 'ABCD'
  AddrBB = ADDRESS(BB) 
  LenBB  = LEN(BB)
  MESSAGE('ABCD 0 ['  & CRC32( AddrBB, LENBB    , 0 ) & ']' |
           & '|Len+1['& CRC32( AddrBB, LENBB + 1, 0 ) & ']' |
         ) 

  OPEN(Window)
  POST(EVENT:User:CheckMore)
  ACCEPT 
    CASE EVENT()
      OF EVENT:User:checkMore ; DO CheckSeeds
                                DISPLAY(?Seed)
    END 
  END 


CheckSeeds ROUTINE 
  DATA 
rou:Seed ULONG,AUTO
rou:MatchFound BOOL(FALSE)
  CODE 

  LOOP rou:seed = Seed to Seed + 1000
    IF CRC32( ADDRBB, LENBB, rou:Seed ) = CorrectAnswer
                                                 
       MESSAGE('Seed['& rou:Seed &']')
       rou:MatchFound = TRUE 
       BREAK 
    END 

    IF CRC32( ADDRBB, LENBB + 1, rou:Seed ) = CorrectAnswer

       MESSAGE('Seed['& rou:Seed &'] with Len+1')
       rou:MatchFound = TRUE 
       BREAK 
    END 
    IF rou:Seed < seed
      MESSAGE('Seed overflow')
      BREAK 
    END 
  END 

  IF ~rou:MatchFound 
     Seed += 1000
     POST(EVENT:User:CheckMore)
  END 


! match 130,156,088 with Len+1
5

Thanks. But, I need some proof it will match on all comparisons not a specific input string.

It was my error for using an internal crc method 20 years ago.

I used the calculator at http://www.sunshine2k.de/coding/javascript/crc/crc_js.html and tried a bunch of different options.

I think I found the options that match the clarion calc. Using a seed = 0 which is “initial value” on the site and a final xor value of 0. It matches the clarion calc for two different input strings: ABCD and ABCDEFGHIJKLMNopqrtsuvwzzz.

I think I can create a custom calc now that matches it. Thanks.

6

Here is the clarion usage that matches the other CRC32 calcs

bxor ( CRC32 ( address (bb), len (bb), 0FFFFFFFFh), 0FFFFFFFFh)

initial seed of 0FFFFFFFFh then xor at end (or compliment it)

I didn’t test below but assume this is the correct usage pattern

1st call
crc = CRC32** ( address (bb), len (bb), 0FFFFFFFFh)

2nd + call
crc = CRC32(address(xx), len (xx), crc)

finally

crc = bxor(crc, 0FFFFFFFFh)

7

Jared,
You could make Clarion result equals to Java result with what you have found.
The opposite is trickier, as I think the included Java implementation does not give you access to the initial seed.
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/zip/CRC32.java

You could do reversing each byte of the input first: b ^ 0xff , then reverse the result ^ 0xffffffff. To test that compare the Clarion result for ABCD: 4199808953 (0xFA53FFB9) then on
http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
choose CRC-32 / Custom / change Final Xor Value to 0x0 / choose Bytes on CRC Input Data: 0xBE 0xBD 0xBC 0xBB (ABCD ASCII byte reversed)

Or use other implementations
https://introcs.cs.princeton.edu/java/61data/CRC32.java.html

Interesting optimizations
http://pzemtsov.github.io/2015/11/21/crc32-intrinsic-java-or-c-with-jni.html
Federico

8

Thanks. I ended up using the algorithm and table from the un-restricted c file source below to create yet another java version of crc32.

https://web.mit.edu/freebsd/head/sys/libkern/crc32.c

Converting the C to java was a pleasure compared to the amount of time spent researching the details.

9

Hi Jared

here is a Clarion version that I did for someone on the newsgroups a few years ago.

it should be compatible with other languages etc and is very fast:

VitCRC              PROCEDURE  (*STRING pStr)     
i    long,auto
crc  ulong,auto

  CODE
  if ~address(pStr) then return 0. ! just in case
  crc = 0FFFFFFFFh
  loop i = 1 to size(pStr)
    crc = BXOR(crc,Val(pStr[i]))
    loop 8 TIMES
      crc = BXOR(BSHIFT(crc, -1), BAND(0EDB88320h, -(BAND(crc,1))))
    end 
  end 
  return BXOR(crc,0FFFFFFFFh)

cheers

Geoff R

10

Thanks.

The code matches the pattern of initial seed of 0FFFFFFFFh with a final BXOR(crc,0FFFFFFFFh).

The code segment below should produce equivalent results to the internal crc32 function in the runtime lib. I didn’t validate it.

loop i = 1 to size(pStr)
    crc = BXOR(crc,Val(pStr[i]))
    loop 8 TIMES
      crc = BXOR(BSHIFT(crc, -1), BAND(0EDB88320h, -(BAND(crc,1))))
    end 
end