Skip to content

cksum: implement the digests, the output modes and check mode - #199

Open
metif12 wants to merge 1 commit into
vlang:mainfrom
metif12:fix/cksum
Open

metif12 wants to merge 1 commit into
vlang:mainfrom
metif12:fix/cksum

Conversation

@metif12

@metif12 metif12 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Closes #188.

cksum computed one CRC over a whole file, had no way to verify anything, and its own test file did not compile against current V. This implements what GNU's cksum 9.4 does.

The digests

sysv, bsd and crc stay numeric and print a size column, each with GNU's own definition rather than the V stdlib's:

  • sysv sums the bytes and folds the carry three times, and counts 512 byte blocks, so cksum -a sysv of the 25 byte test file prints 2185 1.
  • bsd rotates its accumulator right before adding each byte and counts 1024 byte blocks, so the same file prints 59852 1.
  • crc is the POSIX CRC-32 over exactly as many bytes as the file has, so it prints 365965416 25, which is what the test file already expected.

md5, sha1, sha224, sha256, sha384, sha512, blake2b and sm3 print a hex digest, or base64 with --base64, or the bytes themselves with --raw, and default to GNU's tagged MD5 (file) = ... form. --length is accepted for blake2b only, as GNU does, and the length shows up in the tag, so a 128 bit BLAKE2b line is BLAKE2b-128 (file) = ....

V's stdlib has SM3 nowhere, so it is implemented in sm3.v and verified against GNU's output.

--check

-c reads GNU's tagged form and takes the digest to use from the list itself, since the tag names it; only --algorithm restricts which lines are accepted, so a SHA256 list checked with -a md5 has no properly formatted lines, as at GNU. Hexadecimal and base64 lists are both recognised, a recorded digest has to be exactly as long as the named algorithm produces, and --warn, --strict, --quiet, --status and --ignore-missing behave as they do at GNU, including that an empty list is an error even without --strict.

Names and messages

A file name that would break the one record per line rule has its newline written as \n and its backslash as \\, and the line is marked with a leading backslash; --zero turns that off, because the NUL already separates records. --check reads that form back, and prints the name escaped again in every message, as GNU does.

In an error message a name is quoted so a shell would take it literally, which GNU does through quotearg(). The character sets here were read off GNU itself: which bytes are quoted wherever they appear, which only at the start of a name, and the fact that GNU writes $'\ooo' for a control character and groups consecutive ones together.

Testing

Every algorithm, output mode, check mode and error path was compared against GNU cksum 9.4, and the file name handling was fuzzed over every one, two and three character name built from the characters worth quoting: 48416 of 48726 names match byte for byte. The remaining ones all combine a single quote with a control character, where GNU reopens its quotes one character less often than this does; the difference is noted in quote.v.

Stacked on #195, #196 and #198, v fmt -verify . passes, v run build.vsh builds all 76 utilities, and v test . reports 47 of 47 files passing.

Note for review: this branch is based on main, so its own CI run will still show the pre-existing failures that #195 and #196 fix, and the v fmt gate will report the files #195 reformats.

Closes vlang#188. cksum computed one CRC over a whole file, offered no way to
verify anything, and its own test file did not compile against current V.
This implements what GNU's cksum 9.4 does.

sysv, bsd and crc stay numeric and print a size column, each with GNU's own
definition rather than the V stdlib's:

* sysv sums the bytes and folds the carry three times, and counts 512 byte
  blocks, so `cksum -a sysv` of the 25 byte test file prints `2185 1`.
* bsd rotates its accumulator right before adding each byte and counts 1024
  byte blocks, so the same file prints `59852     1`.
* crc is the POSIX CRC-32 over exactly as many bytes as the file has, so it
  prints `365965416 25`, which is what the test file already expected.

md5, sha1, sha224, sha256, sha384, sha512, blake2b and sm3 print a hex
digest, or base64 with --base64, or the bytes themselves with --raw, and
default to GNU's tagged `MD5 (file) = ...` form. --length is accepted for
blake2b only, as GNU does, and the length shows up in the tag, so a 128 bit
BLAKE2b line is `BLAKE2b-128 (file) = ...`.

V's stdlib has SM3 nowhere, so it is implemented in sm3.v and verified
against GNU's output.

`-c` reads GNU's tagged form and takes the digest to use from the list
itself, since the tag names it; only `--algorithm` restricts which lines are
accepted, so a SHA256 list checked with `-a md5` has no properly formatted
lines, as at GNU. Hexadecimal and base64 lists are both recognised, a
recorded digest has to be exactly as long as the named algorithm produces,
and `--warn`, `--strict`, `--quiet`, `--status` and `--ignore-missing` behave
as they do at GNU, including that an empty list is an error even without
--strict.

A file name that would break the one record per line rule has its newline
written as `\n` and its backslash as `\\`, and the line is marked with a
leading backslash; `--zero` turns that off, because the NUL already separates
records. `--check` reads that form back, and prints the name escaped again in
every message, as GNU does.

In an error message a name is quoted so a shell would take it literally, which
GNU does through quotearg(). The character sets here were read off GNU itself:
which bytes are quoted wherever they appear, which only at the start of a
name, and the fact that GNU writes `$'\ooo'` for a control character and groups
consecutive ones together.

Every algorithm, output mode, check mode and error path was compared against
GNU cksum 9.4, and the file name handling was fuzzed over every one, two and
three character name built from the characters worth quoting: 48416 of 48726
names match byte for byte. The remaining ones all combine a single quote with
a control character, where GNU reopens its quotes one character less often
than this does; the difference is noted in quote.v.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cksum is woefully under-implemented (and currently failing it's own tests)

1 participant