Conversation
Closes vlang#188. cksum computed one CRC over a whole file, offered no way to verify anything, and its own test file did not compile against current V. This implements what GNU's cksum 9.4 does. sysv, bsd and crc stay numeric and print a size column, each with GNU's own definition rather than the V stdlib's: * sysv sums the bytes and folds the carry three times, and counts 512 byte blocks, so `cksum -a sysv` of the 25 byte test file prints `2185 1`. * bsd rotates its accumulator right before adding each byte and counts 1024 byte blocks, so the same file prints `59852 1`. * crc is the POSIX CRC-32 over exactly as many bytes as the file has, so it prints `365965416 25`, which is what the test file already expected. md5, sha1, sha224, sha256, sha384, sha512, blake2b and sm3 print a hex digest, or base64 with --base64, or the bytes themselves with --raw, and default to GNU's tagged `MD5 (file) = ...` form. --length is accepted for blake2b only, as GNU does, and the length shows up in the tag, so a 128 bit BLAKE2b line is `BLAKE2b-128 (file) = ...`. V's stdlib has SM3 nowhere, so it is implemented in sm3.v and verified against GNU's output. `-c` reads GNU's tagged form and takes the digest to use from the list itself, since the tag names it; only `--algorithm` restricts which lines are accepted, so a SHA256 list checked with `-a md5` has no properly formatted lines, as at GNU. Hexadecimal and base64 lists are both recognised, a recorded digest has to be exactly as long as the named algorithm produces, and `--warn`, `--strict`, `--quiet`, `--status` and `--ignore-missing` behave as they do at GNU, including that an empty list is an error even without --strict. A file name that would break the one record per line rule has its newline written as `\n` and its backslash as `\\`, and the line is marked with a leading backslash; `--zero` turns that off, because the NUL already separates records. `--check` reads that form back, and prints the name escaped again in every message, as GNU does. In an error message a name is quoted so a shell would take it literally, which GNU does through quotearg(). The character sets here were read off GNU itself: which bytes are quoted wherever they appear, which only at the start of a name, and the fact that GNU writes `$'\ooo'` for a control character and groups consecutive ones together. Every algorithm, output mode, check mode and error path was compared against GNU cksum 9.4, and the file name handling was fuzzed over every one, two and three character name built from the characters worth quoting: 48416 of 48726 names match byte for byte. The remaining ones all combine a single quote with a control character, where GNU reopens its quotes one character less often than this does; the difference is noted in quote.v.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #188.
cksumcomputed one CRC over a whole file, had no way to verify anything, and its own test file did not compile against current V. This implements what GNU's cksum 9.4 does.The digests
sysv,bsdandcrcstay numeric and print a size column, each with GNU's own definition rather than the V stdlib's:sysvsums the bytes and folds the carry three times, and counts 512 byte blocks, socksum -a sysvof the 25 byte test file prints2185 1.bsdrotates its accumulator right before adding each byte and counts 1024 byte blocks, so the same file prints59852 1.crcis the POSIX CRC-32 over exactly as many bytes as the file has, so it prints365965416 25, which is what the test file already expected.md5,sha1,sha224,sha256,sha384,sha512,blake2bandsm3print a hex digest, or base64 with--base64, or the bytes themselves with--raw, and default to GNU's taggedMD5 (file) = ...form.--lengthis accepted forblake2bonly, as GNU does, and the length shows up in the tag, so a 128 bit BLAKE2b line isBLAKE2b-128 (file) = ....V's stdlib has SM3 nowhere, so it is implemented in
sm3.vand verified against GNU's output.--check-creads GNU's tagged form and takes the digest to use from the list itself, since the tag names it; only--algorithmrestricts which lines are accepted, so a SHA256 list checked with-a md5has no properly formatted lines, as at GNU. Hexadecimal and base64 lists are both recognised, a recorded digest has to be exactly as long as the named algorithm produces, and--warn,--strict,--quiet,--statusand--ignore-missingbehave as they do at GNU, including that an empty list is an error even without--strict.Names and messages
A file name that would break the one record per line rule has its newline written as
\nand its backslash as\\, and the line is marked with a leading backslash;--zeroturns that off, because the NUL already separates records.--checkreads that form back, and prints the name escaped again in every message, as GNU does.In an error message a name is quoted so a shell would take it literally, which GNU does through
quotearg(). The character sets here were read off GNU itself: which bytes are quoted wherever they appear, which only at the start of a name, and the fact that GNU writes$'\ooo'for a control character and groups consecutive ones together.Testing
Every algorithm, output mode, check mode and error path was compared against GNU cksum 9.4, and the file name handling was fuzzed over every one, two and three character name built from the characters worth quoting: 48416 of 48726 names match byte for byte. The remaining ones all combine a single quote with a control character, where GNU reopens its quotes one character less often than this does; the difference is noted in
quote.v.Stacked on #195, #196 and #198,
v fmt -verify .passes,v run build.vshbuilds all 76 utilities, andv test .reports 47 of 47 files passing.Note for review: this branch is based on
main, so its own CI run will still show the pre-existing failures that #195 and #196 fix, and thev fmtgate will report the files #195 reformats.