sort: implement GNU's orderings, keys, check mode and file handling - #200
Merged
Merged
Conversation
Closes vlang#192. `sort` sorted each input file on its own and concatenated the results, offered five orderings of the eleven GNU has, had no working `--stable`, and reported a missing file as `fn (string) IError`. This implements what GNU's sort 9.4 does. `sort f1 f2` sorts the inputs together. The old code sorted each file separately and appended them, so `sort f2 f1` could report an order the files did not have. `-m` is the option that means "each input is sorted already", and it now merges the streams without sorting them, so a line out of place in its own file stays there. Reading also changed: stdin no longer goes through a temporary file, `-z` makes the NUL the record separator, and `--files0-from` reads the list of names. The line number a disorder is reported at is counted during reading rather than by comparing the sorted output with the input. `-g`, `-h`, `-M`, `-V` and `--sort=WORD` were missing, and the one `-n` had was GNU's general numeric one. Each is now a separate grammar, because GNU's are not interchangeable: * `-n` takes blanks, a minus and digits. A leading `+` weighs nothing, so `+5` sorts as zero. * `-g` takes what strtod does, including a leading `+`, an exponent and a hexadecimal float, and weighs a line with no number as minus infinity. The words `inf` and `nan` are recognised, since V's `atof64` returns 0 for both. * `-h` adds a size suffix, and takes an exponent only with a capital E. Its units are capital-only from P upwards: GNU weighs `1p` as one and `1P` as a petabyte. * `-M` weighs a month name and then the text after it, so `JAN` precedes `JANUARY`. * `-V` weighs digit runs as numbers. Its character order is not the byte order: sorting every printable character as a one character name shows that a dot and a tilde come first, then digits, then letters, then the rest — but only at the start of the key, since `ab` sorts before `a.b`. After equal numbers or equal months, GNU does not weigh the rest of the line; the last-resort comparison of the whole lines does that. `-s` turns the last resort off, and so does `-u`, which is why `-u -n` keeps the first line of a run of equal numbers in the order the lines arrived in. V's sort is not stable, so the position of each line is part of the comparison. `-k` took its ordering letters from only one place, and read a field as if `-t` were always in effect. A key is now read as `F[.C][OPTS][,F[.C][OPTS]]` with the letters allowed on either side of the comma, so `-k2n,2` and `-k2,2n` agree. Without `-t`, a field starts at the whitespace before it and the first field has none to start from, which is what makes a line of nothing but blanks have an empty second field. `-b` steps past that whitespace. A character position is the last character of the key counted from the start of its field, and a line shorter than that stops early. A field number of zero is rejected, as at GNU, rather than read as the end of the line. `-c` reports `FILE:LINE: disorder: LINE` and stops at the first one; `-C` says nothing. With `-u` a line whose key equals the one before it counts as a disorder, and `-c` takes only one input. The messages for a bad key, a multi-character `-t` and an extra operand with `-c` now match GNU's, including which of them get the advice about `--help` and which quote the name. A table of 43 cases and a randomised matrix of 413 invocations over six sets of awkward inputs were compared against GNU, across every ordering, key form, check mode and error path. Five seeds give 413 of 413 on three of them and 409 of 413 on the other two. The four that differ are all `-h` weighing a plain number of seven digits or more against a suffixed one: GNU puts `1000000` before `1K`, which no value for the K is consistent with, and the reasoning is in `number.v`. The tests were rewritten as well: they tested `sort(options)` and `parse_sort_key`, both of which are gone, and three of them asserted an order that GNU does not produce.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #192.
sortsorted each input file on its own and concatenated the results, offered five orderings of the eleven GNU has, had no working--stable, and reported a missing file asfn (string) IError. This implements what GNU's sort 9.4 does.One stream, not one sort per file
sort f1 f2sorts the inputs together. The old code sorted each file separately and appended them, sosort f2 f1could report an order the files did not have.-mis the option that means "each input is sorted already", and it now merges the streams without sorting them, so a line out of place in its own file stays there.Reading also changed: stdin no longer goes through a temporary file,
-zmakes the NUL the record separator, and--files0-fromreads the list of names. The line number a disorder is reported at is counted while reading rather than by comparing the sorted output with the input.The orderings
-g,-h,-M,-Vand--sort=WORDwere missing, and the one-nhad was GNU's general numeric one. Each is now a separate grammar, because GNU's are not interchangeable:-ntakes blanks, a minus and digits. A leading+weighs nothing, so+5sorts as zero.-gtakes whatstrtoddoes, including a leading+, an exponent and a hexadecimal float, and weighs a line with no number as minus infinity. The wordsinfandnanare recognised, since V'satof64returns 0 for both.-hadds a size suffix and takes an exponent only with a capital E. Its units are capital-only from P upwards: GNU weighs1pas one and1Pas a petabyte.-Mweighs a month name and then the text after it, soJANprecedesJANUARY.-Vweighs digit runs as numbers. Its character order is not the byte order: sorting every printable character as a one-character name shows a dot and a tilde first, then digits, then letters, then the rest — but only at the start of the key, sinceabsorts beforea.b.After equal numbers or equal months, GNU does not weigh the rest of the line; the last-resort comparison of the whole lines does that.
-sturns the last resort off, and so does-u, which is why-u -nkeeps the first line of a run of equal numbers in the order the lines arrived in. V's sort is not stable, so the position of each line is part of the comparison.Keys
-ktook its ordering letters from only one place and read a field as if-twere always in effect. A key is now read asF[.C][OPTS][,F[.C][OPTS]]with the letters allowed on either side of the comma, so-k2n,2and-k2,2nagree.Without
-t, a field starts at the whitespace before it, and the first field has none to start from — which is what makes a line of nothing but blanks have an empty second field.-bsteps past that whitespace. A character position is the last character of the key counted from the start of its field, and a line shorter than that stops early. A field number of zero is rejected, as at GNU, rather than read as the end of the line.Checking and messages
-creportsFILE:LINE: disorder: LINEand stops at the first one;-Csays nothing. With-ua line whose key equals the one before it counts as a disorder, and-ctakes only one input. The messages for a bad key, a multi-character-tand an extra operand with-cnow match GNU's, including which of them get the advice about--helpand which quote the name.Testing
A table of 43 cases and a randomised matrix of 413 invocations over six sets of awkward inputs were compared against GNU, across every ordering, key form, check mode and error path. Five seeds give 413 of 413 on three of them and 409 of 413 on the other two.
The four that differ are all
-hweighing a plain number of seven digits or more against a suffixed one: GNU puts1000000before1K, which no value for the K is consistent with, since it would have to be above 1e7 and at most 1024^4. The reasoning is innumber.v. Every other comparison of a suffix against a plain number agrees, including1Kagainst 1024 and2Kagainst 1M.The tests were rewritten as well: they tested
sort(options)andparse_sort_key, both of which are gone, and three of them asserted an order GNU does not produce.Stacked on #195, #196, #197 and #198,
v fmt -verify .passes,v run build.vshbuilds all 76 utilities, andv test .reports 47 of 47 files passing.Note for review: this branch is based on
main, so its own CI run will still show the pre-existing failures that #195 and #196 fix, and thev fmtgate will report the files #195 reformats.