Repository navigation
Conversation
strip_comments ran one regular expression whose block-comment alternative (/\*.*?\*/ for C-style and SQL sources, <!--.*?--> for HTML) searches to the end of the text from every opener that is never closed. A file with many unclosed openers took quadratic time: 4 s for 64 KiB, about 20 minutes extrapolated to the 1 MiB per-file cap, all before the time budget is next checked. An opener can only close when the last closer in the text starts after it, so the scan now uses the full scanner before that point and the same scanner without its block-comment alternative after it, and tries a match only at characters where one can start. The output is unchanged (checked against the old implementation on 360,000 generated inputs and on the verifier's own sources); 1 MiB of unclosed openers now takes 0.1 s. All 70 copies of redundancy.py stay byte-identical. The change alters the tasks' content hashes, so it ships with the next task release.
LANGUAGES mapped .less, .sass and .scss to css but not .css, so plain stylesheets, a large share of a reproduced website, were skipped by the code-redundancy measurement. The css comment style was already defined. The measurement is reported beside the score and does not enter it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #61
Two changes to the code-redundancy measurement in
redundancy.py, which is reported beside the score ineval.json.Linear comment stripping
The block-comment alternatives (
/\*.*?\*/for C-style and SQL,<!--.*?-->for HTML) searched to the end of the text from every opener that never closes, so many unclosed openers took quadratic time.An opener can close only if the last closer in the text starts after it.
strip_commentsnow uses the full scanner before that point and the same scanner without its block-comment alternative after it, and attempts a match only at characters where one can start./*/*(the per-file cap)The output is identical to the old implementation on 360,000 generated inputs across all comment styles and on the verifier's sources.
Plain
.cssLANGUAGESmapped.less,.sassand.scsstocssbut not.css, so plain stylesheets were not measured. The second commit adds it.Release
All 70 copies of
redundancy.pystay byte-identical. The change alters the tasks' content hashes, so it ships with the next task release.Tests
tests/test_redundancy.pyloads the module fromtasks/medium: comment and string handling, unclosed openers left as code, a timing bound on 128 KiB of unclosed openers (the old code takes 15 s, over the 2 s bound), and a duplicated.cssfile pair measured at redundancy 0.5 (nocssentry before the change).uv run python -m unittest discover -s testsuv run ruff check scripts src testsuv run ruff format --check scripts src testsuv run python scripts/check_tasks.py --code-only