[BUGFIX] Catch random gibberish that has digits mixed into it - #28
Open
svewap wants to merge 2 commits into
Open
[BUGFIX] Catch random gibberish that has digits mixed into it#28svewap wants to merge 2 commits into
svewap wants to merge 2 commits into
Conversation
A submission in which every field was salad with a digit or two dropped in
passed the entropy filter untouched on a live cancellation form:
Name ZYWVj7hyXv Vorname AmJj19D9Y5 Telefon 9KI0nB1YVM
Vertragsbezeichnung JuT8l9hsQJ E-Mail qsdixon@yahoo.com
gibberishShare() cut its tokens with /[^\p{L}]+/u, i.e. at every non-letter,
digits included. "ZYWVj7hyXv" therefore arrived as "ZYWVj" and "hyXv" - five
and four characters, both far below gibberishTokenLength - and the same for the
other three fields, so the share came out at 0.00. The combined-entropy band
could not help either: 5.074 bits per character sits comfortably inside the
permitted 1.8 to 5.8. The single digit in the middle was the whole trick, and
it is cheap for a generator to keep using.
Tokens are now cut at non-alphanumeric characters and judged by what they are
made of, because the letter-only conditions do not carry over:
- letters only: unchanged (normalized entropy, consonant run, case flip or
vowel ratio),
- letters and digits: alternation rhythm plus mid-token case flips. Entropy is
useless here since "AS-2024-1234" is as repetition-free as "ZYWVj7hyXv";
what separates them is that a person writes letters and digits in blocks and
keeps one case, while a generator alternates repeatedly and flips case
mid-token,
- digits only: never suspicious. This case has to be explicit -
longestConsonantRun() counts every non-vowel, so "4194840183" would score a
run of its full length and a vowel ratio of zero. The old tokenizer only
avoided rejecting phone numbers by discarding digits before it looked.
gibberishTokenLength drops from 12 to 8 in the same commit, because the obvious
next move for a bot that has just lost the digits is a ten-letter random name,
and at twelve that was not even looked at. The length was never where the
precision came from: measured against the hunspell de_DE list with
Build/Scripts/measure-entropy-spam-false-positives.php, added here so the
figures in the docblocks can be checked rather than believed, the flag rate is
0.15% of 219250 tokens at eight against 0.20% of 134095 at twelve. The
consonant-run condition carries it, and eight is where real surnames such as
"Schwerdt" (normalized entropy 1.0, no repeated letter at all) live - they pass
on their consonant run of four.
The three new thresholds are options like the rest, so a form that legitimately
receives machine-shaped identifiers can relax them.
Test corpus extended in both directions. The expensive one is the negative:
contract numbers, IBANs, customer numbers, course designations with a year,
phone numbers in three notations. A false positive on a cancellation form is a
missed deadline for the site owner, not a spam mail.
(cherry picked from commit 10928a9)
Separate commit so the entry can name the SHA of the change it describes - amending it into the code commit rewrites exactly that SHA.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port of the
release/v13fix (10928a97,c0dfbc55), which went in directly because a live site was affected.Why
A submission in which every field was salad with a digit or two dropped in passed the entropy filter untouched on a live cancellation form:
gibberishShare()cut its tokens with/[^\p{L}]+/u, i.e. at every non-letter, digits included.ZYWVj7hyXvtherefore arrived asZYWVjandhyXv— five and four characters, both far belowgibberishTokenLength— so the share came out at 0.00. The combined-entropy band could not help either: 5.074 bits per character sits comfortably inside the permitted 1.8–5.8.What changed
Tokens are cut at non-alphanumeric characters and judged by what they are made of, because the letter-only conditions do not carry over:
AS-2024-1234is as repetition-free asZYWVj7hyXv); what separates them is that a person writes letters and digits in blocks and keeps one case,longestConsonantRun()counts every non-vowel:4194840183would otherwise score a maximal run and a vowel ratio of zero.gibberishTokenLengthdrops 12 → 8, since a ten-letter random name was not even looked at before. Against the hunspell de_DE list the flag rate is 0.15 % of 219 250 tokens at eight versus 0.20 % of 134 095 at twelve — the consonant-run condition carries the precision, not the length.Build/Scripts/measure-entropy-spam-false-positives.phpis added so the figures in the docblocks can be checked rather than believed.Tests
SpamCorpusgainsmixedAlnumBotTokens(),legitimateAlnumSubmissions()andreportedDigitSeededSubmission(), so unit and functional tests assert against the same payloads. The expensive direction is the negative one: contract numbers, IBANs, customer numbers, course designations with a year, phone numbers in three notations. A false positive on a cancellation form is a missed deadline for the site owner, not a spam mail.398 unit tests green locally; functional tests left to CI (local PHP is 8.5, see
CLAUDE.md).