Skip to content

Leaner generated validation code (~1.5x on object-heavy schemas), no behavior change - #233

Closed
John-Cusack wants to merge 1 commit into
horejsek:masterfrom
John-Cusack:tier1-generated-code-shapes
Closed

John-Cusack wants to merge 1 commit into
horejsek:masterfrom
John-Cusack:tier1-generated-code-shapes

Conversation

@John-Cusack

Copy link
Copy Markdown

Leaner generated validation code (~1.5x on object-heavy schemas), no behavior change

What

Three pure-Python codegen-shape changes to the generated validate
functions (fastjsonschema/draft04.py only — same shapes apply to
draft06/draft07 if you want them; kept this PR to draft04 to
stay reviewable):

  1. required without set ops (generate_required): emit
    [key for key in [...] if key not in data] instead of
    set([...]) - data.keys(). Same exception type, message
    (str(sorted(missing))), definition, and rule; the set
    construction moves off the valid path.
  2. no dead _keys set (generate_properties /
    generate_pattern_properties / generate_additional_properties):
    skip building, probing (in), and pruning (.remove) the
    {variable}_keys set when no consumer exists at that object level
    (additionalProperties absent/True/{} and no
    propertyNames: false); direct if "key" in {variable} guards
    instead.
  3. enum tuple membership (generate_enum): for all-string enums
    emit if x not in ('a', 'b', ...) instead of the per-option
    isinstance-and-== or-chain. Same message, definition, rule.

Correctness guard included: trivially-checkless schemas
(properties: {}, patternProperties: {}, vacuous
additionalProperties: true/{}) previously relied on the
unconditional _keys line to keep the generated if block non-empty;
with it gone, those shapes emitted an empty if body
(IndentationError at compile). The trivial returns are now hoisted
above the block so no empty if is ever generated (verified: those
schemas compile and validate correctly on this branch).

Measured

Ryzen 9 5950X, CPython 3.12.3, frozen 3000-doc nested-order batch
(compile-once excluded by construction), each arm a fresh subprocess,
in-process timing after warmup, min-of-7 per arm with arms interleaved
round-robin (shared box), peak RSS via /proc VmHWM:

Track Stock (master @ 738da70) Patched Speedup
rep (half-invalid) 0.3355 s 0.2262 s 1.48x
valid-only 0.2942 s 0.1697 s 1.73x
x1 (single small doc) 0.0142 s 0.0074 s 1.92x

Ratios stable across two full interleaved runs (rep 1.48x both runs;
valid 1.62–1.73x; x1 1.88–1.92x). Peak RSS flat across arms (~23 MB
rep/valid, ~17.6 MB x1 both sides). Outcome checksums (sha256 over
valid/invalid outcomes) and error samples (message, name, rule,
definition on the first 40 docs) agree stock-vs-patched on all
tracks.

Reproduce (at 738da70 vs this branch, venv per Makefile
pip install -e .[devel] + the JSON-Schema-Test-Suite submodule):

python bench_fjs.py rep   # + valid, x1; repeat per arm, take min

(Bench scripts with the frozen corpus available on request — say the
word and I'll attach them.)

Test / lint proof

  • pytest -W default --benchmark-skip (their make test gate):
    4539 passed, 12 skipped, 435 xfailed, 950 xpassed — zero
    failures (includes the JSON-Schema-Test-Suite submodule cases).
  • pylint fastjsonschema: 9.90/10, byte-identical score and warning
    set to unmodified master (no new warnings from this change).
  • CHANGELOG.txt entry added under === UNRELEASED, matching
    contributor practice.

required without set ops, no dead _keys set when nothing consumes
it, tuple membership for all-string enums. Same exception types,
messages, definitions and rules; full suite green, checksums and
error samples identical stock-vs-patched on the frozen workload.
@John-Cusack

Copy link
Copy Markdown
Author

Closing this. The title and changelog say no behavior change, which isn't true: {"properties": []} now compiles instead of raising AttributeError, and an all-string enum now accepts a non-string object whose __eq__ matches one of the strings. On the repo's performance.py schema it is also only about 6% faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant