Skip to content

Add route B, a pandoc filter per folder, and compare it with route A - #35

Merged
peterbjohnson merged 6 commits into
mainfrom
wb/t37
Sep 23, 2026
Merged

peterbjohnson merged 6 commits into
mainfrom
wb/t37

Conversation

@peterbjohnson

@peterbjohnson peterbjohnson commented Sep 23, 2026

Copy link
Copy Markdown
Member

Every ticket writes its tests before its code, and its pull request reports a run over a target set or a corpus folder with the number of flagged fields. See docs/plan.md.

Today the convert function converts one document through route A, the direct model call.
It runs route B, the pandoc filter, only when a filter file is supplied by hand. This ticket
adds the folder case. For a folder, the tool pairs each sheet with its solutions file by
name (for example Sheet1.tex with Sheet1_solutions.tex, or Sheet1.pdf with
Sheet1_Sol.pdf, ignoring case). The model writes one filter from the first sheet in the
folder, and pandoc runs that filter over every sheet. The model also converts each sheet
directly. For each sheet, the tool compares the two results field by field, sends the
disputed fields to the adjudication call, and writes one line of report: the number of
fields, the number agreed without a call, the number the adjudicator decided, and the number
flagged for a person. The ticket is done when the nine PHYS sheets, with their solutions
files, have run live and the pull request shows the report.


Workbench ticket t37.

Live run over the nine PHYS40002 pairs (manager, 2026-09-23 01:20)

pairs: 9, with a solutions file: 9
   mechanics_23-24_PS1.tex <-> mechanics_23-24_PS1_solutions.tex
   mechanics_23-24_PS2.tex <-> mechanics_23-24_PS2_solutions.tex
   mechanics_23-24_PS3.tex <-> mechanics_23-24_PS3_solutions.tex
   mechanics_23-24_PS4.tex <-> mechanics_23-24_PS4_solutions.tex
   mechanics_23-24_PS5.tex <-> mechanics_23-24_PS5_solutions.tex
   mechanics_23-24_PS6.tex <-> mechanics_23-24_PS6_solutions.tex
   mechanics_23-24_PS7.tex <-> mechanics_23-24_PS7_solutions.tex
   mechanics_23-24_PS8.tex <-> mechanics_23-24_PS8_solutions.tex
   mechanics_23-24_PS9.tex <-> mechanics_23-24_PS9_solutions.tex
mechanics_23-24_PS1: 65 fields, agreed 45, defaulted 5, adjudicated 15, flagged 3
mechanics_23-24_PS2: 103 fields, agreed 54, defaulted 26, adjudicated 23, flagged 18
mechanics_23-24_PS3: 53 fields, agreed 42, defaulted 0, adjudicated 11, flagged 4
mechanics_23-24_PS4: 96 fields, agreed 74, defaulted 1, adjudicated 21, flagged 9
mechanics_23-24_PS5: 37 fields, agreed 28, defaulted 1, adjudicated 8, flagged 3
mechanics_23-24_PS6: 53 fields, agreed 38, defaulted 2, adjudicated 13, flagged 3
mechanics_23-24_PS7: 46 fields, agreed 33, defaulted 2, adjudicated 11, flagged 3
mechanics_23-24_PS8: 63 fields, agreed 43, defaulted 2, adjudicated 18, flagged 8
mechanics_23-24_PS9: 15 fields, agreed 12, defaulted 0, adjudicated 3, flagged 1
9 sheets: 531 fields, agreed 369, defaulted 39, adjudicated 123, flagged 52
tokens: 244962
route_b_error: none
  mechanics_23-24_PS1: answers filled 15, worked solutions filled 15, flags 3
  mechanics_23-24_PS2: answers filled 22, worked solutions filled 23, flags 18
  mechanics_23-24_PS3: answers filled 13, worked solutions filled 13, flags 4
  mechanics_23-24_PS4: answers filled 22, worked solutions filled 22, flags 9
  mechanics_23-24_PS5: answers filled 9, worked solutions filled 9, flags 3
  mechanics_23-24_PS6: answers filled 12, worked solutions filled 12, flags 3
  mechanics_23-24_PS7: answers filled 12, worked solutions filled 12, flags 3
  mechanics_23-24_PS8: answers filled 15, worked solutions filled 15, flags 8
  mechanics_23-24_PS9: answers filled 3, worked solutions filled 3, flags 1

@peterbjohnson
peterbjohnson merged commit 4638c90 into main Sep 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant