diff --git a/docs/plan.md b/docs/plan.md new file mode 100644 index 0000000..8d343e1 --- /dev/null +++ b/docs/plan.md @@ -0,0 +1,125 @@ +# Plan: two routes from a document to a set + +Date: 2026-09-21. This plan replaces the spec engine, the draft commands and the fixing +loop on the agent's path. That code stays in the in2lambda package and is not used by the +agent. + +## Goal + +The agent converts one document (PDF, tex, docx or markdown), with its solutions document +where one exists, into a Lambda Feedback set. The set must match the set the author would +enter by hand. The measure is a comparison with exported sets: the ME2 introduction pair +first, then sets the maintainer has validated on the platform. + +## The route of one document + +```mermaid +flowchart TD + D[Document
PDF, tex, docx, md] --> C + S[Solutions document
where one exists] --> C + C["1. Conversion to markdown
Mathpix for PDF (cached), pandoc for tex and docx"] + C --> A["2. Route A: direct call
one model call returns the set as JSON
(title, main text, parts: content, options, answer, worked solution)"] + C --> B["3. Route B: filter for the set
one model call per folder writes a Lua filter;
pandoc runs it over every document, no further call"] + A --> V["4. Verbatim check
every paragraph of every field must occur in the markdown"] + B --> V + V --> T1["5. Tier 1: mechanical agreement
fields compared after folding whitespace and notation
agreed fields are accepted"] + T1 -->|disputed fields only| T2["6. Tier 2: adjudication
one small model call: both versions and their source lines
returns take A, take B, same, or needs a person"] + T1 -->|agreed fields| SET + T2 -->|take A, take B, same| SET + T2 -->|needs a person| T3 + V -->|not verbatim| T3 + T1 -->|question or part found by one route only| T3 + T3["7. Tier 3: flags
the list a person reads: field, both versions, reason"] + SET["8. The set
in2lambda's export model and zip writer;
KaTeX findings reported as warnings"] + SET --> CMP["9. Comparison
in2lambda compares the built set with the export where one exists"] + E[Exported set
where one exists] --> CMP + CMP --> R[Report
fields, agreed, adjudicated, flagged, not verbatim, tokens, differences] + T3 --> R +``` + +1. **Conversion to markdown.** Mathpix converts a PDF; the result is cached by the PDF's + hash. pandoc converts tex and docx. Both exist. +2. **Route A: the direct call.** One model call receives the markdown, and the solutions + markdown where one exists, and returns the set as JSON in the export's shape: for each + question, the title, the main text, and the parts, each with content, options, answer + and worked solution. The prompt instructs the model to copy text and never to write it. +3. **Route B: a filter written for the set.** For a folder of documents with one + structure, one model call receives an abbreviated view of pandoc's tree of the first + document and returns a Lua filter that emits the same JSON. pandoc runs the filter over + every document in the folder. No further model call is made. +4. **The verbatim check.** Every paragraph of every field from either route must occur in + the markdown the route read, after whitespace is folded. A field that fails is flagged. +5. **Tier 1: mechanical agreement.** The two replies are compared field by field after + folding whitespace and notation that renders identically: `\left(` and `(`, `~` and + `\,` and a space, `\mathrm{~m}` and `\mathrm{m}`. A field on which the replies agree + is accepted. +6. **Tier 2: adjudication.** One model call per document receives the disputed fields + only: both versions and the source lines each was quoted from. For each field the + call returns one of: same meaning, take A, take B, or needs a person, with a one-line + reason. The call may choose one of the two texts or a passage of the source. The + verbatim check runs on the chosen text, so the call cannot introduce its own words. +7. **Tier 3: flags.** The report lists the fields a person must read: fields the + adjudicator returned as "needs a person", fields that failed the verbatim check, and + questions or parts that one route found and the other did not. Each flag shows both + versions and the reason. +8. **The set.** in2lambda's export model and zip writer write the set. KaTeX findings on + the maths are reported as warnings. No check blocks the write. +9. **The comparison.** Where an exported set exists for the document, in2lambda's + comparison reports the differences between the built set and the export. + +## What the agent keeps and what it sets aside + +Kept: the OCR and its cache; the model backends; in2lambda's export model, zip writer and +comparison; the corpus sweep and its results table, re-pointed at this route; the web page, +re-pointed at this route. + +Set aside on the agent's path: the YAML spec and its layouts; the draft with its blocks, +layers and command log; the fixing loop; the coverage check, the delimiter check and the +xelatex compile as conditions for writing a set. The in2lambda package keeps this code. + +## What the tiers detect + +The verbatim check detects a route inventing or rewording text. The structural comparison +detects a question or part that one route missed. The adjudication resolves wording the +two routes read differently. + +The tiers do not detect an error in the markdown. Both routes read the same OCR output, so +a word Mathpix misread, or a separator line Mathpix read as a minus sign, passes every +tier. The comparison with an exported set, or a reader, detects those. An OCR check is +separate work. + +## Response areas + +The built set has no response areas: the answer boxes and their marking are not on the +sheet, and choosing them requires judgement. A model call per part, given the part's +content, options and final answer, can propose them, and the ME2 export gives 13 to +measure against. This is deferred. + +## Cost + +At Sonnet-class rates, about $3 per million input tokens and $15 per million output +tokens: the direct call costs about $0.17 per sheet with solutions and $0.06 for a +questions-only sheet; a filter costs about $0.05 per folder; an adjudication call costs +under a cent. One hundred sheets in ten folders through both routes cost $7 to $18. + +## Results so far + +The ME2 pair, live: 5 questions, 60 fields, 0 flags, titles equal to the export's, one +model call, 66 seconds with the OCR cached. The PHYS40002 folder, route B: one filter +written from the first sheet in 41 seconds; pandoc converted all 9 sheets, 65 questions, +467 fields, 1 field flagged. + +## Order of work + +1. Run both routes with the solutions files over the PHYS40002 folder and report the flag + counts. +2. Run one PDF from the UCL_MechEng folder. +3. Review the flags with the maintainer. +4. Re-point the sweep, the page and the gate at this route, as tickets the maintainer has + reviewed (docs/tickets-draft.md). + +## Measures + +Per document: fields returned, fields agreed at tier 1, fields adjudicated, fields flagged, +fields not verbatim, tokens used; and the comparison table where an export exists. Per +folder: the totals of the same. diff --git a/docs/route.html b/docs/route.html new file mode 100644 index 0000000..4b3de20 --- /dev/null +++ b/docs/route.html @@ -0,0 +1,33 @@ + +Route of one document + + +

The route of one document

+
+flowchart TD
+    D[Document
PDF, tex, docx, md] --> C + S[Solutions document
where one exists] --> C + C["1. Conversion to markdown
Mathpix for PDF (cached), pandoc for tex and docx"] + C --> A["2. Route A: direct call
one model call returns the set as JSON
(title, main text, parts: content, options, answer, worked solution)"] + C --> B["3. Route B: filter for the set
one model call per folder writes a Lua filter;
pandoc runs it over every document, no further call"] + A --> V["4. Verbatim check
every paragraph of every field must occur in the markdown"] + B --> V + V --> T1["5. Tier 1: mechanical agreement
fields compared after folding whitespace and notation
agreed fields are accepted"] + T1 -->|disputed fields only| T2["6. Tier 2: adjudication
one small model call: both versions and their source lines
returns take A, take B, same, or needs a person"] + T1 -->|agreed fields| SET + T2 -->|take A, take B, same| SET + T2 -->|needs a person| T3 + V -->|not verbatim| T3 + T1 -->|question or part found by one route only| T3 + T3["7. Tier 3: flags
the list a person reads: field, both versions, reason"] + SET["8. The set
in2lambda's export model and zip writer;
KaTeX findings reported as warnings"] + SET --> CMP["9. Comparison
in2lambda compares the built set with the export where one exists"] + E[Exported set
where one exists] --> CMP + CMP --> R[Report
fields, agreed, adjudicated, flagged, not verbatim, tokens, differences] + T3 --> R +
+

Source: docs/route.mmd, also embedded in docs/plan.md.

+ \ No newline at end of file diff --git a/docs/route.mmd b/docs/route.mmd new file mode 100644 index 0000000..0bba533 --- /dev/null +++ b/docs/route.mmd @@ -0,0 +1,21 @@ +flowchart TD + D[Document
PDF, tex, docx, md] --> C + S[Solutions document
where one exists] --> C + C["1. Conversion to markdown
Mathpix for PDF (cached), pandoc for tex and docx"] + C --> A["2. Route A: direct call
one model call returns the set as JSON
(title, main text, parts: content, options, answer, worked solution)"] + C --> B["3. Route B: filter for the set
one model call per folder writes a Lua filter;
pandoc runs it over every document, no further call"] + A --> V["4. Verbatim check
every paragraph of every field must occur in the markdown"] + B --> V + V --> T1["5. Tier 1: mechanical agreement
fields compared after folding whitespace and notation
agreed fields are accepted"] + T1 -->|disputed fields only| T2["6. Tier 2: adjudication
one small model call: both versions and their source lines
returns take A, take B, same, or needs a person"] + T1 -->|agreed fields| SET + T2 -->|take A, take B, same| SET + T2 -->|needs a person| T3 + V -->|not verbatim| T3 + T1 -->|question or part found by one route only| T3 + T3["7. Tier 3: flags
the list a person reads: field, both versions, reason"] + SET["8. The set
in2lambda's export model and zip writer;
KaTeX findings reported as warnings"] + SET --> CMP["9. Comparison
in2lambda compares the built set with the export where one exists"] + E[Exported set
where one exists] --> CMP + CMP --> R[Report
fields, agreed, adjudicated, flagged, not verbatim, tokens, differences] + T3 --> R diff --git a/docs/tickets-draft.md b/docs/tickets-draft.md new file mode 100644 index 0000000..00b4543 --- /dev/null +++ b/docs/tickets-draft.md @@ -0,0 +1,77 @@ +# Ticket drafts, for review. None of these is on the board. + +Every ticket writes its tests before its code. Every ticket's pull request reports a run over +a target set or a corpus folder, with the number of flagged fields. + +## 1. Convert a whole folder of sheets through both routes and compare the two results + +Today the `convert` function converts one document through route A, the direct model call. +It runs route B, the pandoc filter, only when a filter file is supplied by hand. This ticket +adds the folder case. For a folder, the tool pairs each sheet with its solutions file by +name (for example `Sheet1.tex` with `Sheet1_solutions.tex`, or `Sheet1.pdf` with +`Sheet1_Sol.pdf`, ignoring case). The model writes one filter from the first sheet in the +folder, and pandoc runs that filter over every sheet. The model also converts each sheet +directly. For each sheet, the tool compares the two results field by field, sends the +disputed fields to the adjudication call, and writes one line of report: the number of +fields, the number agreed without a call, the number the adjudicator decided, and the number +flagged for a person. The ticket is done when the nine PHYS sheets, with their solutions +files, have run live and the pull request shows the report. + +## 2. Add a `convert` command to the command line + +The command is `in2lambda-agent convert DOCUMENT [--solutions FILE] [--filter FILE | +--write-filter] [--out DIRECTORY]`. It prints the report and writes the zip. The existing +`run` command keeps the spec route behind a flag, `--route spec`, and the README describes +`convert` as the way to convert a document. The ticket is done when the ME2 pair converts +from the command line with no flagged field. + +## 3. Point the corpus sweep at the new route + +The `corpus` command runs the folder conversion of ticket 1 over each set in the corpus and +writes `results.csv` with one row per sheet and these columns: set, sheet, questions, parts, +fields, agreed, adjudicated, flagged, not verbatim, tokens, seconds, and a reason where the +sheet produced no set. The ticket is done when the three ExampleContents folders have been +swept and the pull request shows the table. + +## 4. Compare each target document with the set Lambda Feedback exported from it + +A target is a folder holding one set: one questions document, an optional solutions document +whose name ends in `_solutions`, and the folder Lambda Feedback exported for that set, named +`set_`. Targets live under `ExampleContents/targets/`, either directly +(`targets/ME2_Fluids_introduction/`) or grouped by course +(`targets/EART40013_Mathematical_Methods_II/CW1/`, `.../CW2/`). The tool finds a target by +its `set_*` folder. For each target, the tool converts the documents and compares the built +set with the export through in2lambda's comparison function. Differences the maintainer has +accepted are listed in a file beside the target's saved filter. The ticket is done when the +ME2 target and the two EART40013 targets report their known differences and no other. + +## 5. Point the web page at the new route + +The page runs the `convert` command, shows each stage line as it happens, and shows each +flagged field with both versions and the reason. The ticket is done when the ME2 pair runs +from the page. + +## 6. Fix two faults in the input markdown + +First, pandoc writes an underlined run in a docx file as `[text]{.underline}`, which Lambda +Feedback does not render. The tool converts docx files with pandoc's `bracketed_spans` +extension switched off, so the text is written without the brackets. Second, when Mathpix +reads a PDF that Lambda Feedback printed, it reads the horizontal separator lines as minus +signs at the start or end of the neighbouring maths. The tool flags a display maths that +begins or ends with a lone minus sign. The ticket is done when the MECH docx sheets carry no +bracketed span and the ME2 solutions report the fields with a stray minus sign. + +## 7. Remove the spec route from the agent + +After tickets 1 to 5 are merged, delete the spec writer, the fixing loop and the wrapper +around the draft commands from the agent, together with their tests. The in2lambda package +keeps its spec engine unchanged. The ticket is done when the agent's tests pass and +`in2lambda-agent --help` lists `convert`, `corpus`, `gate`, `compare` and `ui`. + +## 8. Create response areas (deferred, not scheduled) + +The built set has no response areas, so an imported part has no answer box. A model call +per part could propose them: the kind of box, the text before it, the answer in the +platform's machine form, and the correct options for a multiple-choice part. in2lambda's +`ResponseArea` class writes all three kinds. The ME2 export holds 13 response areas to +measure against. diff --git a/in2lambda_agent/routes.py b/in2lambda_agent/routes.py new file mode 100644 index 0000000..a2f657e --- /dev/null +++ b/in2lambda_agent/routes.py @@ -0,0 +1,385 @@ +"""Two routes from a document to a set, and review only where they differ. + +Route A, `direct`: one model call reads the markdown and answers with the set in the +export's shape. Route B, `run_filter`: a Lua filter the model wrote for the set's +structure, run by pandoc with no model call. Every field either route returns must be a +quote of the markdown (`not_verbatim`). The two replies are compared field by field +(`disputed`); a disputed field goes to a small second call that may pick one side or a +passage of the source, never its own words (`adjudicate`); what neither settles is a flag +for a person (`reconcile`). `to_set` and `build` write the result with in2lambda. + +A reply is a list of questions: {"title", "main_text", "parts": [{"content", +"options", "answer", "worked_solution"}]}. Field keys are 1-based: `q2.p1.content`. +""" + +from __future__ import annotations + +import json +import re +import subprocess +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Optional + +from in2lambda.api.part import Part +from in2lambda.api.question import Question +from in2lambda.api.set import Set + +from in2lambda_agent.model import Backend, Reply, choose_backend +from in2lambda_agent.settings import Settings, load_settings + +Reply_ = list[dict[str, Any]] + +TEXT_FIELDS = ("content", "answer", "worked_solution") + +_FOLDS = ( + ("\\left(", "("), ("\\right)", ")"), ("\\left[", "["), ("\\right]", "]"), + ("\\mathrm{~", "\\mathrm{"), ("\\text {", "\\text{"), ("\\space", " "), + ("\\,", " "), ("\;", " "), ("~", " "), (" ", " "), +) +_IMAGE = re.compile(r"!\[[^\]]*\]\(([^)\s]+)[^)]*\)") +_RULE = re.compile(r"^\s*(-{3,}|\*{3,})\s*$", re.M) + + +def fold(text: str) -> str: + """Text as compared: notation that renders the same reads the same.""" + text = _RULE.sub("", text or "") + # An image is compared as "an image here": the export renames every media file. + text = re.sub(r"!\[[^\]]*\]\([^)]*\)(\{[^}]*\})?", "![img]", text) + for old, new in _FOLDS: + text = text.replace(old, new) + text = re.sub(r"\s*([=+\-*/,.:;()\[\]{}^_])\s*", r"\1", text) + return " ".join(text.split()) + + +def _squash(text: str) -> str: + return " ".join((text or "").split()) + + +def fields(reply: Reply_) -> dict[str, str]: + """Every text field of a reply by key, options included.""" + found: dict[str, str] = {} + for i, q in enumerate(reply, 1): + found[f"q{i}.title"] = q.get("title", "") + found[f"q{i}.main_text"] = q.get("main_text", "") + for j, p in enumerate(q.get("parts", []), 1): + for name in TEXT_FIELDS: + found[f"q{i}.p{j}.{name}"] = p.get(name, "") or "" + for k, option in enumerate(p.get("options", []) or [], 1): + found[f"q{i}.p{j}.options[{k}]"] = option + return found + + +def not_verbatim(reply: Reply_, source: str) -> list[str]: + """The fields that are not quotes of the source; titles are not quotes. + + A field may be several paragraphs quoted from different places - a question's text + before and after its parts - so each paragraph is looked for on its own. + """ + haystack = _squash(source) + found = [] + for key, text in fields(reply).items(): + if key.endswith(".title"): + continue + paragraphs = [_squash(p) for p in re.split(r"\n\s*\n", text or "") if _squash(p)] + if any(p not in haystack for p in paragraphs): + found.append(key) + return found + + +def disputed(a: Reply_, b: Reply_) -> list[str]: + """Where two replies differ: a question or part one lacks, or a field worded differently.""" + found: list[str] = [] + for i in range(1, max(len(a), len(b)) + 1): + if i > len(a) or i > len(b): + found.append(f"q{i}") + continue + pa, pb = a[i - 1].get("parts", []), b[i - 1].get("parts", []) + for j in range(1, max(len(pa), len(pb)) + 1): + if j > len(pa) or j > len(pb): + found.append(f"q{i}.p{j}") + fa, fb = fields(a), fields(b) + for key in fa: + if key in fb and fold(fa[key]) != fold(fb[key]) and not any(key.startswith(s + ".") for s in found): + found.append(key) + return found + + +def to_set(reply: Reply_, name: str = "set", directory: Optional[Path] = None) -> Set: + """The reply as in2lambda's Set. Images named in the texts are attached where they exist.""" + built = Set(_name=name) + for q in reply: + question = Question(title=q.get("title", ""), main_text=q.get("main_text", "")) + for p in q.get("parts", []): + question.parts.append( + Part(text=p.get("content", "") or "", worked_solution=p.get("worked_solution", "") or "", answer=p.get("answer", "") or "") + ) + if directory is not None: + for text in [question.main_text] + [t for p in question.parts for t in (p.text, p.worked_solution, p.answer)]: + for ref in _IMAGE.findall(text): + path = Path(directory) / ref + if path.is_file() and str(path) not in question.images: + question.images.append(str(path)) + built.questions.append(question) + return built + + +def build(built: Set, out_dir: Path) -> Path: + """Writes the set's folder and zip under out_dir; returns the zip.""" + Path(out_dir).mkdir(parents=True, exist_ok=True) + built.to_json(str(out_dir)) + # in2lambda names the folder and the zip after the set. + return Path(out_dir) / f"{built._name}.zip" + + +# --- route A ------------------------------------------------------------------------- + +SYSTEM = ( + "You convert a problem sheet, and its solutions where given, from markdown into a JSON " + "list of questions for a learning platform. Answer with JSON only, no prose, no code fence." +) + + +def _prompt(markdown: str, solutions: Optional[str]) -> str: + return ( + "Return a JSON array, one object per question in order: " + '{"title": str, "main_text": str, "parts": [{"content": str, "options": [str], "answer": str, "worked_solution": str}]}. ' + "title: the question's name without its number, or an empty string where the sheet gives none. " + "main_text: the statement shared by all parts, copied verbatim from the QUESTIONS markdown, maths and image " + "references included; leave out timing lines, star ratings and sentences addressed to the student about the " + "course. parts: one per lettered or numbered sub-question; content is the part's own statement copied verbatim " + "without its label; a question with no sub-questions has one part whose content is empty. options holds the " + "choices of a multiple-choice part, verbatim, and those choices are then not in content; an empty list otherwise. " + "answer is the part's final answer and worked_solution its worked solution, each copied verbatim from the " + "SOLUTIONS markdown where there is one, or empty strings. Copy; never paraphrase, never invent.\n\n" + "QUESTIONS markdown:\n\n" + markdown + + ("\n\nSOLUTIONS markdown:\n\n" + solutions if solutions else "") + ) + + +def _json(text: str) -> Any: + return json.loads(re.sub(r"^```(json)?\s*|\s*```$", "", text.strip())) + + +def direct(markdown: str, solutions: Optional[str], backend: Backend) -> tuple[Reply_, Reply]: + """Route A: one call, the reply as a list of questions, and the call's usage.""" + reply = backend.call(SYSTEM, _prompt(markdown, solutions)) + return _json(reply.text), reply + + +# --- route B ------------------------------------------------------------------------- + + +def run_filter(lua: Path, document: Path) -> Reply_: + """Route B at run time: pandoc, the filter, and the JSON it wrote. No model.""" + out = subprocess.run( + ["pandoc", str(document), "--lua-filter", str(lua), "-t", "plain", "--wrap=none"], + capture_output=True, check=True, + ) + return json.loads(out.stdout.decode("utf-8").strip()) + + +# --- tiers 2 and 3 --------------------------------------------------------------------- + +ADJUDICATE = ( + "Two readings of a problem sheet disagree on some fields. For each field you are given reading A, reading B and " + "the lines of the source they were taken from. Answer with JSON only: a list of " + '{"field": str, "choice": "A"|"B"|"text"|"person", "text": str, "reason": str}. ' + "Choose A or B where one is the faithful copy of the source; choose text, with the exact passage of the source, " + "where neither is; choose person where the two disagree on what belongs in the field. Never write words of your own." +) + + +def _source_lines(source: str, *texts: str, around: int = 2) -> str: + lines = source.splitlines() + hits: set[int] = set() + for text in texts: + probe = _squash(text)[:40] + for n, line in enumerate(lines): + if probe and probe[:25] in _squash(line): + hits.update(range(max(0, n - around), min(len(lines), n + around + 1))) + return "\n".join(lines[n] for n in sorted(hits)) or "(not found)" + + +def adjudicate(a: Reply_, b: Reply_, keys: list[str], source: str, backend: Backend) -> dict[str, tuple[str, str]]: + """Tier 2: one small call over the disputed fields; a choice it may not make becomes 'person'.""" + fa, fb = fields(a), fields(b) + keys = [k for k in keys if k in fa and k in fb] + if not keys: + return {} + shown = "\n\n".join( + f"FIELD {k}\nA: {fa[k]}\nB: {fb[k]}\nSOURCE LINES:\n{_source_lines(source, fa[k], fb[k])}" for k in keys + ) + reply = backend.call(ADJUDICATE, shown) + verdicts: dict[str, tuple[str, str]] = {k: ("person", "no verdict") for k in keys} + for v in _json(reply.text): + k, choice, reason = v.get("field"), v.get("choice"), v.get("reason", "") + if k not in verdicts: + continue + if choice in ("A", "B"): + verdicts[k] = (choice, reason) + elif choice == "text" and _squash(v.get("text", "")) and _squash(v["text"]) in _squash(source): + verdicts[k] = ("text:" + v["text"], reason) + else: + verdicts[k] = ("person", reason or "the adjudicator's own words") + return verdicts + + +@dataclass +class Flag: + field: str + a: str + b: str + reason: str + + +@dataclass +class Reconciled: + fields: Reply_ + agreed: int + adjudicated: int + flags: list[Flag] = field(default_factory=list) + + +def _set_field(reply: Reply_, key: str, text: str) -> None: + m = re.fullmatch(r"q(\d+)(?:\.p(\d+))?\.(\w+)(?:\[(\d+)\])?", key) + q, p, name, k = m.group(1), m.group(2), m.group(3), m.group(4) + target = reply[int(q) - 1] if p is None else reply[int(q) - 1]["parts"][int(p) - 1] + if k is None: + target[name] = text + else: + target[name][int(k) - 1] = text + + +def reconcile(a: Reply_, b: Reply_, source: str, backend: Optional[Backend] = None) -> Reconciled: + """Tiers 1 to 3: agreed fields kept, disputes adjudicated, the rest flagged. Starts from A.""" + merged = json.loads(json.dumps(a)) + keys = disputed(a, b) + structural = [k for k in keys if re.fullmatch(r"q\d+(\.p\d+)?", k)] + wording = [k for k in keys if k not in structural] + fa, fb = fields(a), fields(b) + result = Reconciled(fields=merged, agreed=len(fa) - len(wording), adjudicated=len(wording)) + for k in structural: + result.flags.append(Flag(k, "present" if k in _structure(a) else "absent", "present" if k in _structure(b) else "absent", "one route did not find it")) + verdicts = adjudicate(a, b, wording, source, backend) if wording and backend is not None else {} + for k in wording: + choice, reason = verdicts.get(k, ("person", "not adjudicated")) + if choice == "B": + _set_field(merged, k, fb[k]) + elif choice.startswith("text:"): + _set_field(merged, k, choice[5:]) + elif choice == "person": + result.flags.append(Flag(k, fa[k], fb[k], reason)) + for k in not_verbatim(merged, source): + if not any(f.field == k for f in result.flags): + result.flags.append(Flag(k, fields(merged)[k], "", "not a quote of the source")) + return result + + +def _structure(reply: Reply_) -> set[str]: + return {f"q{i}" for i in range(1, len(reply) + 1)} | { + f"q{i}.p{j}" for i, q in enumerate(reply, 1) for j in range(1, len(q.get("parts", [])) + 1) + } + + +# --- one document end to end ----------------------------------------------------------- + + +@dataclass +class Converted: + set: Set + zip_path: Optional[Path] + flags: list[Flag] + reply: Reply_ + tokens: int = 0 + + +def markdown_of(document: Path, cache_dir: Path, settings: Settings) -> tuple[str, Path]: + """The document as markdown, and the folder its images are in.""" + document = Path(document) + if document.suffix.lower() == ".pdf": + from in2lambda_agent.mathpix import MathpixClient + from in2lambda_agent.ocr import ocr_pdf + + ocr = ocr_pdf(document, cache_dir=cache_dir, client=MathpixClient.from_settings(settings)) + return ocr.markdown.read_text(encoding="utf-8"), ocr.markdown.parent + if document.suffix.lower() in (".md", ".markdown"): + return document.read_text(encoding="utf-8"), document.parent + out = subprocess.run(["pandoc", str(document), "-t", "commonmark_x", "--wrap=none"], capture_output=True, check=True) + return out.stdout.decode("utf-8"), document.parent + + +def convert( + document: Path, + solutions: Optional[Path] = None, + *, + out_dir: Path = Path("out"), + cache_dir: Path = Path(".in2lambda-agent"), + backend: Optional[Backend] = None, + settings: Optional[Settings] = None, + lua: Optional[Path] = None, + name: str = "set", +) -> Converted: + """Route A, route B where a filter is given, reconcile, verify, write.""" + settings = settings or load_settings() + backend = backend or choose_backend(settings) + markdown, images = markdown_of(document, cache_dir, settings) + solutions_md = markdown_of(solutions, cache_dir, settings)[0] if solutions else None + source = markdown + ("\n" + solutions_md if solutions_md else "") + reply, usage = direct(markdown, solutions_md, backend) + tokens = usage.usage.input_tokens + usage.usage.output_tokens + if lua is not None: + other = run_filter(lua, document) + reconciled = reconcile(reply, other, source, backend) + reply, flags = reconciled.fields, reconciled.flags + else: + flags = [Flag(k, fields(reply)[k], "", "not a quote of the source") for k in not_verbatim(reply, source)] + built = to_set(reply, name=name, directory=images) + return Converted(set=built, zip_path=build(built, out_dir), flags=flags, reply=reply, tokens=tokens) + + +# --- route B: writing the filter ------------------------------------------------------- + +FILTER_SYSTEM = "You write pandoc Lua filters. Answer with the Lua source only, no prose, no code fence." + + +def structure(document: Path) -> str: + """Pandoc's tree of a document, abbreviated to one line per block, for the filter-writing call.""" + ast = json.loads(subprocess.check_output(["pandoc", str(document), "-t", "json"])) + + def text(inlines: list) -> str: + return "".join(i.get("c", "") if i["t"] == "Str" else " " if i["t"] == "Space" else "$" if i["t"] == "Math" else "" for i in inlines) + + def brief(b: dict, depth: int = 0) -> list[str]: + t, pad = b["t"], " " * depth + if t in ("Para", "Plain"): + return [f"{pad}{t}: {text(b['c'])[:70]}"] + if t == "Header": + return [f"{pad}Header({b['c'][0]}): {text(b['c'][2])[:70]}"] + if t in ("OrderedList", "BulletList"): + items = b["c"][1] if t == "OrderedList" else b["c"] + out = [f"{pad}{t} with {len(items)} items"] + for item in items[:40]: + out.append(f"{pad} item:") + for sub in item: + out += brief(sub, depth + 2) + return out + if t == "Div": + return [f"{pad}Div classes={b['c'][0][1]}"] + [l for sub in b["c"][1] for l in brief(sub, depth + 1)] + return [f"{pad}{t}"] + + return "\n".join(l for b in ast["blocks"] for l in brief(b)) + + +def write_filter(document: Path, backend: Backend) -> tuple[str, Reply]: + """Route B's one call: a Lua filter for the structure of this document's set.""" + version = subprocess.check_output(["pandoc", "--version"]).decode().split()[1] + prompt = f"""A problem sheet is read by pandoc {version}. Its block structure (pandoc's AST, abbreviated) is: + +{structure(document)} + +Write a Lua filter that replaces the whole document with one CodeBlock holding a JSON array: one object per question, in order, +{{"title": "", "main_text": "...", "parts": [{{"content": "...", "options": [], "answer": "", "worked_solution": ""}}]}} +Rules: a question is a top-level item of the numbered list of questions, or a section where the sheet uses headings; its main_text is the question's own paragraphs; its parts are the items of a numbered list nested inside it, each part's content being that nested item's paragraphs; a question with no nested list has one part with empty content. Render each text with pandoc.write(pandoc.Pandoc(blocks), "commonmark_x", {{wrap_text = "wrap-none"}}), keeping maths and images. Leave title empty unless the sheet names its questions. Ignore headings and figures that belong to no question. Build the JSON string by hand: escape only the double quote, the backslash and ASCII control characters (bytes below 32) - never any other byte, so that UTF-8 text passes through unchanged. Return the filter as: function Pandoc(doc) ... return pandoc.Pandoc({{pandoc.CodeBlock(json)}}) end.""" + reply = backend.call(FILTER_SYSTEM, prompt) + return re.sub(r"^```(lua)?\s*|\s*```$", "", reply.text.strip()), reply diff --git a/tests/fixtures/me2/direct.json b/tests/fixtures/me2/direct.json new file mode 100644 index 0000000..6190d98 --- /dev/null +++ b/tests/fixtures/me2/direct.json @@ -0,0 +1 @@ +[{"title":"Hydraulic scale","main_text":"A piston of diameter $D=0.1 \\mathrm{~m}$ is fitted inside a U-shaped tube filled with liquid mercury (with density in $\\rho_{\\mathrm{Hg}}=$ $13,540 \\mathrm{~kg} / \\mathrm{m}^{3}$ ), as shown by the sketch below. The mercury rises by $h=1 \\mathrm{~mm}$ under the weight of the piston.\n![](media/58e96592-84f6-41d7-9ab9-2276d8808071-1.jpg)","parts":[{"content":"What is the mass, $m$, of the piston?","options":[],"answer":"$$\nm=0.106 \\mathrm{~kg}\n$$","worked_solution":"The weight of the piston applies a pressure $p=4 m g /\\left(\\pi D^{2}\\right)$. This pressure adds to the atmospheric pressure, so that the pressure on the mercury on the left-hand side of the tube is $p+p_{a t}$. Applying the hydrostatic equations (or Bernoulli's equations for steady, inviscid and irrotational fluids) we find:\n\n- $$\np+p_{\\mathrm{at}}=p_{\\mathrm{at}}+\\rho_{\\mathrm{w}} \\sigma_{\\mathrm{Hg}} g h\n$$\nwhere $\\sigma$ is 'specific gravity'(density relative to water).\n\nHence:\n\n- $$\n\\begin{aligned}\n\\frac{4 m g}{\\pi D^{2}} & =\\rho_{\\mathrm{w}} \\sigma_{\\mathrm{Hg}} g h \\\\\nm & =\\frac{\\pi}{4} \\rho_{\\mathrm{w}} \\sigma_{\\mathrm{Hg}} h D^{2} \\\\\n& =\\frac{\\pi}{4} * 1000 * 13.54 * 10^{-3} * 10^{-2} \\\\\n& =0.106 \\mathrm{~kg}\n\\end{aligned}\n$$"},{"content":"If this result is to be used to measure the piston's weight, what would you do to improve the accuracy of the measurement?","options":[],"answer":"Selecting a fluid with a smaller specific gravity.","worked_solution":"To improve the accuracy of the measurement, $D$, being fixed by the piston's size, it is best to makehbigger (this is what is read), which translates in selecting a fluid with a smaller specific gravity."}]},{"title":"Friction on a plate","main_text":"A fluid with density $\\rho=800 \\mathrm{~kg} / \\mathrm{m}^{3}$, flows at $U_{0}=3 \\mathrm{~m} / \\mathrm{s}$ over a flat plate of length $L=1 \\mathrm{~m}$ and width $W=1 \\mathrm{~m}$. At the trailing edge the boundary-layer thickness is $\\delta=25 \\mathrm{~mm}$. Assume the velocity profile at the trailing edge to be linear (in the image shown), and the flow to be two-dimensional.\n![](media/58e96592-84f6-41d7-9ab9-2276d8808071-1-2.jpg)","parts":[{"content":"Compute the mass flow rate across the top surface of the control volume (noted \"ab\" in the figure).","options":[],"answer":"$$\n\\dot{m}=30 \\mathrm{~kg} / \\mathrm{s}\n$$","worked_solution":"From mass conservation:\n\n$$\n-\\dot{m}_{a d}=\\dot{m}_{a b}+\\dot{m}_{b c}-\n$$\n\nThis becomes:\n\n$$\n-\\rho W \\int_{0}^{\\delta} U_{0} \\mathrm{~d} y=\\dot{m}_{a b}+\\rho W \\int_{0}^{\\delta} u(y) \\mathrm{d} y-\n$$\n\nSince we know that $u(y)$ is linear, we can express it in terms of $y$ and $U_{0}$ :\n\n$$\nu(y)=U_{0} \\frac{y}{\\delta},\n$$\n\nTherefore:\n\n$$\n\\begin{aligned}\n\\dot{m}_{a b} & =\\rho W U_{0}\\left(\\int_{0}^{\\delta} 1-\\frac{y}{\\delta} \\mathrm{~d} y\\right) \\\\\n\\dot{m}_{a b} & =\\rho W U_{0} \\frac{\\delta}{2} \\\\\n\\dot{m}_{a b} & =(800)(1)(3) \\frac{25 \\times 10^{-3}}{2} . \\\\\n& \\dot{m}=30 \\mathrm{~kg} / \\mathrm{s}\n\\end{aligned}\n$$"},{"content":"Determine the drag force on the plate.","options":[],"answer":"$$\nF_{\\text {plate }}=30 \\mathrm{~N}\n$$","worked_solution":"From momentum conservation:\n\n$$\nF_{\\text {fluid }}=M_{a b}+M_{b c}-M_{a d}\n$$\n\nSince we are calculating the horizontal resultant force, we need to consider the horizontal momentum in the section \"ab\"; this means that we ignore any vertical velocities. Since the velocity of the fluid at $y=\\delta$ (i.e. at \"ab\") is always $U_{0}$, we can say:\n\n$$\nM_{a b}=U_{0} \\dot{m}_{a b},\n$$\n\nWe then also convert the other momentum flowrate terms into their mathematical forms, and proceed to find the force of the plate on the fluid, as shown below:\n\n$$\nF_{\\text {fluid }}=M_{a b}+\\rho W \\int_{0}^{\\delta} u(y)^{2} \\mathrm{dy}-\\rho W \\int_{0}^{\\delta} U_{0}^{2} \\mathrm{dy}\n$$\n\n$$\nF_{\\text {fluid }}=U_{0} \\dot{m}_{a b}+\\rho W U_{0}^{2}\\left(\\int_{0}^{\\delta} \\frac{y^{2}}{\\delta^{2}}-1 \\mathrm{dy}\\right)\n$$\n\n$$\nF_{\\text {fluid }}=(3)(30)+(800)(1)(3)^{2}\\left[-\\frac{2}{3}\\left(25 \\times 10^{-3}\\right)\\right]\n$$\n\n$$\nF_{\\text {fluid }}=-30 \\mathrm{~N}\n$$\n\nHowever, the question asks us to find the drag force of the fluid on the plate, hence:\n\n$$\nF_{\\text {plate }}=-F_{\\text {fluid }}\n$$\n\n$$\nF_{\\text {plate }}=30 \\mathrm{~N}\n$$"}]},{"title":"Towing a submarine","main_text":"A submerged submarine is towed horizontally at a steady speed $U$ in deep still water. An axially-symmetrical wake is formed behind the submarine in which the water velocity may be assumed to vary linearly from $U$ on the axis to zero at a radius of $R$. The variation of the water pressure with depth may be assumed to be unaffected by the presence of the submarine. The density of the water is $\\rho$. Using a control-volume analysis, we want to find the required power to tow the submarine. For both choices of control volumes ( A and B as shown above), derive an expression for:\n![](media/58e96592-84f6-41d7-9ab9-2276d8808071-2.jpg)","parts":[{"content":"The drag force $F$ of the submarine.","options":[],"answer":"$$\nF=\\frac{\\pi}{6} \\rho U^{2} R^{2}\n$$","worked_solution":"The mass flow rate entering the control volume is\n\n$$\n\\dot{m}_{\\mathrm{in}}=\\rho U A_{\\mathrm{disk}}=\\rho U \\pi R^{2}\n$$\n\n- \n\n(since the velocity is uniform). The mass exiting the control volume on the left (the wake) is:\n-\n\n$$\n\\dot{m}_{\\text {wake }}=2 \\pi \\rho \\int_{r=0}^{r=R} r(U r / R) \\mathrm{d} r\n$$\n\nHence, the mass flow rate leaving the control volume through the side $\\left(\\dot{m}_{\\text {side }}\\right)$ is:\n\n$$\n\\begin{aligned}\n\\dot{m}_{\\text {side }} & =\\dot{m}_{\\text {in }}-\\dot{m}_{\\text {wake }} \\\\\n& =\\rho U \\pi R^{2}-2 \\rho U \\pi \\int_{r=0}^{r=R} \\frac{r^{2}}{R} \\mathrm{~d} r \\\\\n& =\\rho U \\pi R^{2}-2 \\rho U \\pi\\left[\\frac{1}{3} \\frac{r^{3}}{R}\\right]_{r=0}^{r=R} \\\\\n& =\\rho U \\pi R^{2}-\\frac{2}{3} \\rho U \\pi R^{2} \\\\\n& =\\frac{1}{3} \\rho U \\pi R^{2}\n\\end{aligned}\n$$\n\nForce-momentum equation (FME):\n\nThe momentum flowrate entering the control volume is $M_{\\text {in }}=\\dot{m}_{\\text {in }} U$. The momentum flowrate exiting the control volume on the left (the wake) is\n-\n\n$$\nM_{\\text {wake }}=2 \\pi \\rho \\int_{r=0}^{r=R} r(U r / R)^{2} \\mathrm{~d} r\n$$\n\n- \n\nThe mass leaving the control volume by the side is also contributing to the removal of momentum, $M_{\\text {side }}=\\dot{m}_{\\text {side }} U$. Hence, the momentum leaving the control volume is $M_{\\text {out }}=M_{\\text {wake }}+M_{\\text {side }}$.\n\nThe FME reads:\n-\n\n$$\nM_{\\text {out }}-M_{\\text {in }}=-F+F_{p},\n$$\n\n- \n\nwhere $F_{p}$ represents pressure forces. However, we assume the pressure to be unaffected by the presence of the submarine, which leaves the hydrostatic pressure force. Since the hydrostatic pressure has the same linear profile on both sides of the control volume (front and wake), it contributes nothing to the horizontal force. Hence:\n-\n\n$$\n\\begin{aligned}\n-F & =2 \\pi \\rho \\int_{r=0}^{r=R} r(U r / R)^{2} \\mathrm{~d} r+\\dot{m}_{\\text {side }} U-\\dot{m}_{\\text {in }} U \\\\\n& =2 \\rho U^{2} \\pi \\int_{r=0}^{r=R}\\left(r^{3} / R^{2}\\right) \\mathrm{d} r+\\frac{1}{3} \\rho U^{2} \\pi R^{2}-\\rho U^{2} \\pi R^{2} \\\\\n& =2 \\rho U^{2} \\pi\\left[\\frac{1}{4} \\frac{r^{4}}{R^{2}}\\right]_{r=0}^{r=R}-\\frac{2}{3} \\rho U^{2} \\pi R^{2} \\\\\n& =\\frac{1}{2} \\rho U^{2} \\pi R^{2}-\\frac{2}{3} \\rho U^{2} \\pi R^{2}\n\\end{aligned}\n$$\n\n- \n\nWhich gives:\n\n$$\nF=\\frac{\\pi}{6} \\rho U^{2} R^{2}\n$$"},{"content":"The power $P$ required to tow the submarine.","options":[],"answer":"$$\nP=\\frac{\\pi}{6} \\rho U^{3} R^{2}\n$$","worked_solution":"By definition, the power is $F . U$. Therefore:\n\n$$\nP=\\frac{\\pi}{6} \\rho U^{3} R^{2}\n$$"}]},{"title":"Molecules, particles, and continuum","main_text":"Let us consider still air in standard atmospheric conditions at ground level: $T_{0}=273.15 \\mathrm{~K}, p_{0}=1.00 \\mathrm{bar}$. For simplicity, we assume air to be made of exactly the same diatomic molecules (a fair assumption) with molar mass $M=28.8 \\mathrm{~g} / \\mathrm{mol}$. Each molecule is modelled as a hard sphere of diameter $\\sigma=1.54 \\times 10^{-10} \\mathrm{~m}$. Consequently, air is considered to behave as an ideal gas. The Avogadro number is $\\mathcal{N}_{A}=6.02 \\times 10^{23} \\mathrm{~mol}^{-1}$, and the universal gas constant is $\\tilde{R}=8.314 \\mathrm{~J} /(\\mathrm{mol} \\cdot \\mathrm{K})$.","parts":[{"content":"Calculate the number of molecules $n_{0}$ per unit volume.\n\n(Note that in the response area below you can use exponential notation, e.g. $5.7 \\mathrm{e} 13 \\mathrm{~m}^{\\wedge}(-3)$ is an acceptable input - but an incorrect answer!).","options":[],"answer":"$$\nn_{0} \\approx 2.65 \\times 10^{25} \\mathrm{~m}^{-3} \\text { (molecules per metre cubed) }\n$$","worked_solution":"The fluid density is directly related to the particle density: $\\rho_{0}=n_{0} m / V$ (assuming all molecules to be the same). In this question, we take $V$ to be a cubic meter. Therefore, we need to calculate $\\rho_{0}$ and $m$ (the mass of one molecule).\n\nThe density can be computed from the ideal-gas law (the gas is assumed to be ideal): $\\rho_{0}=p_{0} /\\left(R T_{0}\\right)$ where $R=\\tilde{R} / M$.\n\nThe mass is directly computed from the Avogadro number and the molecular mass: $m=M / \\mathcal{N}_{A}$.\n\nHence:\n\n$$\nn_{0}=\\frac{p_{0} \\mathcal{N}_{A} V}{\\tilde{R} T_{0}}=\\frac{\\left(10^{5}\\right)\\left(6.02 \\times 10^{23}\\right)(1)}{(8.314)(273.15)}\n$$\n\n$$\nn_{0} \\approx 2.65 \\times 10^{25} \\text { molecules per metre cubed }\n$$"},{"content":"It can be shown that the mean-free path in the hard-sphere model is $\\ell=1 /\\left(\\sqrt{2} \\pi \\sigma^{2} n\\right)$. Give its numerical value at ground level:\n\nIf we are concerned with an engineering problem with length-scale $L \\approx 1 \\mathrm{~m}$, what should the size of a fluid particle be?","options":[],"answer":"$$\n\\begin{aligned}\n& \\ell_{0} \\approx 3.58 \\times 10^{-7} \\mathrm{~m} \\\\\n& 0.1 \\mathrm{~mm}}\n$$","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"$ \\dot{m}= $","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"comparePhysicalQuantities","livePreview":false,"gradeParams":{"rtol":0.05,"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[{"id":"075c2763-2164-4716-9204-4f15e3918ae4","payload":"30 kgs-1","expectedResponse":{"isCorrect":false}}],"cases":[{"id":"6f5e6262-6245-4a6c-83de-ff8dd8474ef5","answer":"30 kgs-1","feedback":"To enter a negative exponent, put the exponent in parentheses (e.g. ```kg*s^(-1)``` ). The expression ```kgs-1``` is interpreted differently and is incorrect.. ","isCorrect":false,"params":null}],"response":{"responseInput":{"responseType":"NUMERIC_UNITS","answer":"30 kg/s","config":null}}}],"workedSolution":{"content":"From mass conservation:\n\n---\n\n$\\dot{m}_{ad}=\\dot{m}_{ab}+\\dot{m}_{bc}$\n\n---\n\nThis becomes:\n\n---\n\n$\\rho W\\Large{\\int}_{\\small 0}^{\\small \\delta}\\normalsize{U_0\\space \\mathrm{d}y}=\\dot{m}_{ab}+\\rho W\\Large{\\int}_{\\small 0}^{\\small \\delta}\\normalsize{u(y)\\space\\mathrm{d}y}$\n\n---\n\nSince we know that $u(y)$ is linear, we can express it in terms of $y$ and $U_0$:\n\n---\n\n$$\nu(y)=U_0\\frac{y}{\\delta},\n$$\n\n---\n\nTherefore:\n\n$$\n\\begin{aligned}\\dot{m}_{ab}&=\\rho W U_0\\left(\\Large{\\int}_{\\small 0}^{\\small \\delta}\\normalsize{1-\\frac{y}{\\delta}\\space\\mathrm{d}y} \\right)\\\\\\dot{m}_{ab}&=\\rho W U_0\\frac{\\delta}{2},\\\\\\dot{m}_{ab}&=(800)(1)(3)\\frac{25\\times 10^{-3}}{2}.\\end{aligned}\n$$\n\n---\n\n$$\n\\dot{m}= \\boxed{30 \\space \\mathrm{kg/s} \\>}\n$$","children":[]}},{"orderNumber":1,"content":"Determine the drag force on the plate.","answerContent":"$$\nF_{\\mathrm{plate}}= \\boxed{30\\space \\mathrm{N}\\space}\n$$","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"$F_D =$","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"comparePhysicalQuantities","livePreview":false,"gradeParams":{"rtol":0.05,"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"NUMERIC_UNITS","answer":"30 N","config":null}}}],"workedSolution":{"content":"From momentum conservation:\n\n***\n\n***\n\n$$\nF_{\\small \\mathrm{fluid}}=M_{ab}+M_{bc}-M_{ad}\n$$\n\n***\n\nSince we are calculating the horizontal resultant force, we need to consider the horizontal momentum in the section \"ab\"; this means that we ignore any vertical velocities. Since the velocity of the fluid at $y=\\delta$ (i.e. at \"ab\") is always $U_0$, we can say:\n\n***\n\n$$\nM_{ab}=U_0\\dot{m}_{ab},\n$$\n\n***\n\nWe then also convert the other momentum flowrate terms into their mathematical forms, and proceed to find the force of the plate on the fluid, as shown below:\n\n***\n\n$$\nF_{\\mathrm{fluid}}=M_{ab}+\\rho W\\Large{\\int}_{\\small 0}^{\\small \\delta}\\normalsize{u(y)^2\\space\\mathrm{dy}}-\\rho W\\Large{\\int}_{\\small 0}^{\\small \\delta}\\normalsize{U_0^2\\space \\mathrm{dy}}\n$$\n\n***\n\n$$\nF_{\\mathrm{fluid}}=U_0\\dot{m}_{ab}+\\rho WU_0^2\\left(\\LARGE{\\int}_{\\small 0}^{\\small \\delta}\\normalsize \\frac{y^2}{\\delta^2}-1\\space\\mathrm{dy} \\right)\n$$\n\n***\n\n$$\nF_{\\mathrm{fluid}}=(3)(30)+(800)(1)(3)^2\\left[ -\\frac{2}{3}(25\\times 10^{-3})\\right]\n$$\n\n***\n\n$$\nF_{\\mathrm{fluid}}=-30 \\space \\mathrm{N}\n$$\n\n***\n\nHowever, the question asks us to find the drag force **of the fluid on the plate**, hence:\n\n***\n\n$$\nF_{\\mathrm{plate}}=-F_{\\mathrm{fluid}}\n$$\n\n***\n\n$$\nF_{\\mathrm{plate}}= \\boxed{30\\space \\mathrm{N}\\space}\n$$\n","children":[]}}]} \ No newline at end of file diff --git a/tests/fixtures/me2/export/question_002_Towing_a_submarine.json b/tests/fixtures/me2/export/question_002_Towing_a_submarine.json new file mode 100644 index 0000000..3f899a3 --- /dev/null +++ b/tests/fixtures/me2/export/question_002_Towing_a_submarine.json @@ -0,0 +1 @@ +{"orderNumber":2,"title":"Towing a submarine","skill":0.6666666666666666,"guidance":"This is another revision question from ME1, for those who need extra practice. ","durationLowerBound":15,"durationUpperBound":20,"masterContent":"A submerged submarine is towed horizontally at a steady speed $U$ in deep still water. An axially-symmetrical wake is formed behind the submarine in which the water velocity may be assumed to vary linearly from $U$ on the axis to zero at a radius of $R$. The variation of the water pressure with depth may be assumed to be unaffected by the presence of the submarine. The density of the water is $\\rho$. Using a control-volume analysis, we want to find the required power to tow the submarine. For both choices of control volumes (A and B as shown above), derive an expression for:\n\n![pictureTag](question_002_Towing_a_submarine_0001.png){ width=60% }","publish":true,"displayFinalAnswer":true,"displayStructuredTutorial":true,"displayWorkedSolution":true,"displayChatbot":true,"parts":[{"orderNumber":0,"content":"The drag force $F$ of the submarine.","answerContent":"$$\n\\boxed{F = \\frac{\\pi}{6}\\rho U^2 R^2}\n$$","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"$F=$","postResponseText":"","inputSymbols":[{"symbol":"\\(R\\)","code":"R","aliases":["r"],"isVisible":true},{"symbol":"\\(U\\)","code":"U","aliases":["u","V","v"],"isVisible":true},{"symbol":"\\(\\rho\\)","code":"rho","aliases":["Rho","RHO","ρ"],"isVisible":true},{"symbol":"\\(\\pi\\)","code":"pi","aliases":["Pi","PI","π"],"isVisible":true}],"displayInputSymbols":true,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"symbolicEqual","livePreview":true,"gradeParams":{"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[{"id":"afae51c0-4d51-4272-b978-f7d3edc3d5a4","payload":"(pi/6)*(rho)*(U**2)*(R**2)","expectedResponse":{"isCorrect":true}},{"id":"98837460-146f-4433-9f28-495196072542","payload":"(pi/6)*(rho)*(u**2)*(R**2)","expectedResponse":{"isCorrect":true}},{"id":"aa114fbe-7a4d-4435-9c0f-21d95e83e314","payload":"(pi/6)*(rho)*(U**2)*(r**2)","expectedResponse":{"isCorrect":true}},{"id":"449d18b4-4822-4306-b2bb-66669ad58ae8","payload":"(pi/6)*(Rho)*(U**2)*(R**2)","expectedResponse":{"isCorrect":true}},{"id":"560c9d4f-a984-4250-a13b-8067ebc8442d","payload":"(Pi/6)*(rho)*(U**2)*(R**2)","expectedResponse":{"isCorrect":true}}],"cases":[{"id":"9d168319-b09e-4277-a1cf-624682a901de","answer":"(pi)*(rho)*(U**2)*(R**2)","feedback":"The form of your expression looks right, but there's a missing factor.","isCorrect":false,"params":null},{"id":"42868d79-452d-4b75-ada5-c16728b073ac","answer":"pi*rho*U^2*R^2","feedback":"","isCorrect":false,"params":null},{"id":"f2ce4773-174b-4acb-9dd8-8acae1f04423","answer":"pi*rho*U^2*R^2","feedback":"","isCorrect":false,"params":null},{"id":"85b72db0-345e-4d68-8593-fbfca8660e7c","answer":"pi*rho*U^2","feedback":"The submitted expression has some of the components of the correct answer, but is incomplete.","isCorrect":false,"params":null},{"id":"c114967b-8409-411f-971e-0d067a528498","answer":"U*U","feedback":"","isCorrect":false,"params":null},{"id":"684d65fe-282d-485c-915e-8103baabb4dc","answer":"-(pi/6)*(rho)*(U**2)*(R**2)","feedback":"The magnitude is correct. The sign is opposite to the reference answer but as the question did not define the coordinate system explicitly then this is presumably correct.","isCorrect":true,"params":null},{"id":"f996f895-0d2c-4a9c-8d57-d1397000230c","answer":"pi*R**2*U**2*rho/2","feedback":"The form of your expression is correct but the coefficient is incorrect.","isCorrect":false,"params":null},{"id":"56a70181-4cb6-4bc3-ae5e-daae498f4fcf","answer":"-(rho/2U^2piR^2)","feedback":"The form of your expression looks right, but there's a missing factor.","isCorrect":false,"params":null}],"response":{"responseInput":{"responseType":"MATH_SINGLE_LINE","answer":"(pi/6)*(rho)*(U**2)*(R**2)","config":{"allowPhoto":true,"allowHandwrite":true,"enableRefinement":true}}}}],"workedSolution":{"content":"The mass flow rate entering the control volume is\n\n---\n\n$$\n\\dot{m}_{\\mathrm{in}} = \\rho U A_{\\mathrm{disk}} = \\rho U \\pi R^2\n$$\n\n---\n\n(since the velocity is uniform). The mass exiting the control volume on the left (the wake) is:\n\n---\n\n$$\n\\dot{m}_{\\mathrm{wake}} = 2\\pi\\rho \\int_{r=0}^{r=R} r (U r /R)\\mathrm{d}r\n$$\n\n---\n\nHence, the mass flow rate leaving the control volume through the side ($\\dot{m}_{\\mathrm{side}}$) is:\n\n---\n\n$$\n\\begin{aligned}\\dot{m}_{\\mathrm{side}} &= \\dot{m}_{\\mathrm{in}} - \\dot{m}_{\\mathrm{wake}} \\\\&= \\rho U \\pi R^2 - 2\\rho U \\pi \\int_{r=0}^{r=R} \\frac{r^2 }{R}\\mathrm{d}r \\\\&= \\rho U \\pi R^2 - 2\\rho U \\pi \\left[\\frac{1}{3}\\frac{r^3}{R}\\right]_{r=0}^{r=R}\\\\ &= \\rho U \\pi R^2 - \\frac{2}{3}\\rho U \\pi R^2 \\\\&= \\frac{1}{3}\\rho U \\pi R^2\\end{aligned}\n$$\n\n---\n\nForce-momentum equation (FME):\n\n \n\n---\n\nThe momentum flowrate entering the control volume is $M_{\\mathrm{in}} = \\dot{m}_{\\mathrm{in}} U$. The momentum flowrate exiting the control volume on the left (the wake) is\n\n---\n\n$$\nM_{\\mathrm{wake}} = 2\\pi\\rho \\int_{r=0}^{r=R} r (U r /R)^2\\mathrm{d}r.\n$$\n\n---\n\nThe mass leaving the control volume by the side is also contributing to the removal of momentum, $M_{\\mathrm{side}} = \\dot{m}_{\\mathrm{side}} U$. Hence, the momentum leaving the control volume is $M_{out} = M_{\\mathrm{wake}} + M_{\\mathrm{side}}$.\n\n---\n\nThe FME reads:\n\n---\n\n$$\nM_{out} - M_{\\mathrm{in}} = - F + F_p,\n$$\n\n---\n\nwhere $F_p$ represents pressure forces. However, we assume the pressure to be unaffected by the presence of the submarine, which leaves the hydrostatic pressure force. Since the hydrostatic pressure has the same linear profile on both sides of the control volume (front and wake), it contributes nothing to the horizontal force. Hence:\n\n---\n\n$$\n\\begin{aligned}- F &= 2\\pi\\rho \\int_{r=0}^{r=R} r (U r /R)^2\\mathrm{d}r + \\dot{m}_{\\mathrm{side}} U - \\dot{m}_{\\mathrm{in}} U \\\\&= 2\\rho U^2 \\pi \\int_{r=0}^{r=R} (r^3/R^2)\\mathrm{d}r + \\frac{1}{3}\\rho U^2 \\pi R^2 - \\rho U^2\\pi R^2\\\\ &= 2\\rho U^2 \\pi \\left[\\frac{1}{4}\\frac{r^4}{R^2} \\right]_{r=0}^{r=R} - \\frac{2}{3}\\rho U^2 \\pi R^2 \\\\&= \\frac{1}{2}\\rho U^2 \\pi R^2 - \\frac{2}{3}\\rho U^2 \\pi R^2\\end{aligned}\n$$\n\n---\n\nWhich gives:\n\n$$\n\\boxed{F = \\frac{\\pi}{6}\\rho U^2 R^2}\n$$","children":[]}},{"orderNumber":1,"content":"The power $P$ required to tow the submarine.","answerContent":"$$\n\\boxed{P = \\frac{\\pi }{6}\\rho U^3 R^2}\n$$","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"$P=$","postResponseText":"","inputSymbols":[{"symbol":"\\(U\\)","code":"U","aliases":["u"],"isVisible":true},{"symbol":"\\(\\pi\\)","code":"pi","aliases":["Pi","PI"],"isVisible":true},{"symbol":"\\(\\rho\\)","code":"rho","aliases":["Rho","RHO"],"isVisible":true}],"displayInputSymbols":true,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"symbolicEqual","livePreview":true,"gradeParams":{"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[{"id":"fda9c491-2b6a-4ffe-9631-070acc2cb35e","payload":"(pi/6)*(rho)*(U**3)*(R**2)","expectedResponse":{"isCorrect":true}},{"id":"93a81ae6-6e75-4e23-b280-25888b8afcff","payload":"pi/6*rho*U^3R^2","expectedResponse":{"isCorrect":true}},{"id":"c56f9f18-7dfd-4f03-9078-c9bc0aae1c42","payload":"(pi)*(rho)*(U**3)*(R**2)","expectedResponse":{"isCorrect":false}}],"cases":[{"id":"916b0fb6-d28f-4f16-a892-8c8855a39866","answer":"(pi)*(rho)*(U**3)*(R**2)","feedback":"This answer is close in form but is incorrect by a numerical factor. ","isCorrect":false,"params":null}],"response":{"responseInput":{"responseType":"MATH_SINGLE_LINE","answer":"(pi/6)*(rho)*(U**3)*(R**2)","config":{"allowPhoto":true,"allowHandwrite":true,"enableRefinement":true}}}}],"workedSolution":{"content":"By definition, the power is $F.U$. Therefore:\n\n***\n\n$$\n\\boxed{P = \\frac{\\pi }{6}\\rho U^3 R^2}\n$$\n","children":[]}}]} \ No newline at end of file diff --git a/tests/fixtures/me2/export/question_003_Molecules,_particles,_and_continuum.json b/tests/fixtures/me2/export/question_003_Molecules,_particles,_and_continuum.json new file mode 100644 index 0000000..aab1f2a --- /dev/null +++ b/tests/fixtures/me2/export/question_003_Molecules,_particles,_and_continuum.json @@ -0,0 +1 @@ +{"orderNumber":3,"title":"Molecules, particles, and continuum","skill":0.6666666666666666,"guidance":"This question bridges ME1 and ME2, exploring the continuum hypothesis and the definition of a fluid particle.","durationLowerBound":20,"durationUpperBound":25,"masterContent":"Let us consider still air in standard atmospheric conditions at ground level: $T_0 = 273.15\\,\\mathrm{K}$, $p_0 = 1.00\\,\\mathrm{bar}$. For simplicity, we assume air to be made of exactly the same diatomic molecules (a fair assumption) with molar mass $M = 28.8\\,\\mathrm{g/mol}$. Each molecule is modelled as a hard sphere of diameter $\\sigma = 1.54\\times 10^{-10}\\,\\mathrm{m}$. Consequently, air is considered to behave as an ideal gas. The Avogadro number is $\\mathcal{N}_A = 6.02\\times 10^{23}\\,\\mathrm{mol}^{-1}$, and the universal gas constant is $\\tilde{R} = 8.314\\,\\mathrm{J/(mol\\cdot K)}$.","publish":true,"displayFinalAnswer":true,"displayStructuredTutorial":true,"displayWorkedSolution":true,"displayChatbot":true,"parts":[{"orderNumber":0,"content":"Calculate the number of molecules $n_{\\small 0}$ per unit volume.\n\n \n\n(Note that in the response area below you can use exponential notation, e.g. `5.7e13 m^(-3)` is an acceptable input - but an incorrect answer!).\n","answerContent":"$$\nn_{\\small 0}\\approx \\boxed{2.65 \\times 10^{25}\\,\\mathrm{m}^{-3}~\\text{(molecules per metre cubed)} \\space}\n$$\n\nTo enter in the checker, for example:\n\n`2.65e25 m^(-3)`\n","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"$n_0 \\approx$ ","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"comparePhysicalQuantities","livePreview":false,"gradeParams":{"rtol":0.1,"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"NUMERIC_UNITS","answer":"2.65e+25 m^(-3)","config":null}}}],"workedSolution":{"content":"The fluid density is directly related to the particle density: $\\rho_0 = n_{\\small 0} m /V$ (assuming all molecules to be the same). In this question, we take $V$ to be a cubic meter. Therefore, we need to calculate $\\rho_0$ and $m$ (the mass of one molecule).\n\n***\n\nThe density can be computed from the ideal-gas law (the gas is assumed to be ideal): $\\rho_0 = p_0 / (R T_0)$ where $R = \\tilde{R}/M$.\n\n***\n\nThe mass is directly computed from the Avogadro number and the molecular mass: $m = M/\\mathcal{N}_A$.\n\n***\n\nHence:\n\n***\n\n$$\nn_{\\small 0} = \\frac{p_0 \\mathcal{N}_A V}{\\tilde{R} T_0} = \\frac{(10^5)(6.02\\times 10^{23})(1)}{(8.314)(273.15)} \n$$\n\n\n\n***\n\n$$\nn_{\\small 0}\\approx \\boxed{2.65 \\times 10^{25}\\,\\text{molecules per metre cubed} \\space}\n$$\n","children":[]}},{"orderNumber":1,"content":"It can be shown that the mean-free path in the hard-sphere model is $\\ell = 1/(\\sqrt{2}\\pi \\sigma^2 n)$. Give its numerical value at ground level:","answerContent":"$$\n\\boxed{\\ell_{\\small{0}}\\approx 3.58\\times 10^{-7}\\space\\mathrm{m}\\space},\n$$\n\n$$\n \\space \\boxed{0.1~\\mathrm{mm}< d < 1~\\mathrm{mm}}.\n$$\n","responseAreas":[{"orderNumber":1,"contentAfter":"","preResponseText":"\\(d=\\)","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"comparePhysicalQuantities","livePreview":false,"gradeParams":{"rtol":0.85,"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"NUMERIC_UNITS","answer":"0.541 mm","config":null}}},{"orderNumber":0,"contentAfter":"If we are concerned with an engineering problem with length-scale $L\\approx 1~\\mathrm{m}$, what should the size of a fluid particle be?","preResponseText":"\\(\\ell_0=\\)","postResponseText":"","inputSymbols":[{"symbol":"$\\mu$","code":"mu","aliases":[],"isVisible":true}],"displayInputSymbols":false,"includeInPdf":false,"saveAllowed":false,"evaluationFunctionName":"comparePhysicalQuantities","livePreview":false,"gradeParams":{"rtol":0.00005,"strict_syntax":false},"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"NUMERIC_UNITS","answer":"3.58e-7 m","config":null}}}],"workedSolution":{"content":"$$\n\\ell_0 = \\frac{1}{\\sqrt{2}\\pi\\sigma^2 n_{0}} = \\frac{1}{\\sqrt{2}\\pi (1.54\\times 10^{-10})^2 (2.65 \\times 10^{25})} \\approx 3.58 \\times 10^{-7}\\,\\mathrm{m}\n$$\n\n***\n\nThe fluid particle size $d$ must be such that:\n\n***\n\n$$\n10^{-7}\\ \\mathrm{m} \\ll d \\ll 10^{0}\\ \\mathrm{m} \\quad \\longrightarrow \\quad 0.1\\ \\mathrm{mm} < d < 1\\,\\mathrm{mm}\n$$\n","children":[]}},{"orderNumber":2,"content":"Aircraft designers assume air to be a continuum medium. The density of air decreases with altitude as follows:\n\n$$\n\\\\\\rho(z) = \\rho_0 \\left[1 - \\frac{g}{c_p T_0}z\\right]^{1/(\\gamma-1)}\\\\\n$$\n\nwhere $\\rho_0$, $T_0$ are the density and temperature at ground level and $z$ is the altitude measured from the ground. The gravitational acceleration is $g = 9.8\\,\\mathrm{m/s}^2$, the specific heat at constant pressure is $c_p = 0.83\\,\\mathrm{kJ/(kg \\cdot K)}$ and the heat capacity ratio is $\\gamma = 7/5$.\n\nAs density decreases, the mean-free path is expected to increase. Therefore, there must be a height $H$ from which the continuum assumption is no longer valid. Using the results and assumptions from parts (a) and (b), and assuming that the aircraft designer is concerned with scales of order one meter, is it reasonable to use the continuum model for an airline at 10 km altitude? Show your working.","answerContent":"The designers' assumption appears to be adequate.","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":true,"saveAllowed":false,"evaluationFunctionName":"arrayEqual","livePreview":false,"gradeParams":null,"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#22C55E","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"MULTIPLE_CHOICE","answer":[true,false],"config":{"single":true,"options":["Yes","No"],"randomise":false}}}}],"workedSolution":{"content":"First, we show that the density ratio $\\rho/\\rho_0$ can be written in terms of particle densities:\n\n***\n\n$$\n\\frac{\\rho}{\\rho_0} = \\frac{n m}{V}\\cdot\\frac{{V}}{n_{0} m} = \\frac{n}{n_{0}}\n$$\n\n***\n\nFrom part (b) the mean free path is given by $\\ell = 1/(\\sqrt{2}\\pi\\sigma^2 n)$, hence:\n\n***\n\n$$\n\\frac{\\rho}{\\rho_0} = \\frac{n}{n_{0}} = \\frac{\\sqrt{2}\\pi\\sigma^2\\ell_0}{\\sqrt{2}\\pi\\sigma^2\\ell} = \\frac{\\ell_0}{\\ell}\n$$\n\n***\n\nFor an altitude of $H=10~\\mathrm{km}$,\n\n***\n\n$$\n\\frac{\\rho_{\\small{z=H}}}{\\rho_0} = \\frac{\\ell_{\\small 0}}{\\ell_{\\small z=H}} = \\left[1 - \\frac{g}{c_p T_0} H\\right]^{1/(\\gamma-1)}\n$$\n\n***\n\nRearranging for the mean-free path at altitude,\n\n***\n\n$$\n\\begin{aligned} \\ell_{\\small z=H} &= \\ell_{\\small 0}\\left[1 - \\frac{g}{c_p T_0} H\\right]^{-1/(\\gamma-1)}\\\\\\ell_{\\small z=H} &= 3.58\\times 10^{-7}\\left[1 - \\left(\\frac{9.8*10^4}{830*273.15}\\right)\\right]^{(-5/2)} \\\\&= 1.5\\,\\mathrm{\\mu m}.\\space\\space \\end{aligned}\n$$\n\n***\n\nwhere $\\ell_0$ is taken from part (b). For the engineering length scale $L=1~\\mathrm{m}$ a particle size $\\ell \\ll d \\ll L$ is **still (more-or-less!) possible**. The requirement is not a strict ratio of $10^3$ and we must be practical in our decision making. For example, the length scale of 1 m is quite rough; and there are also likely to be much larger sources of error in any computations based on the model - not least the parameters used in the computation. However, for *significantly* smaller engineering scales or higher altitudes, careful attention to the practical accuracy necessary from the computations carried out using the continuum assumption would be required.\n","children":[]}}]} \ No newline at end of file diff --git a/tests/fixtures/me2/export/question_004_Frames_of_reference.json b/tests/fixtures/me2/export/question_004_Frames_of_reference.json new file mode 100644 index 0000000..3956cee --- /dev/null +++ b/tests/fixtures/me2/export/question_004_Frames_of_reference.json @@ -0,0 +1 @@ +{"orderNumber":4,"title":"Frames of reference","skill":0.3333333333333333,"guidance":"A simple test of your basic understanding of Eulerian and Lagrangian frames of reference (content covered in Lecture 1 in ME2).","durationLowerBound":3,"durationUpperBound":5,"masterContent":"","publish":true,"displayFinalAnswer":true,"displayStructuredTutorial":true,"displayWorkedSolution":true,"displayChatbot":true,"parts":[{"orderNumber":0,"content":"What is $\\vec{\\chi}_\\mathrm{}(t)$?\n","answerContent":"The location of a particle at time, $t$.","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":true,"saveAllowed":false,"evaluationFunctionName":"arrayEqual","livePreview":false,"gradeParams":null,"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#C4CDD5","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"MULTIPLE_CHOICE","answer":[false,false,true,false],"config":{"single":true,"options":["A fixed location, $x$, in space for all time, \\(t\\).","The Lagrange multiplier in time, $t$.","The location of a particle, $\\mathrm{p}$, at time, $t$.","The relation between velocity, $\\mathrm{p}$, and space, $x$, at time, $t$."],"randomise":false}}}}]},{"orderNumber":1,"content":"When does $\\vec{u}(\\vec{x},t)=\\frac{\\mathrm{d}\\vec{\\chi}_\\mathrm{p}}{\\mathrm{d}t}$?\n\n \n\nWhen ...\n","answerContent":"$\\vec{x}=\\vec{\\chi}_\\mathrm{p}$.","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":true,"saveAllowed":false,"evaluationFunctionName":"arrayEqual","livePreview":false,"gradeParams":null,"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#C4CDD5","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"MULTIPLE_CHOICE","answer":[false,false,false,false,true,false,false],"config":{"single":true,"options":["$t=0$.","$\\vec{x}=0$.","$\\vec{u}=0$.","$\\vec{u}=\\vec{x}$.","$\\vec{x}=\\vec{\\chi}_{\\mathrm{P}}$.","$\\vec{u}=\\vec{\\chi}_{\\mathrm{P}}$.","$t=t_{\\mathrm{P}}$."],"randomise":false}}}}]},{"orderNumber":2,"content":"For a Eulerian velocity field $\\vec{u}(\\vec{x},t)$, which of the following would be sufficient to evaluate a particular value of the field?","answerContent":"A specific particle, $\\vec{\\chi}$, at a specific time, $t$.","responseAreas":[{"orderNumber":0,"contentAfter":"","preResponseText":"","postResponseText":"","inputSymbols":[],"displayInputSymbols":false,"includeInPdf":true,"saveAllowed":false,"evaluationFunctionName":"arrayEqual","livePreview":false,"gradeParams":null,"separateFeedback":true,"commonFeedbackColor":"#C4CDD5","correctFeedbackColor":"#C4CDD5","correctFeedbackPrefix":"Correct","incorrectFeedbackColor":"#ff5630","incorrectFeedbackPrefix":"Incorrect","tests":[],"cases":[],"response":{"responseInput":{"responseType":"MULTIPLE_CHOICE","answer":[false,false,false,false,true,false],"config":{"single":true,"options":["Pressure, $p$, temperature, $T$ and density, $\\rho$.","A Eulerian frame of reference.","A region of space, $\\vec{x}$, and a collection of particles $t$.","The continuum hypothesis.","A specific particle, $\\vec{\\chi}$, at a specific time, $t$.","A specific particle, $\\vec{\\chi}$, and a point in space $\\vec{x}$."],"randomise":false}}}}],"workedSolution":{"content":"In mathematical notation we write\n\n$$\n\\vec{u}(\\vec{x},t),\n$$\n\nmeaning that the Eulerian vector velocity $\\vec{u}$ can be evaluated for a given point in space in a Eulerian frame of reference, $\\vec{x}$, and a point in time, $t$. This option, however, was not in the list.\n\nThere is an alternative definition, which is to choose a specific particle, $\\vec{\\chi}(t)$ and a point in time, $t$, because this definition provides, implicitly, a point in space and time.\n\nIt is not sufficient to define a specific particle and a point in space, firstly because the particle may not ever pass that point in space; and secondly because if it does, we cannot guarantee in general that it should pass that point once and only once, so we have not defined a unique point in time. We awarded half-points for this response because it is also not completely wrong --- it is sufficient to evaluate some values of the Eulerian velocity field, but not necessarily all and not necessary unique values.\n","children":[]}}]} \ No newline at end of file diff --git a/tests/fixtures/me2/export/set_Introduction.json b/tests/fixtures/me2/export/set_Introduction.json new file mode 100644 index 0000000..87eb4eb --- /dev/null +++ b/tests/fixtures/me2/export/set_Introduction.json @@ -0,0 +1 @@ +{"name":"Introduction","description":"","isSurvey":false,"releasedAt":"2025-09-29T11:00:00.000Z","manuallyHidden":true,"finalAnswerVisibility":"OPEN_WITH_WARNINGS","workedSolutionVisibility":"OPEN_WITH_WARNINGS","structuredTutorialVisibility":"OPEN","chatbotVisibility":"OPEN"} \ No newline at end of file diff --git a/tests/fixtures/me2/questions.md b/tests/fixtures/me2/questions.md new file mode 100644 index 0000000..3b49813 --- /dev/null +++ b/tests/fixtures/me2/questions.md @@ -0,0 +1,92 @@ +## MECH50010 Fluid Mechanics 2 Problem Set \#1 Introduction + +### 1.1 Hydraulic scale + +5-10 mins +This is a gentle warm up question to get into the swing of things after a long summer. +A piston of diameter $D=0.1 \mathrm{~m}$ is fitted inside a U-shaped tube filled with liquid mercury (with density in $\rho_{\mathrm{Hg}}=$ $13,540 \mathrm{~kg} / \mathrm{m}^{3}$ ), as shown by the sketch below. The mercury rises by $h=1 \mathrm{~mm}$ under the weight of the piston. +![](media/58e96592-84f6-41d7-9ab9-2276d8808071-1.jpg) + +(a) What is the mass, $m$, of the piston? +(b) If this result is to be used to measure the piston's weight, what would you do to improve the accuracy of the measurement? + +### 1.2 Friction on a plate + +★★ 15-20 mins +This question is to refresh yourself using control volumes, which were used in ME1 and will be used often in ME2. +A fluid with density $\rho=800 \mathrm{~kg} / \mathrm{m}^{3}$, flows at $U_{0}=3 \mathrm{~m} / \mathrm{s}$ over a flat plate of length $L=1 \mathrm{~m}$ and width $W=1 \mathrm{~m}$. At the trailing edge the boundary-layer thickness is $\delta=25 \mathrm{~mm}$. Assume the velocity profile at the trailing edge to be linear (in the image shown), and the flow to be two-dimensional. +![](media/58e96592-84f6-41d7-9ab9-2276d8808071-1-2.jpg) + +(a) Compute the mass flow rate across the top surface of the control volume (noted "ab" in the figure). +(b) Determine the drag force on the plate. + +### 1.3 Towing a submarine + +- 15-20 mins + +This is another revision question from ME1, for those who need extra practice. +A submerged submarine is towed horizontally at a steady speed $U$ in deep still water. An axially-symmetrical wake is formed behind the submarine in which the water velocity may be assumed to vary linearly from $U$ on the axis to zero at a radius of $R$. The variation of the water pressure with depth may be assumed to be unaffected by the presence of the submarine. The density of the water is $\rho$. Using a control-volume analysis, we want to find the required power to tow the submarine. For both choices of control volumes ( A and B as shown above), derive an expression for: +![](media/58e96592-84f6-41d7-9ab9-2276d8808071-2.jpg) + +(a) The drag force $F$ of the submarine. +(b) The power $P$ required to tow the submarine. + +### 1.4 Molecules, particles, and continuum + +- 20-25 mins + +This question bridges ME1 and ME2, exploring the continuum hypothesis and the definition of a fluid particle. +Let us consider still air in standard atmospheric conditions at ground level: $T_{0}=273.15 \mathrm{~K}, p_{0}=1.00 \mathrm{bar}$. For simplicity, we assume air to be made of exactly the same diatomic molecules (a fair assumption) with molar mass $M=28.8 \mathrm{~g} / \mathrm{mol}$. Each molecule is modelled as a hard sphere of diameter $\sigma=1.54 \times 10^{-10} \mathrm{~m}$. Consequently, air is considered to behave as an ideal gas. The Avogadro number is $\mathcal{N}_{A}=6.02 \times 10^{23} \mathrm{~mol}^{-1}$, and the universal gas constant is $\tilde{R}=8.314 \mathrm{~J} /(\mathrm{mol} \cdot \mathrm{K})$. + +(a) Calculate the number of molecules $n_{0}$ per unit volume. + +(Note that in the response area below you can use exponential notation, e.g. $5.7 \mathrm{e} 13 \mathrm{~m}^{\wedge}(-3)$ is an acceptable input - but an incorrect answer!). + +(b) It can be shown that the mean-free path in the hard-sphere model is $\ell=1 /\left(\sqrt{2} \pi \sigma^{2} n\right)$. Give its numerical value at ground level: + +If we are concerned with an engineering problem with length-scale $L \approx 1 \mathrm{~m}$, what should the size of a fluid particle be? + +(c) Aircraft designers assume air to be a continuum medium. The density of air decreases with altitude as follows: +$$ +\rho(z)=\rho_{0}\left[1-\frac{g}{c_{p} T_{0}} z\right]^{1 /(\gamma-1)} +$$ + +where $\rho_{0}, T_{0}$ are the density and temperature at ground level and $z$ is the altitude measured from the ground. The gravitational acceleration is $g=9.8 \mathrm{~m} / \mathrm{s}^{2}$, the specific heat at constant pressure is $c_{p}=0.83 \mathrm{~kJ} /(\mathrm{kg} \cdot \mathrm{K})$ and the heat capacity ratio is $\gamma=7 / 5$. + +As density decreases, the mean-free path is expected to increase. Therefore, there must be a height $H$ from which the continuum assumption is no longer valid. Using the results and assumptions from parts (a) and (b), and assuming that the aircraft designer is concerned with scales of order one meter, is it reasonable to use the continuum model for an airline at 10 km altitude? Show your working. + +- Yes +- No + +### 1.5 Frames of reference + +- 3-5 mins + +A simple test of your basic understanding of Eulerian and Lagrangian frames of reference (content covered in Lecture 1 in ME2). + +(a) What is $\vec{\chi}(t)$ ? + - A fixed location, $x$, in space for all time, (t). + - The Lagrange multiplier in time, $t$. + - The location of a particle, p, at time, $t$. + - The relation between velocity, p , and space, $x$, at time, $t$. +(b) When does $\vec{u}(\vec{x}, t)=\frac{\mathrm{d} \vec{\chi}_{\mathrm{p}}}{\mathrm{d} t}$ ? + +When ... + +- $t=0$. +- $\vec{x}=0$. +- $\vec{u}=0$. +- $\vec{u}=\vec{x}$. +- $\vec{x}=\vec{\chi}_{\mathrm{P}}$. +- $\vec{u}=\vec{\chi}_{\mathrm{P}}$. +- $t=t_{\mathrm{P}}$. +(c) For a Eulerian velocity field $\vec{u}(\vec{x}, t)$, which of the following would be sufficient to evaluate a particular value of the field? +- Pressure, $p$, temperature, $T$ and density, $\rho$. +- A Eulerian frame of reference. +- A region of space, $\vec{x}$, and a collection of particles $t$. +- The continuum hypothesis. +- A specific particle, $\vec{\chi}$, at a specific time, $t$. +- A specific particle, $\vec{\chi}$, and a point in space $\vec{x}$. + +Generated: Wed Dec 102025 16:30:22 GMT+0000 (Coordinated Universal Time) + diff --git a/tests/fixtures/me2/solutions.md b/tests/fixtures/me2/solutions.md new file mode 100644 index 0000000..2c85123 --- /dev/null +++ b/tests/fixtures/me2/solutions.md @@ -0,0 +1,359 @@ +## MECH50010 Fluid Mechanics 2 Problem Set \#1 + +## Introduction + +### 1.1 Hydraulic scale + +- 5-10 mins + +This is a gentle warm up question to get into the swing of things after a long summer. + +(a) Final answer +$$ +m=0.106 \mathrm{~kg} +$$ + +Worked solutions +The weight of the piston applies a pressure $p=4 m g /\left(\pi D^{2}\right)$. This pressure adds to the atmospheric pressure, so that the pressure on the mercury on the left-hand side of the tube is $p+p_{a t}$. Applying the hydrostatic equations (or Bernoulli's equations for steady, inviscid and irrotational fluids) we find: + +- $$ +p+p_{\mathrm{at}}=p_{\mathrm{at}}+\rho_{\mathrm{w}} \sigma_{\mathrm{Hg}} g h +$$ +where $\sigma$ is 'specific gravity'(density relative to water). + +Hence: + +- $$ +\begin{aligned} +\frac{4 m g}{\pi D^{2}} & =\rho_{\mathrm{w}} \sigma_{\mathrm{Hg}} g h \\ +m & =\frac{\pi}{4} \rho_{\mathrm{w}} \sigma_{\mathrm{Hg}} h D^{2} \\ +& =\frac{\pi}{4} * 1000 * 13.54 * 10^{-3} * 10^{-2} \\ +& =0.106 \mathrm{~kg} +\end{aligned} +$$ +(b) + +Final answer +Selecting a fluid with a smaller specific gravity. + +Worked solutions +To improve the accuracy of the measurement, $D$, being fixed by the piston's size, it is best to makehbigger (this is what is read), which translates in selecting a fluid with a smaller specific gravity. + +### 1.2 Friction on a plate + +- ★ 15-20 mins + +This question is to refresh yourself using control volumes, which were used in ME1 and will be used often in ME2. + +(a) + +Final answer + +$$ +\dot{m}=30 \mathrm{~kg} / \mathrm{s} +$$ + +Worked solutions +From mass conservation: + +$$ +-\dot{m}_{a d}=\dot{m}_{a b}+\dot{m}_{b c}- +$$ + +This becomes: + +$$ +-\rho W \int_{0}^{\delta} U_{0} \mathrm{~d} y=\dot{m}_{a b}+\rho W \int_{0}^{\delta} u(y) \mathrm{d} y- +$$ + +Since we know that $u(y)$ is linear, we can express it in terms of $y$ and $U_{0}$ : + +$$ +u(y)=U_{0} \frac{y}{\delta}, +$$ + +Therefore: + +$$ +\begin{aligned} +\dot{m}_{a b} & =\rho W U_{0}\left(\int_{0}^{\delta} 1-\frac{y}{\delta} \mathrm{~d} y\right) \\ +\dot{m}_{a b} & =\rho W U_{0} \frac{\delta}{2} \\ +\dot{m}_{a b} & =(800)(1)(3) \frac{25 \times 10^{-3}}{2} . \\ +& \dot{m}=30 \mathrm{~kg} / \mathrm{s} +\end{aligned} +$$ + +(b) + +Final answer + +$$ +F_{\text {plate }}=30 \mathrm{~N} +$$ + +Worked solutions +From momentum conservation: + +$$ +F_{\text {fluid }}=M_{a b}+M_{b c}-M_{a d} +$$ + +Since we are calculating the horizontal resultant force, we need to consider the horizontal momentum in the section "ab"; this means that we ignore any vertical velocities. Since the velocity of the fluid at $y=\delta$ (i.e. at "ab") is always $U_{0}$, we can say: + +$$ +M_{a b}=U_{0} \dot{m}_{a b}, +$$ + +We then also convert the other momentum flowrate terms into their mathematical forms, and proceed to find the force of the plate on the fluid, as shown below: + +$$ +F_{\text {fluid }}=M_{a b}+\rho W \int_{0}^{\delta} u(y)^{2} \mathrm{dy}-\rho W \int_{0}^{\delta} U_{0}^{2} \mathrm{dy} +$$ + +$$ +F_{\text {fluid }}=U_{0} \dot{m}_{a b}+\rho W U_{0}^{2}\left(\int_{0}^{\delta} \frac{y^{2}}{\delta^{2}}-1 \mathrm{dy}\right) +$$ + +$$ +F_{\text {fluid }}=(3)(30)+(800)(1)(3)^{2}\left[-\frac{2}{3}\left(25 \times 10^{-3}\right)\right] +$$ + +$$ +F_{\text {fluid }}=-30 \mathrm{~N} +$$ + +However, the question asks us to find the drag force of the fluid on the plate, hence: + +$$ +F_{\text {plate }}=-F_{\text {fluid }} +$$ + +$$ +F_{\text {plate }}=30 \mathrm{~N} +$$ + +### 1.3 Towing a submarine + +- 15-20 mins + +This is another revision question from ME1, for those who need extra practice. +(a) + +Final answer + +$$ +F=\frac{\pi}{6} \rho U^{2} R^{2} +$$ + +Worked solutions +The mass flow rate entering the control volume is + +$$ +\dot{m}_{\mathrm{in}}=\rho U A_{\mathrm{disk}}=\rho U \pi R^{2} +$$ + +- + +(since the velocity is uniform). The mass exiting the control volume on the left (the wake) is: +- + +$$ +\dot{m}_{\text {wake }}=2 \pi \rho \int_{r=0}^{r=R} r(U r / R) \mathrm{d} r +$$ + +Hence, the mass flow rate leaving the control volume through the side $\left(\dot{m}_{\text {side }}\right)$ is: + +$$ +\begin{aligned} +\dot{m}_{\text {side }} & =\dot{m}_{\text {in }}-\dot{m}_{\text {wake }} \\ +& =\rho U \pi R^{2}-2 \rho U \pi \int_{r=0}^{r=R} \frac{r^{2}}{R} \mathrm{~d} r \\ +& =\rho U \pi R^{2}-2 \rho U \pi\left[\frac{1}{3} \frac{r^{3}}{R}\right]_{r=0}^{r=R} \\ +& =\rho U \pi R^{2}-\frac{2}{3} \rho U \pi R^{2} \\ +& =\frac{1}{3} \rho U \pi R^{2} +\end{aligned} +$$ + +Force-momentum equation (FME): + +The momentum flowrate entering the control volume is $M_{\text {in }}=\dot{m}_{\text {in }} U$. The momentum flowrate exiting the control volume on the left (the wake) is +- + +$$ +M_{\text {wake }}=2 \pi \rho \int_{r=0}^{r=R} r(U r / R)^{2} \mathrm{~d} r +$$ + +- + +The mass leaving the control volume by the side is also contributing to the removal of momentum, $M_{\text {side }}=\dot{m}_{\text {side }} U$. Hence, the momentum leaving the control volume is $M_{\text {out }}=M_{\text {wake }}+M_{\text {side }}$. + +The FME reads: +- + +$$ +M_{\text {out }}-M_{\text {in }}=-F+F_{p}, +$$ + +- + +where $F_{p}$ represents pressure forces. However, we assume the pressure to be unaffected by the presence of the submarine, which leaves the hydrostatic pressure force. Since the hydrostatic pressure has the same linear profile on both sides of the control volume (front and wake), it contributes nothing to the horizontal force. Hence: +- + +$$ +\begin{aligned} +-F & =2 \pi \rho \int_{r=0}^{r=R} r(U r / R)^{2} \mathrm{~d} r+\dot{m}_{\text {side }} U-\dot{m}_{\text {in }} U \\ +& =2 \rho U^{2} \pi \int_{r=0}^{r=R}\left(r^{3} / R^{2}\right) \mathrm{d} r+\frac{1}{3} \rho U^{2} \pi R^{2}-\rho U^{2} \pi R^{2} \\ +& =2 \rho U^{2} \pi\left[\frac{1}{4} \frac{r^{4}}{R^{2}}\right]_{r=0}^{r=R}-\frac{2}{3} \rho U^{2} \pi R^{2} \\ +& =\frac{1}{2} \rho U^{2} \pi R^{2}-\frac{2}{3} \rho U^{2} \pi R^{2} +\end{aligned} +$$ + +- + +Which gives: + +$$ +F=\frac{\pi}{6} \rho U^{2} R^{2} +$$ + +(b) + +Final answer + +$$ +P=\frac{\pi}{6} \rho U^{3} R^{2} +$$ + +Worked solutions +By definition, the power is $F . U$. Therefore: + +$$ +P=\frac{\pi}{6} \rho U^{3} R^{2} +$$ + +### 1.4 Molecules, particles, and continuum + +- ★ 20-25 mins + +This question bridges ME1 and ME2, exploring the continuum hypothesis and the definition of a fluid particle. +(a) + +Final answer + +$$ +n_{0} \approx 2.65 \times 10^{25} \mathrm{~m}^{-3} \text { (molecules per metre cubed) } +$$ + +To enter in the checker, for example: +$2.65 e 25 \mathrm{~m}^{\wedge}(-3)$ + +Worked solutions +The fluid density is directly related to the particle density: $\rho_{0}=n_{0} m / V$ (assuming all molecules to be the same). In this question, we take $V$ to be a cubic meter. Therefore, we need to calculate $\rho_{0}$ and $m$ (the mass of one molecule). + +The density can be computed from the ideal-gas law (the gas is assumed to be ideal): $\rho_{0}=p_{0} /\left(R T_{0}\right)$ where $R=\tilde{R} / M$. + +The mass is directly computed from the Avogadro number and the molecular mass: $m=M / \mathcal{N}_{A}$. + +Hence: + +$$ +n_{0}=\frac{p_{0} \mathcal{N}_{A} V}{\tilde{R} T_{0}}=\frac{\left(10^{5}\right)\left(6.02 \times 10^{23}\right)(1)}{(8.314)(273.15)} +$$ + +$$ +n_{0} \approx 2.65 \times 10^{25} \text { molecules per metre cubed } +$$ + +(b) + +Final answer + +$$ +\begin{aligned} +& \ell_{0} \approx 3.58 \times 10^{-7} \mathrm{~m} \\ +& 0.1 \mathrm{~mm} 0 then + local parts = {} + for _, sub in ipairs(nested) do + local sown = split_item(sub) + parts[#parts + 1] = '{"content": "' .. esc(render(sown)) .. '"}' + end + piece = piece .. ', "parts": [' .. table.concat(parts, ', ') .. ']' + else + piece = piece .. ', "parts": []' + end + out[#out + 1] = piece .. '}' + end + local json = '[' .. table.concat(out, ',\n') .. ']' + return pandoc.Pandoc({ pandoc.CodeBlock(json) }) +end \ No newline at end of file diff --git a/tests/test_routes.py b/tests/test_routes.py new file mode 100644 index 0000000..195ea8e --- /dev/null +++ b/tests/test_routes.py @@ -0,0 +1,190 @@ +"""The two-route conversion: direct call, filter per set, agreement, adjudication, flags. + +Written before the module, from the ME2 introduction pair: the questions PDF and the +solutions PDF Lambda Feedback printed from its own set (their Mathpix markdown is the +fixture), the set as the platform exported it, and one direct-route reply the model gave +for them on 2026-09-21. The PHYS sheet tests read a private corpus and skip without it. +""" + +import copy +import difflib +import json +import os +import re +import shutil +import subprocess +import zipfile +from pathlib import Path + +import pytest + +from conftest import FakeBackend + +import in2lambda_agent.routes as routes + +ME2 = Path(__file__).parent / "fixtures" / "me2" +QUESTIONS = (ME2 / "questions.md").read_text() +SOLUTIONS = (ME2 / "solutions.md").read_text() +REPLY = json.loads((ME2 / "direct.json").read_text()) +EXPORT = ME2 / "export" +PHYS = Path( + "/Users/peterbjohnson/code/lambdafeedback/in2lambda-agent/ExampleContents/" + "PHYS40002-Mechanics/problem_sheets_and_figures" +) +FILTER = Path(__file__).parent / "fixtures" / "ps1-filter.lua" + +live = pytest.mark.skipif(not os.environ.get("IN2LAMBDA_LIVE"), reason="calls Mathpix and a model") + + +def exported(): + return [json.loads(f.read_text()) for f in sorted(EXPORT.glob("question_*.json"))] + + +def ratio(a, b): + fold = lambda s: " ".join(routes.fold(s).split()) + return difflib.SequenceMatcher(None, fold(a), fold(b)).ratio() + + +# --- verbatim ------------------------------------------------------------------------- + + +def test_every_field_of_the_reply_is_a_quote_of_the_markdown(): + assert routes.not_verbatim(REPLY, QUESTIONS + "\n" + SOLUTIONS) == [] + + +def test_a_field_the_model_reworded_is_named(): + reworded = copy.deepcopy(REPLY) + reworded[0]["parts"][0]["content"] = "What is the mass of the piston, in kg?" + assert routes.not_verbatim(reworded, QUESTIONS + "\n" + SOLUTIONS) == ["q1.p1.content"] + + +def test_an_empty_field_is_not_a_quote_of_anything_and_is_not_flagged(): + empty = copy.deepcopy(REPLY) + empty[4]["parts"][0]["worked_solution"] = "" + assert routes.not_verbatim(empty, QUESTIONS + "\n" + SOLUTIONS) == [] + + +# --- tier 1: agreement ---------------------------------------------------------------- + + +def test_two_identical_replies_dispute_nothing(): + assert routes.disputed(REPLY, copy.deepcopy(REPLY)) == [] + + +def test_notation_that_renders_the_same_is_not_a_dispute(): + other = copy.deepcopy(REPLY) + text = other[0]["main_text"] + other[0]["main_text"] = text.replace(r"\mathrm{~m}", r"\space\mathrm{m}").replace("\\left(", "(").replace("\\right)", ")") + other[0]["parts"][0]["content"] = " " + other[0]["parts"][0]["content"].replace(" ", " ") + assert routes.disputed(REPLY, other) == [] + + +def test_different_wording_is_a_dispute_naming_the_field(): + other = copy.deepcopy(REPLY) + other[1]["parts"][1]["content"] = "Determine the drag force on the plate, in newtons." + assert routes.disputed(REPLY, other) == ["q2.p2.content"] + + +def test_a_part_one_route_did_not_find_is_a_structural_dispute(): + other = copy.deepcopy(REPLY) + del other[3]["parts"][2] + assert routes.disputed(REPLY, other) == ["q4.p3"] + + +# --- the set ---------------------------------------------------------------------------- + + +def test_the_reply_becomes_a_set_with_the_exports_questions(): + built = routes.to_set(REPLY, name="Introduction") + assert [q.title for q in built.questions] == [q["title"] for q in exported()] + assert [len(q.parts) for q in built.questions] == [len(q["parts"]) for q in exported()] + + +def test_the_zip_holds_one_file_per_question_with_the_exports_keys(tmp_path): + zip_path = routes.build(routes.to_set(REPLY, name="Introduction"), tmp_path / "out") + names = zipfile.ZipFile(zip_path).namelist() + assert sorted(n for n in names if n.startswith("question_")) == sorted(f.name for f in EXPORT.glob("question_*.json")) + written = json.loads(zipfile.ZipFile(zip_path).read("question_000_Hydraulic_scale.json")) + export_keys = set(exported()[0].keys()) + assert set(written.keys()) <= export_keys + assert set(written["parts"][0].keys()) <= set(exported()[0]["parts"][0].keys()) + + +def test_the_reply_matches_the_export_it_was_printed_from(): + built = routes.to_set(REPLY, name="Introduction") + for q, e in zip(built.questions, exported()): + assert ratio(q.main_text, e["masterContent"]) >= 0.9, q.title + for p, ep in zip(q.parts, e["parts"]): + assert ratio(p.text, ep["content"]) >= 0.6, (q.title, ep["content"][:40]) + # 0.5, not higher: Mathpix reads the platform's separator lines as minus signs + # inside the maths of the printed worked solutions (Towing a submarine, part 1). + expected = (ep.get("workedSolution") or {}).get("content", "") + if expected: + assert ratio(p.worked_solution, expected) >= 0.5, (q.title, "worked solution") + + +def test_the_options_of_a_multiple_choice_part_are_kept_apart_from_its_text(): + built = routes.to_set(REPLY, name="Introduction") + frames = built.questions[4] + assert "location of a particle" not in frames.parts[0].text + assert len(REPLY[4]["parts"][0]["options"]) == 4 + + +# --- tier 2: adjudication --------------------------------------------------------------- + + +def test_the_adjudicator_may_pick_one_side_and_its_pick_is_kept(): + other = copy.deepcopy(REPLY) + other[1]["parts"][1]["content"] = "Determine the drag force on the plate, in newtons." + backend = FakeBackend(json.dumps([{"field": "q2.p2.content", "choice": "A", "reason": "B adds words the source lacks"}])) + verdicts = routes.adjudicate(REPLY, other, ["q2.p2.content"], QUESTIONS + "\n" + SOLUTIONS, backend) + assert verdicts == {"q2.p2.content": ("A", "B adds words the source lacks")} + ((_, prompt),) = backend.calls + assert "Determine the drag force on the plate." in prompt and "in newtons" in prompt + assert len(prompt) < 4000 # the disputed field and its source lines, not the document + + +def test_the_adjudicators_own_words_are_refused_and_the_field_is_flagged(): + other = copy.deepcopy(REPLY) + other[1]["parts"][1]["content"] = "Determine the drag force on the plate, in newtons." + backend = FakeBackend(json.dumps([{"field": "q2.p2.content", "choice": "text", "text": "Find the drag on the plate.", "reason": "shorter"}])) + verdicts = routes.adjudicate(REPLY, other, ["q2.p2.content"], QUESTIONS + "\n" + SOLUTIONS, backend) + assert verdicts["q2.p2.content"][0] == "person" + + +# --- tier 3: the report ----------------------------------------------------------------- + + +def test_the_report_lists_only_what_a_person_must_read(): + other = copy.deepcopy(REPLY) + other[1]["parts"][1]["content"] = "Determine the drag force on the plate, in newtons." + del other[3]["parts"][2] + result = routes.reconcile(REPLY, other, QUESTIONS + "\n" + SOLUTIONS, backend=FakeBackend(json.dumps([{"field": "q2.p2.content", "choice": "A", "reason": "B adds words"}]))) + assert result.agreed >= 50 + assert [f.field for f in result.flags] == ["q4.p3"] + assert result.fields[1]["parts"][1]["content"] == REPLY[1]["parts"][1]["content"] + + +# --- route B: the filter ---------------------------------------------------------------- + + +@pytest.mark.skipif(not PHYS.is_dir() or shutil.which("pandoc") is None, reason="private corpus and pandoc") +def test_a_filter_written_for_the_set_reads_a_sheet_with_pandoc_alone(): + reply = routes.run_filter(FILTER, PHYS / "mechanics_23-24_PS1.tex") + assert len(reply) == 10 + assert [len(q["parts"]) for q in reply] == [0] * 9 + [5] + markdown = subprocess.check_output(["pandoc", str(PHYS / "mechanics_23-24_PS1.tex"), "-t", "commonmark_x", "--wrap=none"]).decode() + assert routes.not_verbatim(reply, markdown) == [] + + +# --- live ------------------------------------------------------------------------------- + + +@live +def test_the_me2_pair_converts_with_no_flag(tmp_path): + target = Path("ExampleContents/targets/ME2_Fluids_introduction") + (pdf,) = [p for p in target.glob("*.pdf") if "solutions" not in p.name] + (solutions,) = target.glob("*solutions.pdf") + result = routes.convert(pdf, solutions=solutions, out_dir=tmp_path / "out") + assert result.flags == [] + assert [q.title for q in result.set.questions] == [q["title"] for q in exported()]