From 177cdf62e8bba703ad0b3d8b99fbaa1817868ac7 Mon Sep 17 00:00:00 2001 From: wwq Date: Thu, 27 Aug 2026 14:16:14 +0800 Subject: [PATCH] Add FrontierCS algorithmic-problem-solving skill --- skills/task-oriented/FrontierCS/README.md | 80 + .../algorithmic-problem-solving/SKILL.md | 189 + .../references/heuristic-search.md | 288 + .../references/technique-selection.md | 135 + .../checker-and-local-evaluation/SKILL.md | 301 + .../contest-solver-engineering/SKILL.md | 202 + .../interactive-problem-solving/SKILL.md | 140 + .../model-and-route-algorithms/SKILL.md | 204 + .../sub-skills/plateau-escape/SKILL.md | 248 + .../SKILL.md | 148 + .../sub-skills/testlib-cpp-judging/SKILL.md | 146 + .../references/testlib-usage.md | 303 + .../references/troubleshooting.md | 73 + .../testlib-cpp-judging/scripts/testlib.h | 6252 +++++++++++++++++ .../validation-and-experiments/SKILL.md | 171 + 15 files changed, 8880 insertions(+) create mode 100644 skills/task-oriented/FrontierCS/README.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/heuristic-search.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/technique-selection.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/checker-and-local-evaluation/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/contest-solver-engineering/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/interactive-problem-solving/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/model-and-route-algorithms/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/plateau-escape/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/reactive-online-decision-problem-solving/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/SKILL.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/testlib-usage.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/troubleshooting.md create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/scripts/testlib.h create mode 100644 skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/validation-and-experiments/SKILL.md diff --git a/skills/task-oriented/FrontierCS/README.md b/skills/task-oriented/FrontierCS/README.md new file mode 100644 index 000000000..0c6015ca0 --- /dev/null +++ b/skills/task-oriented/FrontierCS/README.md @@ -0,0 +1,80 @@ +# Algorithmic Problem-Solving Recovery + +`algorithmic-problem-solving` is an evidence-driven recovery system for algorithmic, competitive-programming, interactive, online, and scored optimization tasks. It is not a template collection or a replacement for the first focused solution attempt. Once that attempt fails to achieve a verified full result, the skill preserves trusted work, identifies the failing layer, and selects the narrowest justified recovery route. + +```text +reproduce the failure -> recover the operational contract +-> classify the failing layer -> run the selected recovery route +-> validate a challenger independently -> promote the best legal champion +-> escalate structurally when local improvement has stalled +``` + +## Activation and routing + +The root `SKILL.md` is the recovery entry point. It activates after few non-full submissions, after the first non-full candidate for an interactive or scored heuristic/optimization task, before a non-full final delivery, or when evidence shows a correctness, resource, evaluator, protocol, or policy failure. It does not apply before a fresh problem's first focused attempt, and it stops after a verified full pass or maximum score. + +Sub-skills are targeted internal routes. The root selects them from the observed failure rather than loading every branch at once. Mandatory or paired routes remain additive when the task requires them. + +## Codex adaptation + +The repository does not ship product-specific `agents/openai.yaml` files, but a Codex adapter should allow implicit invocation only for the root router. Add or merge this policy into `algorithmic-problem-solving/agents/openai.yaml`: + +```yaml +policy: + allow_implicit_invocation: true +``` + +Every sub-skill should disable implicit invocation so Codex cannot bypass the root diagnosis and routing logic. For example, `algorithmic-problem-solving/sub-skills/interactive-problem-solving/agents/openai.yaml` should contain: + +```yaml +policy: + allow_implicit_invocation: false +``` + +Apply the same `false` policy to every other sub-skill. The snippets show only the invocation policy; no `default_prompt` is required for this routing design. These optional adapter files are intentionally omitted from the runtime structure below. + +## Structure + +```text +algorithmic-problem-solving/ +├── SKILL.md # Recovery router, escalation gates, artifact discipline, and final release rules +├── references/ +│ ├── heuristic-search.md # Quantified search budgets, representations, incremental evaluation, neighborhoods, and optimizers +│ └── technique-selection.md # Algorithm-family selection after the model and resource envelope are trusted +└── sub-skills/ + ├── checker-and-local-evaluation/ + │ └── SKILL.md # Independent checkers, scorers, interactors, simulators, generators, and local evaluation + ├── contest-solver-engineering/ + │ └── SKILL.md # Toolchain, numeric, memory, runtime, I/O, randomness, and implementation recovery + ├── interactive-problem-solving/ + │ └── SKILL.md # Protocol modeling, hidden hypotheses, information-gaining queries, and transcript validation + ├── model-and-route-algorithms/ + │ └── SKILL.md # Contract/model repair, proof obligations, feasibility analysis, and solution-class selection + ├── plateau-escape/ + │ └── SKILL.md # Independent structural review, method research, executable challengers, and evidence-gated promotion + ├── reactive-online-decision-problem-solving/ + │ └── SKILL.md # Estimation, planning, exploration, feedback updates, and risk-aware sequential decisions + ├── testlib-cpp-judging/ + │ ├── SKILL.md # C++ checkers, validators, deterministic generators, interactors, and local judging flow + │ ├── references/ + │ │ ├── testlib-usage.md # Testlib roles, APIs, verdicts, templates, and command-line contracts + │ │ └── troubleshooting.md # Compilation, arguments, strict input, status, reproducibility, and protocol diagnostics + │ └── scripts/ + │ └── testlib.h # Bundled single-header Testlib dependency + └── validation-and-experiments/ + └── SKILL.md # Falsifying tests, independent oracles, paired comparisons, holdouts, and release gates +``` + +## Operating principles + +- **Recover the real contract first.** Read the statement and relevant executable artifacts, separate legality from objective and displayed score, and test any dependency on conflicting interpretations. +- **Classify before editing.** Distinguish model/proof errors, route-selection errors, implementation failures, evaluator uncertainty, weak experimental evidence, search-mechanics problems, information-acquisition failures, and reward-bearing sequential decisions. +- **Select methods from proven premises.** Use the technique catalog only after the model and feasibility envelope are trusted. Every reduction, optimized recurrence, advanced structure, or incomplete route needs an explicit proof or falsification target. +- **Quantify scored search.** Design the representation, invariants, incremental evaluator, reachable neighborhoods, and useful-event rate before choosing an optimizer. Keep `current_state` separate from `best_valid_state`. +- **Build local evaluation only when it is diagnostic.** A missing evaluator or a first non-full candidate is not sufficient by itself. Construct one when concrete legality, score, protocol, replay, or official/local disagreement makes it useful. Structural plateau recovery has its own stricter evaluator gate. +- **Separate interactive and reactive work.** Interactive routing asks what information to acquire; reactive routing chooses reward-bearing actions whose live feedback changes later decisions. Offline repeated evaluation is neither by itself. +- **Preserve artifact roles.** A `fallback` is the simplest guaranteed-valid emergency output, a `challenger` is experimental, a `champion` is the best independently validated legal artifact, and a `baseline` is an external evaluation reference. +- **Promote mechanically from evidence.** A challenger replaces the champion only when it remains legal, respects resource and protocol limits, and improves comparable correctness or scoring evidence. +- **Escalate structure instead of tuning indefinitely.** Once the root's plateau or severe-gap gate is met, freeze the champion, run isolated trajectory review and method research, implement the leading structural routes, and bind adoption or rejection to reproducible results. + +Validation starts from the smallest faithful reproducer and expands only as needed through boundary cases, tiny brute-force oracles, differential and metamorphic tests, evaluator self-tests, paired seeds, holdouts, resource probes, and clean release runs. Only experiments that were actually executed count as evidence. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/SKILL.md new file mode 100644 index 000000000..eb3746f6f --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/SKILL.md @@ -0,0 +1,189 @@ +--- +name: algorithmic-problem-solving +description: >- + Recover solutions after a focused attempt lacks a verified full pass or + maximum score. MUST invoke and read this skill (1) after 2 non-full + submissions, even if high-scoring; (2) after the first non-full candidate for + any interactive or scored heuristic/optimization task, including AHC; and (3) + before any non-full final delivery. Skip only after verified full pass/full + score. For (2), route to an independent local interactor/checker/scorer only + on concrete evaluator, legality, scoring, protocol, or diagnostic evidence, + or when Plateau Escape requires its phase-local gate. Coherent accepted online + submissions may continue, deferring local data/evaluation until it can expose + a problem. When routed and delegation exists, assign a fresh-context evaluator + owner and give it originals rather than solver reasoning. Also use + for WA, TLE/MLE, crashes, invalid output, evaluator disagreement, score gaps, + stalled heuristics, or interactive/reactive failures. Do not use before a + fresh problem's first focused attempt. +--- + +# Algorithmic Problem-Solving Recovery + +Diagnose the failed layer and preserve verified work. Outside Plateau Escape, make the smallest evidence-backed change that can recover a complete or materially stronger solution. Once Plateau Escape is routed, exempt structural work from the smallest-change preference and maximize evidence-weighted expected terminal official score. + +## Positioning + +This is a recovery router, not the default way to solve every contest problem. The focused end-to-end solve is an activation prerequisite and happens outside this skill. If no concrete failure evidence exists, leave this workflow and solve the problem directly. The post-first-candidate rule for interactive and scored heuristic/optimization tasks activates the relevant search or policy recovery guidance; it does not by itself trigger local evaluator construction. Treat local data generation and evaluation as problem-finding tools that may be deferred while accepted online submissions return coherent legality, protocol, and score evidence. + +Once activated, do not restart reflexively. Reconstruct the actual contract and the failed attempt from artifacts, then reuse every verified component that is not implicated by evidence. + +Use artifact terms consistently: + +``` +fallback simplest guaranteed-valid emergency output +challenger experimental candidate evaluated against the champion +champion best independently validated legal artifact +baseline external scoring/evaluation reference, never a solver artifact +``` + +## Recovery snapshot + +Collect only the facts needed to reproduce and localize the failure: + +``` +statement, bounds, and operational evaluator contract +attempted model, algorithm, proof assumptions, and complexity +source/build/run commands and target toolchain +smallest failing input, transcript, or scored instance +expected versus observed result, verdict, score, runtime, and memory +available checker, scorer, interactor, simulator, logs, and submission budget +champion/fallback/challenger paths and remaining uncertainty +``` + +Read every supplied artifact that is directly relevant to the failing layer. Resolve prose/config/evaluator discrepancies explicitly. Prefer their legal intersection when it has no material cost; otherwise record and test any dependency on the operational evaluator. + +## Classify before changing code + +| Evidence | Failing layer to test first | Route | +|---|---|---| +| Contract meaning, state, invariant, recurrence, reduction, or proof is questionable, or brute force finds a mismatch | Problem model and correctness | [model and route algorithms: model repair](sub-skills/model-and-route-algorithms/SKILL.md#1-normalize-the-operational-contract) | +| The model is trusted, but the exact/constructive/hybrid/heuristic boundary is unclear under the real bounds | Feasibility and route choice | [model and route algorithms: feasibility](sub-skills/model-and-route-algorithms/SKILL.md#5-establish-the-operation-and-memory-envelope) | +| The route is known, but its complexity class, algorithm-family premise, or abstract data-structure choice is wrong | Algorithm selection | [technique selection](references/technique-selection.md) | +| The official evaluator is absent or doubtful and that uncertainty blocks a decision; zero/invalid/WA feedback or an impossible score is unexplained; local and remote results disagree; legality, objective, score, simulator, protocol, or episode replay needs independent reconstruction; or Plateau Escape requires its phase-local evaluator gate | Evaluator contract and independent oracle | [checker and local evaluation](sub-skills/checker-and-local-evaluation/SKILL.md) | +| The evaluator is trusted, but oracle choice or counterexample search is unclear, comparisons are noisy, or a challenger needs paired, holdout, metamorphic, or other independent evidence before promotion | Experiment design and promotion evidence | [validation and experiments](sub-skills/validation-and-experiments/SKILL.md) | +| The route is justified, but compilation, crash/UB, overflow, memory layout, TLE/MLE, buffering, solver-side serialization, randomness, deadline handling, or an implemented data-structure invariant fails | Implementation and environment | [contest solver engineering](sub-skills/contest-solver-engineering/SKILL.md) | +| Any non-full AHC or other scored heuristic/optimization task has a legal candidate, whether offline or online; or its construction, representation, moves, deltas, reachability, evaluation, or optimizer behavior is weak | Mandatory scored-search baseline and search mechanics | [heuristic search](references/heuristic-search.md) | +| A verified legal scored champion has stopped improving or remains severely below the meaningful score target | Structural quality escalation | [scored-recovery escalation](#escalate-weak-scored-recovery) | +| The main value of an action is to obtain information and shrink hidden hypotheses | Protocol, query design, inference, adversarial feedback | [interactive problem solving](sub-skills/interactive-problem-solving/SKILL.md) | +| A reward-bearing decision occurs in a live sequential process and new observations or feedback can change later choices | Estimation, planning, exploration, sequential feedback | [reactive online decision problem solving](sub-skills/reactive-online-decision-problem-solving/SKILL.md); for AHC or another scored heuristic task, pair it with [heuristic search](references/heuristic-search.md) | + +Choose the primary failing-layer route, but treat mandatory and paired routes as additive rather than exclusive. Read every document named by the selected route before acting. In particular, a non-full AHC always adds Heuristic Search, and a live reward-bearing AHC adds Reactive without replacing Heuristic Search. Resolve an upstream contract, evaluator, legality, or implementation contradiction before tuning through a downstream route. A symptom may move to another row after one falsifying test; update the diagnosis instead of stacking unrelated fixes. + +Model and Route Algorithms has two internal entry points: begin with model repair when correctness evidence is unresolved, and enter at its feasibility envelope only when the model and proof obligations are already trusted. + +Technique Selection owns whether a data structure or algorithm family matches the required operations and bounds. Contest Solver Engineering owns whether the chosen implementation maintains its invariants. Checker and Local Evaluation owns what submitted output means and independently parses legality and score; Contest Solver Engineering owns solver-side formatting, index conversion, buffering, and flush behavior. + +Use the recovery loop below for a correction with a known deterministic failing test and an obvious focused regression. Route to Validation and Experiments when choosing or constructing the evidence is itself material, stochastic variance can reverse promotion, or independent promotion/release evidence is still missing. + +The checker/local-evaluation sub-skill owns evaluator architecture, independent implementation, and adversarial validation of the evaluator itself. For concrete evaluator code based on `testlib.h`, pair it with [testlib C++ judging](sub-skills/testlib-cpp-judging/SKILL.md). + +## Evidence-triggered local evaluation + +Route to [Checker and Local Evaluation](sub-skills/checker-and-local-evaluation/SKILL.md) only when at least one concrete signal makes evaluator work useful: + +- an official evaluator is absent or doubtful and its behavior now blocks diagnosis, promotion, or release; +- an accepted-looking candidate receives unexplained invalid, zero, WA, discontinuous, or impossible feedback; +- local and official results, repeated official results, or prose and executable evaluator behavior disagree beyond established randomness; +- legality, raw objective, score transformation, simulator transition, or episode replay must be independently reconstructed to distinguish the next hypotheses; +- an interactive run shows a protocol symptom such as deadlock, missing flush, premature EOF, timeout, query-budget disagreement, or unexpected termination; +- final delivery retains material evaluator, protocol, legality, or score uncertainty that coherent official evidence has not resolved; or +- an interactive or scored heuristic/optimization task enters Plateau Escape, whose repeated challenger comparison requires its own phase-local independent evaluator gate. + +Do not route solely because the task is interactive, scored, heuristic, or AHC; because its first candidate is non-full; because no local evaluator exists; or because another submission is planned. If official submissions are accepted, protocol and legality remain stable, scores and diagnostics are coherent with the known contract, and the open question is algorithmic quality, continue through the model, Interactive, Reactive, Heuristic Search, or Plateau admission route as applicable. Defer local construction until it can find a suspected problem or discriminate competing explanations. + +Once routed, start with the smallest artifact and data that can reproduce or falsify the signal. Build a broader generated campaign only when hand-computed cases, one failing transcript, or direct official evidence cannot localize it. Follow Checker and Local Evaluation's complete `coverage strategy -> parameterized generator -> validator -> evaluator -> seed/failure retention` workflow; when using `testlib.h`, pair it with [testlib C++ judging](sub-skills/testlib-cpp-judging/SKILL.md). Batch, randomized, or maximum-scale data must come from reproducible generator code rather than hand-authored large files. + +When routed evaluator work requires implementation or independent validation and delegation is available, assign ownership to a dedicated fresh-context sub-agent that did not author the solver or challenger; do not reuse it as a challenger owner, trajectory critic, method researcher, or other method reviewer for the same task. Give it the verbatim original statement and official original artifacts first, plus only necessary verified toolchain and access facts. Withhold solver internals, solver-derived formulas, main-agent diagnoses, method preferences, estimated scores, and unverified summaries until it has frozen the evaluator contract and initial fixtures. If delegation is unavailable, perform a sealed independent pass with separate files, representations, derivations, and fixtures. + +Plateau Escape retains the stricter phase-local rule: for an interactive or scored heuristic/optimization task, it must implement and validate any missing evaluator artifact before its reviewers compare structural challengers. This exception is triggered by the verified plateau or severe score gap, not by the first non-full candidate. + +## Additive AHC routing + +Every non-full AHC or equivalent scored heuristic/optimization task MUST read Heuristic Search before further optimization, another submission, Plateau Escape, or final delivery. This requirement does not imply a Checker and Local Evaluation route without one of the concrete signals above. If live observations or feedback can change later reward-bearing actions during the same judged execution, it MUST additionally read Reactive Online Decision Problem Solving. These requirements are additive; choosing a primary failing layer does not waive either one. + +Apply the routes as follows: + +``` +offline, complete input, one final output -> Heuristic Search +live observations/feedback change later reward-bearing actions -> Reactive Online Decision Problem Solving -> Heuristic Search +separable query stage primarily reduces hidden hypotheses -> Interactive Problem Solving for that stage; keep the applicable Heuristic and/or Reactive route for score-bearing actions +``` + +Heuristic Search is the mandatory AHC baseline. It owns representation, construction, neighborhoods, rollout/search mechanics, incremental evaluation, reachability, optimizer choice, and time allocation. For a live online AHC, Reactive owns the outer observable/latent state, estimator, planner, explorer, feedback update, horizon, and risk policy; then read Heuristic Search for the concrete action-generation or inner-search machinery. Do not let either document substitute for the other. + +Do not add Reactive merely because the output encodes a long action sequence or compact policy, the scorer simulates turns or randomness after receiving a fixed output, or development repeatedly calls a local/remote evaluator. If the submitted program cannot observe an outcome and adapt its next action during the same judged execution, keep the task on the offline Heuristic route. Once a non-full AHC candidate or judged result exists, its AHC label mandates Heuristic Search, but does not mandate Reactive by itself. + +## Distinguish Interactive from Reactive + +Use this ownership rule when a task has feedback: + +``` +Interactive: understand protocol -> choose query -> update hypotheses -> recover a sufficiently determined hidden answer +Reactive: observe -> estimate -> choose a legal action -> receive reward -> update policy until the horizon/task/budget ends +``` + +If an action primarily eliminates candidate hidden states, route that stage to Interactive. If it must earn reward while learning from live feedback that changes later decisions in the same run or environment, route the overall policy to Reactive. For any AHC or scored heuristic task, keep Heuristic Search as the additive search-mechanics route in either case. A precommitted action sequence, an offline policy artifact, simulated turns after fixed output, stochastic execution without observable feedback, and repeated evaluation across development runs do not make a task Reactive. + +## Evidence-driven recovery loop + +1. Reproduce the failure with the smallest faithful command, input, transcript, or fixed seed available. +2. State one falsifiable diagnosis at the model, algorithm, implementation, evaluator, protocol, or policy layer. +3. Design the smallest discriminating test. Prefer a tiny brute oracle, hand-computed boundary, invariant assertion, transcript replay, profile, or paired champion/challenger comparison. +4. Change one implicated component. Keep the champion untouched and materialize risky work as a challenger. +5. Compile and run the targeted test, then the relevant regression set. Never claim an experiment that was not run. +6. Promote only a legal challenger that fits the resource/protocol budget and improves the required correctness or fixed scoring evidence. +7. Repeat from the new evidence; do not tune a downstream layer while an upstream contract or validity contradiction remains. + +For exact work, demand a corrected proof obligation and try to falsify it on tiny exhaustive cases. For scored work, keep the true raw objective authoritative and verify cached deltas against full recomputation. For stochastic work, compare paired seeds/cases and expand the sample only when variance can reverse the decision. + +## Escalate weak scored recovery + +Apply this gate only after the champion is legal under trusted operational evidence and fits the resource budget. A coherent accepted official result may establish legality for admission without first constructing a local evaluator. Read [Plateau Escape](sub-skills/plateau-escape/SKILL.md) when either signal holds; for an interactive or scored heuristic/optimization task, Plateau Escape will then complete its phase-local independent evaluator gate before comparing challengers: + +- **Measured plateau:** at least two materially different legal challengers, or at least 3 controlled submissions, fail to improve the champion beyond a predeclared threshold above evaluator/seed/runtime noise. Use `1%` of a normalized score scale as the default threshold when such a scale exists. +- **Severe score gap:** after the focused recovery loop, the best legal champion is still below `70%` of a meaningful official maximum or explicit target. + +A controlled comparison uses the same evaluator, comparable budgets, and paired fixed cases/seeds when applicable. A materially different challenger changes the representation, objective or value model, neighborhood reachability, planning horizon, decomposition, relaxation, or algorithm family; changing only a seed, scalar parameter, tie-break, operator percentage, or runtime allocation does not count. + +A single low score is not by itself a measured plateau, but the below 70% rule is an independent mandatory escalation because the remaining gap is too large to justify stopping at local tuning. After the focused recovery loop, this root-level severe-gap signal satisfies Plateau Escape's admission gate. If the score has no meaningful maximum or explicit target, do not invent a halfway point; use the measured-plateau signal instead. Route invalid or contradictory evaluation to Checker and Local Evaluation, implementation/resource failures to Contest Solver Engineering, unclear comparisons to Validation and Experiments, and ordinary search-mechanics weaknesses to Heuristic Search before escalating. + +When escalation applies, preserve the champion and run Plateau Escape's two fresh-context sub-agent roles when delegation is available. Give the trajectory critic the original problem artifacts and verified facts, then the separately labeled attempt log only after it freezes an independent reconstruction. Give the method researcher the complete original problem and only verified necessary contract, budget, target, and evaluation facts; withhold the champion, attempt history, current method labels, diagnoses, risk labels, rejected-method rationales, and the other review until its independent research map is sealed. If delegation is unavailable, perform the two sealed passes and preserve the same information barrier. + +Treat sub-agent work as an executable evidence track, not optional advice: + +- When remote submission is authorized and budget remains, delegate an explicit quota and reserve at least one submission for the method researcher; when the budget permits, also allocate a separate multi-submission iteration quota to each selected rank-1 critic/research track. Prioritize those iterations over easier lower-ranked methods or retaining unused budget for the main agent. Both reviewers should implement, evaluate, improve, and submit their own isolated challengers instead of returning them for subjective main-agent screening. If only the main agent can access the transport, relay each exact sub-agent artifact unchanged and return the raw result for every iteration. +- Encourage both roles to explore aggressively across high-upside representations, objectives, decompositions, neighborhoods, planning horizons, relaxations, and algorithm families, including methods that require a substantial rewrite. Aggressive exploration never waives legality, evaluator fidelity, final judge limits, or submission authorization. +- Require both roles to freeze rankings by the evidence-weighted expected terminal final official aggregate score achievable under the verified remaining hard time, compute, and submission budget. Implementation difficulty, code size, rewrite scope, familiarity, prototype speed, or ability to finish within one sub-agent turn is not a ranking criterion. A turn boundary is not a hard scope limit: continue the same isolated rank-1 owner through follow-up tasks when supported. Task each role to implement its rank-1 applicable method, start with that method's smallest faithful structural prototype, and iteratively repair and improve it through the delegated evaluation/submission quota; `smallest` scopes the first experiment, not the method choice. +- Process both frozen rankings in order. Do not select a lower-ranked method until every higher-ranked method has Plateau Escape's required recorded elimination evidence. Every leading proposal must end as iteratively evaluated, queued for a named test, or rejected with a proof, primary contract citation, failing artifact, or measurement that matches Plateau Escape's rejection gate. +- The main agent may not use `risk too high`, unfamiliarity, code size, restructuring cost, or an unmeasured estimate as a veto. Under the same established final evaluator, a legal challenger with a better exact terminal final score and passing hard constraints is promoted mechanically; opinion cannot override that result. + +Do not stop after collecting reports, defer all experiments back to the main agent, or resume scalar tuning before each selected rank-1 structural track reaches Plateau Escape's iteration stop condition or concrete rejection evidence. + +## Guardrails + +- Separate legality from quality and raw objective from displayed score. +- For scored tasks, optimize the established final official aggregate score; use passed-case count or a proxy only when proven equivalent. +- Never let a failed or weaker challenger replace the champion. +- Preserve a legal fallback when interruption, query limits, or deadlines could otherwise erase all value. +- Isolate checker/scorer/interactor logic from solver-derived formulas when evaluator fidelity is in doubt. +- Prefer the target language for solver changes. For a new checker, scorer, interactor, simulator, or evaluation harness, strongly prefer target-supported C++ and portable C++17 when unspecified; provide exact build and run commands. +- Use research when the diagnosed gap requires external facts or another method family. Treat findings as hypotheses that must pass the same contract, compile, validity, objective, and resource gates as local ideas. +- Do not submit, overwrite a champion, or spend scarce evaluation budget unless the requested workflow authorizes it. Once an explicit quota is delegated, a sub-agent may spend that quota without repeated approval but still may not mutate the champion. + +## Finish + +Rebuild the selected champion cleanly and rerun the focused regression, final serialization or transcript validation, and representative resource probe. If Checker and Local Evaluation was evidence-triggered or Plateau Escape completed its phase-local evaluator gate, rerun that independent evaluator and report its exact command and result. Otherwise, coherent accepted official evidence may remain the evaluation record; do not delay delivery merely to manufacture a local campaign with no unresolved problem hypothesis. Deliver the champion rather than the latest edit. + +Before accepting any scored result below 70% of a meaningful official maximum or explicit target, read and execute Plateau Escape. Use its fresh-context sub-agents with role-specific evidence packets, let every selected rank-1 critic/research track complete its authorized iteration loop and stop condition, and apply the evidence-bound promotion or rejection rule; do not finish with only advice, an unevaluated recommendation, a subjective main-agent veto, or another round of minor parameter tuning. + +An official score of `0`, or below `30%` of a meaningful maximum/target, is an immediate plateau: do not finish; invoke Plateau Escape and MUST run both fresh-context roles (or both sealed passes). Discard prior main-agent diagnoses, rankings, and rejections except independently verified facts. Adopt their common rank-1 route; if they disagree, execute both. The main agent may not veto; run each to its stop/rejection gate and promote only legal measured champions. + +Report only: + +``` +root cause and decisive evidence +implemented correction or promoted challenger +commands/tests actually run and their results +sub-agent submission ids, per-iteration scores, and terminal score when applicable +remaining assumptions or material risk +``` diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/heuristic-search.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/heuristic-search.md new file mode 100644 index 000000000..0e1ade18f --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/heuristic-search.md @@ -0,0 +1,288 @@ +# Quantified Heuristic Search + +Use this reference when a legal scored attempt has exposed a concrete weakness in construction, representation, moves, evaluation, or optimizer behavior. The central question is not which metaheuristic sounds sophisticated. It is whether the representation, moves, evaluation, and useful-event count form an informative reachable process within the time limit. + +If the evidence already meets the admission criteria for [plateau escape](../sub-skills/plateau-escape/SKILL.md), stop scalar tuning and use that sub-skill before selecting another representation or method family. Use this reference for concrete search mechanics and for testing the structural hypotheses returned by the plateau review. + +## 1. Search budget + +Estimate costs before selecting an optimizer: + +``` +C_init construction/restart cost +C_full full legality and true-score recomputation +C_move proposal + precheck + delta + accept/undo +C_repair repair or exact local solve cost and tail +T_search safety_factor * official_limit - I/O - final validation - overhead +N_total approximately (T_search - restarts * C_init) / mean(C_move) +``` + +Measure representative kernels. Include copying, allocation, cache misses, hashing, RNG, timer calls, logging, and variable-cost tails. If the timer is checked every `B` moves, bound overshoot by `B * worst_relevant_move_time`. Reserve more margin for slow judges, shared hosts, interactive I/O, recursion, repairs, or exact subsolves. Never consume the official limit exactly. + +Raw search size alone is not actionable: + +``` +S_raw = product of variable domain sizes +S_effective = S_raw after constraints, symmetry, decomposition, and reachability +``` + +Compare the number and diversity of states that can actually be evaluated with `S_effective` and with the basin structure implied by the moves. + +## 2. Quantify useful events + +For an independent per-trial target probability `p`: + +``` +P(at least one hit in N trials) = 1 - (1-p)^N +N needed for probability q = ceil(log(1-q) / log(1-p)) +``` + +For small `p`, a 99% hit probability needs about `N*p >= 4.6`. Do not multiply probabilities blindly when events are dependent. Use conditional reasoning, bounds, or pilot measurements. + +Instrument the transition funnel per move type: + +``` +p_valid = valid proposals / proposals +p_accept = accepted / valid +p_structural = objective-relevant changes / accepted +p_best = best updates / structural changes +useful/s = proposals/s * p_valid * p_accept * p_structural +gain/s = proposals/s * p_valid * p_accept + * E[max(true_delta, 0) | accepted] +``` + +`gain/s` intentionally does not multiply by `p_structural`: its expectation is over all accepted moves, including zero-gain non-structural moves. If the expectation is estimated only over structural accepted moves, multiply by `p_structural` and condition the expectation on both accepted and structural moves. + +Also record representative move cost, delta distribution, distinct-state/hash rate, and best-update time. Interpret low rates causally: + +- low validity: proposal ignores constraints or representation is wrong; +- high acceptance but little structural change: move edits irrelevant tokens; +- truth stalls while proxy rises: surrogate mismatch; +- more time gives no gain: neighborhood ceiling or disconnected reachability; +- occasional huge gain with bad median: risky heavy-tail policy needing explicit score-system justification. + +For unknown hit rates, pilot `m` comparable trials. If zero hits occur, `3/m` is a common rough 95% upper bound for `p`; if even that gives `N_budget*p << 1`, redesign rather than hoping. + +## 3. Representation and invariants + +Map the objective to its natural combinatorial object: + +``` +routes rather than visited flags +machine/task sequences rather than isolated start times +components/cuts/matchings rather than unrelated labels +blocks/segments rather than individual permutation tokens +regions/boundaries/spatial adjacency rather than full-grid rescans +``` + +Model constraints at their real granularity. An edge conflict need not forbid a whole vertex; one occupied interval need not block a full day; independent channels/capacities should remain separate. + +Prefer: + +``` +valid state -> validity-preserving move -> valid state +``` + +Temporary infeasibility is justified only when violation is cheap and informative, repair is reliable and bounded, adaptive penalties return to feasibility, or infeasible states connect otherwise separated feasible regions. + +Maintain explicit roles: + +``` +current_state, current_search_value, current_true_score +best_valid_state, best_true_score +candidate move, affected set, delta, rollback log +primary structure, inverse indices, local contributions, constraint counts +``` + +### Explicit structure and destroy-and-repair + +When legality or score depends on a path, cycle, component, schedule, or other global object, keep that object as primary state instead of only low-level variables. Use one atomic compound move: + +``` +destroy a bounded substructure -> repair under fixed boundary constraints -> +validate the complete structure -> commit, otherwise exact rollback +``` + +For a path or cycle, for example, remove one bounded subpath and reconnect its preserved endpoints with a bounded search. This crosses coordinated barriers without forcing the optimizer through illegal or low-score scalar intermediate states. Bound repair cost and retain an exact rollback record. + +Remove equivalent states using canonical labels, fixed representatives, sorted interchangeable groups, symmetry-breaking directions, or hashes. Confirm that canonicalization preserves objective and reachability. + +## 4. Fallback and construction + +Retain the champion. If early termination can erase all value and no simple guaranteed-valid fallback exists, add one without replacing the champion. Construction experiments remain challengers. Candidate constructors include: + +- deterministic or randomized greedy; +- restricted candidate lists / GRASP; +- regret, marginal-gain, or gain-density insertion; +- conditional/importance/stratified sampling; +- divide-and-construct or component assembly; +- relaxation rounding plus bounded repair; +- extension of exact small solutions; +- warm start from a legal prior incumbent; +- multiple deliberately diverse starts. + +Measure construction time, validity, raw score, diversity, and score after a fixed improvement budget. The best raw start may not have the best improvement potential. Keep construction diversity tied to a hypothesis about different basins, not merely different RNG seeds. + +## 5. Incremental evaluation + +When the objective is a sum of local contributions and a move affects `A`: + +``` +delta = sum over i in A of (new_local_i - old_local_i) +sample -> identify affected data -> precheck -> apply/delta -> accept or undo +``` + +Implement `apply`, `undo`, `delta`, `validate`, and full true-score recomputation. During development, periodically assert: + +``` +cached score == full score +cached constraints == full constraints +undo(apply(state, move)) == original state +``` + +Target `O(1)`, `O(log n)`, or genuinely local `O(k)` transitions using inverse indices, prefix/Fenwick/segment structures, candidate lists, reusable buffers, delayed updates, and rollback logs. Avoid full-state copies, allocations, and full rescoring in the inner loop unless the state is demonstrably small. + +## 6. Neighborhoods and reachability + +A useful move should preserve or cheaply restore legality, change score-relevant structure, support fast delta/undo, and combine with other moves to reach meaningfully different states. + +Use multiple scales when the problem requires them: + +``` +small: change, swap, relocate, reverse, insert/remove, 2-opt +medium: block exchange, cycle/exchange chain, merge/split, subpath reroute +large: region rebuild, destroy-and-repair, restart, exact local reoptimization +``` + +Check reachability explicitly. Repeated legal moves may accidentally preserve parity, order, components, topology, or a hidden count. Simulated annealing cannot cross a boundary that no move can cross. Add the missing transition, perturbation, restart, temporary infeasibility, or exact local solve. + +Once a simple neighborhood reaches a local optimum, derive the exact delta of the smallest coordinated move and necessary conditions for positive gain. Those conditions often reduce a nominal quadratic or combinatorial scan to a small structural candidate set. Benchmark that stronger, problem-specific neighborhood and its gain per second before using tabu, annealing, or random walks to compensate for weak moves. + +Before adding another metaheuristic, freeze most decisions and ask whether the released local structure becomes a known subproblem. Use [technique selection](technique-selection.md) to choose and reject inner solvers rather than inventing a weak bespoke repair. Strong heuristic solvers often use: + +``` +outer stochastic search chooses what to release -> +bounded deterministic optimizer rebuilds that subproblem -> outer acceptance +``` + +For example, the outer search may release a task set and an inner DP may reschedule it. Bound the inner solver's time, reconstruct its decisions, and validate the combined solution. + +Use LNS when improvement requires coordinated edits, small legal moves cannot cross the relevant barrier, or single-variable changes usually break feasibility. Treat partial destruction and bounded reconstruction as one neighborhood: + +``` +select a related region -> remove/relax variables -> randomized or exact repair -> +validate and score the complete state -> accept/reject -> update best +``` + +Representative destroy-repair pairs include: + +- paths/orders: remove a segment, visits, or a spatial region, preserve the boundary, then reconnect or reinsert; +- groups/assignments: release several groups or merge components, then rebuild with greedy, matching, flow, MST, DP, shortest path, beam search, or a bounded exact solve; +- schedules: clear a time window, task batch, or conflict chain, then repack it with greedy, DP, or min-cost flow. + +Also consider fixing most variables and reoptimizing only one region. The destroyed variables may remain temporarily undecided, but repair must restore legality before acceptance. + +A strong default is LNS neighborhood generation with hill-climbing or simulated-annealing acceptance: + +``` +destroy a substantial related region -> rebuild it with a strong method -> score -> +accept if improving, or accept by SA probability -> retain the best valid state +``` + +Use ALNS when several meaningfully different destroy and repair operators are available. Keep an exploration floor and adapt operator or operator-pair selection from cost-normalized gain, best-update rate, search phase, and stagnation. Bound ruin size and repair time; retain a rare barrier-crossing operator when measurements show long-term value. + +Do not tune temperatures, ruin sizes, operator weights, or thresholds indefinitely. After bounded calibration fails to improve beyond measured noise, change the representation, neighborhood, decomposition, objective, or inner optimizer; parameter tuning should calibrate a sound method, not substitute for improving it. + +## 7. True objective, proxies, and penalties + +The true checker score is authoritative for the best state, final output, and experiments. A proxy or penalty may guide `current` only. + +Use a surrogate when the true score is sparse, discontinuous, or flat. Build it from interpretable partial progress such as completion, repair distance, connectivity, utilization, balance, or a bound gap. Then test alignment: + +- sample transitions and compare proxy and true delta signs/ranks; +- inspect extrema and boundary behavior; +- verify that improving one component cannot silently destroy a strict priority; +- normalize terms by robust observed delta scale; +- prefer lexicographic/staged priorities to unexplained giant weights. + +Constraint preference order: + +``` +feasibility-preserving representation +bounded deterministic repair +adaptive penalty with measured feasible rate +fixed penalty only with a proven dominating scale +``` + +## 8. Optimizer selection + +Choose after state, moves, evaluation, and budget are known. + +| Method | Good fit | Warning signs | +|---|---|---| +| Uniform/biased sampling | Complete candidates are very cheap and useful mass is measurable | Rare coordinated structure, high rejection | +| Randomized greedy / GRASP | Local signal exists but deterministic construction locks in | Candidate list lacks diversity or repair dominates | +| Hill climbing | Improving moves are common and neighborhood is strong | Plateaus, deep basins, unreachable coordinated edits | +| Multi-start / iterated local search | Starts are cheap/diverse and local convergence is fast | Every start reaches the same basin | +| Threshold / late acceptance | Controlled worsening helps without temperature model | Scale drifts or search becomes random walk | +| Simulated annealing | Worsening moves connect basins and delta scale is measurable | Weak/disconnected moves, too few iterations, proxy mismatch | +| Tabu / guided local search | Immediate reversals/cycles or repeated costly features dominate | Tenure/penalty overhead restricts useful moves | +| VNS / LNS / adaptive LNS | Coordinated changes are needed and repair is effective | Unbounded or low-quality repair | +| Beam search | Sequential construction with predictive partial score | Beam collapse, duplicates, large branching/memory | +| Evolutionary/population | Feasible crossover preserves useful building blocks | Children need wholesale repair; diversity collapses | +| MCTS | Sequential decisions, informative rollouts, reusable states | High branching, noisy rollout, few simulations | +| Exact local solve | Fixing most variables yields a tractable subproblem | Calls are too large/frequent or boundaries mis-modeled | + +Do not default to a population method when independent multi-start local search uses the same evaluations more effectively. + +Evaluate stochastic optimizers under the scoring rule that will actually be used. If one program run may launch `N` independent starts and retain the best, compare the empirical best-of-`N` distribution, not only single-run means. For continuous parameters, use a small designed sweep, random/low-discrepancy search, racing, or a suitable optimizer; keep a locked set and count every configuration tried so tuning noise is not mistaken for progress. + +## 9. Calibration + +Examples of parameter derivation: + +``` +SA accepts worsening delta -d with probability p at T: +T = -d / log(p) + +Geometric schedule at progress r in [0,1]: +T(r) = T0 * (Tf/T0)^r + +Restart target for per-run success p_s and K runs: +P(success) = 1 - (1-p_s)^K + +Beam work approximately steps * width * branching +Population work approximately population * generations +MCTS work approximately search_time / rollout_cost +``` + +Estimate `d` from observed loss quantiles and choose desired early/late acceptance. Recalibrate when representation, score normalization, or move mix changes. Do not copy fixed temperatures, move percentages, or ruin sizes across instances without scale normalization. + +## 10. Time policy and multiple instances + +Use a monotonic clock and an internal deadline. Time controls both stopping and policy: + +``` +early: diverse construction / larger moves / model learning +middle: main improvement process +late: intensification and safe local polish +final: validation and serialization of saved best +``` + +For several instances sharing a total budget, allocate by measured move cost, size, log search scale, gap to a bound or external scoring baseline, or pilot improvement rate. Retain I/O and worst-case slack. Define stagnation in both iterations and time when move costs vary. + +## 11. Failure-directed iteration + +| Evidence | First redesign | +|---|---| +| Invalid proposals or repair dominates | Constraint-aware proposal/state invariant | +| Full score/copy/allocation dominates | Local cache, delta, apply/undo, buffer reuse | +| Few true-score-changing moves | Natural representation or stronger compound move | +| Starts remain separated | Reachability move, perturbation, temporary infeasibility, restart | +| Proxy improves but truth does not | Rescale/rebuild proxy; stage or lexicographic objective | +| Normal acceptance but no best gain | Neighborhood or objective issue, not temperature | +| Best-only gains with worse median/tail | Risk policy, more robust construction, validation criterion | +| Runtime occasionally spikes | Cap repair/subsolve, reserve deadline, remove variable tail | +| Cache/rollback drift | Exact recomputation, property tests, simpler state update | + +State, moves, and evaluation usually dominate the choice of metaheuristic. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/technique-selection.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/technique-selection.md new file mode 100644 index 000000000..dc331e9e3 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/references/technique-selection.md @@ -0,0 +1,135 @@ +# Algorithm Technique Selection + +Use this catalog after the problem model is trusted and [model and route algorithms](../sub-skills/model-and-route-algorithms/SKILL.md#5-establish-the-operation-and-memory-envelope) has narrowed the admissible solution classes. It supplies algorithm-family questions and rejection conditions, not copy-paste code. Generate only the implementation the problem needs, then compile and test it; static template libraries often contain hidden convention, overflow, memory, or boundary assumptions. + +## Plateau gate + +If this technique-selection failure occurs after the admission criteria for [plateau escape](../sub-skills/plateau-escape/SKILL.md) are met, do not choose another method family from this catalog yet. Complete that structural review, then use this catalog to compare the returned routes. Before a plateau, use the failed complexity, proof, or brute-test evidence to narrow candidates. + +## Family-level routing + +Test only families admitted by the contract and feasibility record, stopping when a clean fit is established: + +1. **Direct structure:** sorting, counting, prefix/suffix summaries, two pointers, monotonicity, greedy exchange, sweep, or binary search on the answer. +2. **Graph formulation:** connectivity, shortest path, DAG order, SCC, MST, matching, flow/cut, 2-SAT, or difference constraints. +3. **Dynamic programming:** minimal sufficient state, ordering, transition locality, rolling memory, bitset acceleration, and safe dominance. +4. **Algebra, geometry, or strings:** exploit the exact mathematical structure rather than discretizing or searching blindly. +5. **Decomposition:** independent components, separators, treewidth, centroid or block structure, coordinate compression, or offline ordering. +6. **Bounded exponential:** subset DP, meet-in-the-middle, branch-and-bound, iterative deepening, memoized search, or inclusion-exclusion. +7. **General solver:** SAT/SMT, CP-SAT, ILP/MIP, or convex optimization when the environment and model size support it. + +Do not infer hardness from a large raw search space or select a sophisticated structure from surface vocabulary. Verify every family-specific precondition against the trusted model and feasibility envelope. + +## Fundamental transforms + +| Signal | Consider | Reject or modify when | +|---|---|---| +| Feasibility changes monotonically with a value | Binary search on answer plus decision oracle | Predicate is not actually monotone or witness reconstruction is missing | +| Need aggregate over prefixes/ranges | Prefix sums, difference arrays, Fenwick/segment tree, sparse table | Operation/update model does not match associativity/invertibility/idempotence assumptions | +| Ordered values and local comparisons | Sort, coordinate compression, two pointers, monotone stack/queue | Original order is semantic and not recoverable | +| Events become active/inactive in order | Sweep line plus balanced structure | Comparator changes inconsistently or events require dynamic future discovery | +| Many equivalent labels/states | Canonicalization, symmetry breaking, quotient state | Symmetry action does not preserve constraints/objective | +| Offline range queries with cheap endpoint edits | Mo's algorithm and variants | Updates/order dimension makes moves too costly or an online answer is required | +| Offline add/query events across an order | CDQ divide-and-conquer, sweep, BIT | Causality/order or duplicate boundaries are modeled incorrectly | +| Many monotone answer searches share work | Parallel binary search | Per-query predicate is not monotone or batched updates cannot be replayed correctly | + +## Graphs and relations + +| Need | Default candidates | Critical checks | +|---|---|---| +| Reachability/components | DFS/BFS, DSU, SCC | Directedness, offline vs online updates, recursion depth | +| Nonnegative shortest path | Dijkstra, 0-1 BFS for binary weights, Dial for small integers | Negative edges, overflow, stale queue entries | +| Negative weights | Bellman-Ford, DAG DP, potentials/Johnson | Reachable negative cycles and complexity | +| Connect all vertices cheaply | Kruskal/Prim MST | Graphic structure, disconnected inputs, precision/ties | +| Cardinality pairing | Bipartite matching, Hopcroft-Karp | Graph is bipartite and matching is unweighted | +| Weighted assignment | Hungarian, min-cost flow | Rectangular padding, min/max sign, overflow, sparse scale | +| Capacity/cut constraints | Dinic/push-relabel max flow, min cut | Node splitting, integral capacities, graph size | +| Implication choices | 2-SAT via SCC | Clauses truly have at most two literals; extract assignment correctly | +| Static tree path/subtree | Euler tour, LCA, HLD, DSU-on-tree, centroid decomposition | Choose path updates vs subtree aggregation vs distance decomposition precisely | +| Dynamic forest paths | Link-cut tree or offline rollback/decomposition | Static alternatives are simpler; splay invariants and memory are fully tested | + +For a graph reduction, document what vertices and edges mean and prove both directions. Do not use an advanced tree structure merely because the input is a tree. + +## Dynamic programming and exact search + +| Structure | Consider | Main risk | +|---|---|---| +| Small `n` with subset interactions | Bitmask DP, meet-in-the-middle, subset convolution | `2^n` memory, transition factor, reconstruction | +| Sequence with local choices | Prefix DP, automaton DP, interval DP | Missing sufficient history, invalid transition ordering | +| Tree dependencies | Tree DP, rerooting, small-to-large | Parent/child direction, combining children, stack depth | +| Bounded integer sum | Knapsack, bitset shift/or, sparse frontier | Pseudo-polynomial bound and negative values | +| Partition point recurrence | Divide-and-conquer or Knuth optimization | Required monotonicity/quadrangle inequality must be proved | +| Linear transition envelope | Convex hull trick or Li Chao tree | Min/max convention, slope/query order, equal slopes, overflow | +| Few resources/parameters | Multidimensional DP, Pareto frontier | State explosion and unsafe dominance pruning | +| Hard combinatorial core | Branch-and-bound, memoized DFS, iterative deepening | Weak bound/order, duplicate states, exponential tail | + +Always build a small brute oracle before applying a subtle DP optimization. Compare the optimized recurrence against the unoptimized one on random cases. + +## Strings + +| Need | Candidate | Pitfalls | +|---|---|---| +| One pattern in text | KMP or Z-function | Separator choice, prefix indexing, empty pattern | +| Many patterns | Aho-Corasick | Failure/output links, alphabet memory, counting order | +| Palindromic substrings | Manacher | Odd/even conventions and transformed indices | +| Suffix ordering/LCP queries | Suffix array plus Kasai/RMQ | Rank convention, radix vs comparison cost, arbitrary LCP needs RMQ | +| Online distinct substrings/repetitions | Suffix automaton | Clone transitions, occurrence propagation | +| Fast equality as a filter | Double/randomized hashing | Collision remains possible; adversarial inputs and normalization | + +Prefer deterministic algorithms when equality must be certain. If hashing is used in an exact result, combine independent hashes or verify candidates. + +## Number theory and algebra + +| Need | Candidate | Pitfalls | +|---|---|---| +| Modular powers/inverses | Binary exponentiation, extended GCD, Fermat for prime modulus | Inverse existence, multiplication overflow, negative residues | +| Many primes/factors in a range | Linear/classic/segmented sieve | Memory, handling 0/1, segment offset | +| 64-bit primality/factorization | Deterministic 64-bit Miller-Rabin, Pollard rho/Brent | Use a proven uint64 witness set and `__int128`; retry/cycle handling | +| Congruence system | CRT/generalized CRT | Non-coprime consistency and product overflow | +| Polynomial convolution | NTT/FFT | Modulus/root limits, padding, signed recovery, floating error | +| Binomial modulo prime | Factorials/inverses; Lucas for prime `p` when each base-`p` digit binomial is computable within budget, often because `p` is small enough for factorial tables | Composite modulus needs a different method; `O(p)` tables may be too large | + +Do not trust remembered primality witness bounds or root constants without a known source and targeted tests. + +For a standalone template request, pin down the supported domain and operations, then verify delicate constants, witness sets, and asymptotic claims against an authoritative source available in the environment. Return only the requested implementation plus focused tests; do not paste an unrelated template catalog. + +## Geometry + +First choose numeric semantics: exact integer/rational predicates where possible, or floating point with a scale-aware tolerance and consistent boundary policy. + +| Need | Candidate | Pitfalls | +|---|---|---| +| Convex hull | Andrew monotone chain | Duplicates, all collinear, keep/drop boundary points | +| Orientation/intersection | Cross products and bounding boxes | Integer overflow, collinear overlap, open/closed endpoints | +| Polygon area/containment | Shoelace, winding/ray casting, convex binary search | Self-intersection, boundary convention, vertex order | +| Nearest pair | Divide-and-conquer or sweep | Duplicate points, strip invariant, squared overflow | +| Half-plane feasibility | Half-plane intersection or 2D LP | Parallel lines, unbounded/degenerate result, epsilon ordering | + +Test geometry with coincident, collinear, tangent, zero-area, extreme-coordinate, and reversed-orientation cases. + +## Range and advanced structures + +Choose the operation algebra before the structure: + +``` +query type | point/range update | online/offline | persistence | coordinate size +``` + +- Fenwick trees suit prefix groups such as sums/xor; arbitrary min with updates does not inherit the same inverse trick. +- Segment tree lazy tags require a defined composition order. Test sequences of overlapping updates, not only isolated operations. +- Sparse-table O(1) overlapping queries require idempotence (for example min or gcd), not general sums. +- Persistent structures need heap/static-pool sizing and must never mutate shared nodes. Avoid giant node arrays as local stack objects. +- Implicit treaps need push-before-split/merge and deterministic reproducible RNG during debugging. +- Cartesian trees can be height `O(n)`; do not call them balanced search trees. + +## Selecting among several fits + +Rank candidates by: + +1. proof burden and model fidelity; +2. worst-case time and memory with constants; +3. implementation risk under the contest environment; +4. ease of brute/differential validation; +5. extensibility to reconstruction or scoring requirements. + +Use the least complex candidate that clears all five. When two candidates are close, prototype their dominant kernel on representative sizes instead of arguing from folklore constants. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/checker-and-local-evaluation/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/checker-and-local-evaluation/SKILL.md new file mode 100644 index 000000000..67550599c --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/checker-and-local-evaluation/SKILL.md @@ -0,0 +1,301 @@ +--- +name: checker-and-local-evaluation +description: >- + Design, implement, debug, and validate independent C++ checkers, scorers, + interactors, simulators, and local evaluation workflows for algorithmic and + competitive-programming tasks. Use when concrete evidence makes evaluator + reconstruction diagnostic: an official evaluator is absent or doubtful and + blocks a decision, zero/invalid/WA or impossible feedback is unexplained, + local and remote behavior disagrees, output legality or score must be + reconstructed, an interactive protocol fails, or Plateau Escape requires its + phase-local evaluator gate. Do not invoke solely because the first candidate + is non-full or no local evaluator exists; coherent accepted online submissions + may continue while local construction is deferred. Pair with + testlib-cpp-judging when the evaluator or data generator uses testlib.h. +--- + +# Checker and Local Evaluation + +Build an evaluator that is independent of the solver, executable under the target toolchain, and adversarially tested. Treat local construction and evaluation as tools for finding or discriminating a problem, and treat the result as a debugging oracle until official artifacts, hand-computed fixtures, or remote evidence corroborate it. + +## Enter only on diagnostic evidence + +Enter this skill when at least one of these conditions holds: + +- evaluator semantics or missing official artifacts block a concrete diagnosis, promotion, or release decision; +- official feedback is unexplained invalid, zero, WA, discontinuous, impossible under the stated formula, or inconsistent across comparable runs beyond known randomness; +- local and official results or prose and executable behavior disagree; +- legality, raw objective, normalization, transition/reward replay, or a protocol state must be independently reconstructed; +- an interactive run exposes deadlock, flush, EOF, timeout, action-budget, response-order, or termination uncertainty; +- a specific suspected bug, rare edge, resource boundary, or score-quality hypothesis needs generated instances that official feedback cannot isolate; or +- Plateau Escape has already been admitted for an interactive or scored heuristic/optimization task and requires repeated independent challenger evaluation. + +Do not enter merely because a task is interactive, heuristic, or scored; because a first candidate is non-full; because a local tool is absent; or because another online submission is planned. If official submissions are accepted, legality and protocol are stable, scores are coherent with the known contract, and the unresolved question is solver quality, return to the appropriate algorithm, Heuristic Search, Interactive, or Reactive route. Local construction may be postponed until a concrete signal makes it discriminating. + +Once entered, build only the smallest evaluator, transcript, fixture, or generated campaign that can expose or falsify the signal. Stop expanding local infrastructure after the issue is localized, the hypothesis is falsified, or coherent official evidence resolves the uncertainty. Plateau Escape is the deliberate exception: its verified plateau or severe score gap activates the full phase-local evaluator gate before challenger comparison. + +## Assign ownership before coding + +Keep evaluator mechanics separate from solver or policy design: + +| Need | Owner | +|---|---| +| Parse a candidate, validate hard constraints, recompute an objective, or transform it into a score | This skill | +| Implement an interactor/simulator state machine, process wiring contract, transcript, or per-step reward replay | This skill | +| Design hidden-state queries or prove hypothesis separation | [interactive problem solving](../interactive-problem-solving/SKILL.md) | +| Design an estimator, planner, explorer, or reward-bearing online policy | [reactive online decision problem solving](../reactive-online-decision-problem-solving/SKILL.md) | +| Select concrete `testlib.h` registration functions, streams, verdicts, and build commands | [testlib C++ judging](../testlib-cpp-judging/SKILL.md) | +| Compare solver challengers, seeds, holdouts, and noisy experiments | [validation and experiments](../validation-and-experiments/SKILL.md) | + +For a reactive task, let Reactive own action choice and let this skill own the independent transition/reward replayer or judge-facing harness. Development-time repeated scoring of complete offline outputs is not a live reactive process. + +Assign the solver/challenger and evaluator to different owners whenever delegation is available. The solver owner may run the frozen evaluator but must not silently change its contract, fixtures, or acceptance logic to admit a candidate. + +## 1. Recover the evaluator contract + +Record the operational contract before implementing it: + +``` +instance and candidate grammar +hard validity constraints and first-failure policy +raw objective, direction, numeric domain, and aggregation +displayed-score transform, reference data, rounding, and clamps +checker/scorer/interactor arguments, streams, exit statuses, and diagnostics +interactive state, legal actions, feedback order, flush points, budget, and termination +toolchain, time/memory limits, official artifacts, and unresolved discrepancies +``` + +Read every supplied checker, scorer, interactor, runner, configuration file, and sample that can determine this contract. Distinguish official executable behavior from prose and from assumptions. When they differ, document both and test the smallest witness that separates their interpretations. + +Separate these outcomes explicitly: + +``` +candidate invalid +candidate valid with raw objective +candidate valid with normalized or partial score +evaluator, configuration, or judge-data failure +``` + +Never infer a score transform merely from leaderboard behavior. + +## 2. Isolate the evaluator from the solver + +Implement the evaluator through a separate code path. Do not copy solver feasibility tests, cached-delta formulas, or the solver's favored interpretation before independently writing the evaluator contract and fixtures. + +When this evidence-triggered route or Plateau Escape requires evaluator implementation or independent validation, MUST delegate evaluator ownership to a dedicated fresh-context sub-agent whenever delegation is available. Do not assign that sub-agent to implement or tune the solver or challenger it will evaluate, or later reuse it as a challenger owner, trajectory critic, method researcher, or other method reviewer for the same task. Treat this executor-evaluator separation as the default implementation path once evaluator work is actually routed, not as a reason to route early. If the required evaluator is missing or insufficient, the evaluator owner must implement, compile, and validate the missing executable artifact. If a faithful official evaluator already exists, it must instead own independent contract recovery, adversarial fixtures, validation, and any required runner or harness; do not rewrite official source merely to manufacture ownership separation. Require the evaluator owner to read this entire skill and own its end-to-end deliverables; a review memo or pseudocode does not discharge that ownership. + +The evaluator sub-agent's initial packet must contain the original problem evidence, not a main-agent reconstruction: + +``` +verbatim original statement, title/URL, and exact I/O or protocol format +official constraints, samples, configs, visualizer, scorer, and public rules +original instance data or generator and official runner artifacts +target C++ standard, compiler, flags, and available libraries +only necessary verified filesystem, process, resource, and submission-interface facts +``` + +Do not replace the original statement or official files with a summary. Initially withhold solver source, candidate identity, solver-derived formulas, attempted-method labels, main-agent reasoning, diagnoses, expected outcomes, claimed scores, and results that lack raw evaluator evidence. A main-agent statement is not a verified fact merely because it is confident or repeated. + +Require the evaluator sub-agent to freeze a written contract, evaluator artifact, and initial hand-computed/adversarial fixtures before exposing raw candidate outputs, transcripts, or raw official verdict/score responses. Reveal those later artifacts only when needed for differential validation, with their provenance intact and without the main agent's interpretation. Expose solver internals only after an independently reproduced discrepancy proves they are necessary. + +Without delegation, preserve the same information barrier in a sealed pass with separate files, representations, derivations, and fixtures. Do not inspect solver formulas while deriving evaluator formulas. Solver-generated fixtures or agreement between two code paths copied from the same derivation are not independent evidence. + +Strongly prefer C++ for every new checker, scorer, interactor, simulator, and process harness. Use the official standard and flags when specified; otherwise use portable C++17. Use another language only when the user requests it, an official artifact must be modified in place, or a required interface makes faithful C++ implementation impractical. Record the concrete exception. + +Return executable artifacts rather than only pseudocode: + +``` +evaluator source/executable paths, content hashes, and component roles +exact run command and, when source exists, build command +small valid, invalid, boundary, and transcript fixtures +VALID/INVALID or protocol verdict with the first actionable failure +raw objective and direction +normalized score only when its transform is established +assumptions and unresolved official/local discrepancies +``` + +## 3. Choose the evaluator role and interface + +Use the smallest role that matches the contract: + +| Role | Required behavior | +|---|---| +| Checker | Parse one completed output, reject malformed or illegal candidates, and report the first failing rule | +| Scorer | Establish validity first, recompute the raw objective, then apply only a known score transform and aggregation | +| Interactor/simulator | Reproduce the literal judge state machine over bidirectional pipes and emit a verdict plus transcript | +| Reactive episode evaluator | Validate each action against the pre-action state, apply the transition, recompute reward, and aggregate the episode | +| Runner | Supervise processes, deadlines, exit status, streams, and artifacts without duplicating semantic checks | + +For a batch checker or scorer, keep stages explicit and independently testable: + +``` +instance = parse_input(...) +candidate = parse_output(...) +require_output_exhausted() +validate(instance, candidate) +raw = objective(instance, candidate) +score = normalize(raw, public_reference_data) # optional +``` + +For a non-testlib local tool, a simple interface is sufficient: + +``` +local_eval INPUT OUTPUT [ANSWER_OR_CONFIG] +exit 0: valid; stdout contains raw objective and optional established score +exit 1: invalid; stderr contains the first actionable candidate failure +exit 2: evaluator, configuration, or judge-data failure +``` + +Do not impose this generic exit contract on `testlib.h`. Follow [testlib C++ judging](../testlib-cpp-judging/SKILL.md) for its exact three-file invocation, streams, verdicts, points behavior, and interactor registration. + +## Build problem-finding data end to end + +Construct a local input campaign only when it can test a named failure or quality hypothesis. Carry it through this complete chain: + +``` +coverage strategy -> parameterized generator code -> input validator +-> solver -> independent checker/scorer/interactor -> manifest and retained failures +``` + +### 1. Freeze a coverage strategy + +Before generating volume, write a compact coverage table: + +``` +case family | suspected failure/edge | varied parameters and range +size tier | expected oracle/invariant/relation | development or holdout +``` + +Choose only relevant axes from size, density, topology or structure, numeric magnitude, constraint slack, equality/contact, duplication, symmetry/degeneracy, feasibility margin, protocol branch, and randomness/noise. Select the applicable official samples, hand-computed anchors, minimal or maximum boundaries, and families for implicated hard constraints. Add uniform random, biased, structured, adversarial, or metamorphic families only when each has a stated purpose. Random volume without a falsifiable target is not coverage. + +### 2. Implement a parameterized generator + +For more than a handful of cases, any randomized or batch campaign, and every maximum-scale or otherwise large instance, write a problem-specific generator program. Do not hand-edit large inputs or copy-paste many near-duplicates. Expose the applicable family, size, density/bias, structure, and explicit seed or seed tag as command-line parameters so every case is reconstructible from one command. Generate one case per deterministic invocation, or use a small coded batch driver that records each invocation before running it. + +Prefer portable C++17 unless the official toolchain dictates otherwise. When `testlib.h` is available or appropriate, read and use [testlib C++ judging](../testlib-cpp-judging/SKILL.md): Testlib supports deterministic data generation through `registerGen(argc, argv, 1)`, `opt`, `rnd`, and `println`; use standard C++ output when custom no-newline formatting is required. Keep generator logic separate from the solver and from the input validator. + +### 3. Validate every generated input + +Implement or use an independent strict input validator before trusting generated cases. Every generated case intended to be valid must pass it before the solver runs; deliberately invalid validator tests belong in a separate negative suite and must fail for the named reason. Fail the campaign immediately on a validator error rather than teaching the solver or evaluator to accept the generator's mistake. + +Use the literal sequence: + +``` +generator > case.in +validator < case.in +solver < case.in > case.out +independent evaluator case.in case.out [...] +``` + +Pin source or executable hashes and exact build commands for the generator, validator, solver, and evaluator. The validator must enforce grammar, bounds, cross-field constraints, and EOF independently of the generator's construction logic. + +### 4. Pair evaluation and retain provenance + +Compare champion and challenger on the same generated inputs and instance/noise seeds. Keep instance-generation, solver, and judge/noise seeds separate. Record per-case validity, raw objective or protocol verdict, established score, runtime, and first failure instead of only an aggregate. + +Retain a machine-readable or line-oriented manifest containing at least: + +``` +case id and family | full generator command | generator version/hash +instance seed/tag | input hash | validator version/result +solver identity and seed | evaluator version | verdict/raw score/runtime +``` + +On failure, preserve the original input, output, transcript, diagnostics, and full commands. Minimize the failure when practical, but keep both original and reduced artifacts; add the smallest stable reproducer to the regression set. Preserve the exact seed and parameter tuple for every rare, stochastic, invalid, timeout, or score-outlier case. Never retain only a screenshot, aggregate, or unlabeled data file. + +Start with hand-computed and tiny cases. Expand to batch or maximum-scale generation only when needed to expose a rare failure, distinguish close scoring hypotheses, or probe resources. Stop when the target problem is localized or falsified; local campaign size is not a completion metric. + +## 4. Parse and validate by reconstruction + +Validate serialization before semantics: + +- Check required token and line counts, premature EOF, and the exact trailing-output policy. +- Bound integers, finite floating-point values, characters, enums, and declared lengths before allocation or iteration. +- Check one-based versus zero-based indices, duplicates, missing objects, forbidden extras, and which submitted solution counts. +- Detect overflow while parsing, accumulating, multiplying, or normalizing. +- Use a real parser for expressions, grammars, paths, and operation sequences; avoid substring validation. + +Never trust a candidate's claimed score or derived state. Reconstruct the submitted object or replay every operation from the original instance: + +- For constructive output, rebuild the witness and test every hard constraint. +- For graphs, check indices, multiplicity, degrees, connectivity, cycles, paths, capacities, and direction as applicable. +- For geometry, check bounds, orientation, contact semantics, overlap, and exact or tolerance-aware predicates. +- For schedules and assignments, recompute resource use, order, coverage, and cross-object constraints. +- For action sequences, validate the whole action against the pre-action state before mutating state. + +Use wide integer types for costs, products, squared distances, pair counts, and normalization differences. Reject `NaN` and infinity explicitly. + +## 5. Recompute objectives and scores + +Answer these questions independently: + +1. Is the candidate legal? +2. What is its raw objective or episode reward? +3. How does the evaluator transform and aggregate that value into the displayed score? + +Determine minimization versus maximization, integer versus floating arithmetic, the rounding point, clamps, thresholds, piecewise rules, ratios, logarithms, and per-case or per-round aggregation. Establish whether reference data is an optimum, baseline, bound, or jury objective. Define behavior for zero denominators and degenerate equal-reference cases. + +Do not assume the displayed score is the fraction of fully passed test cases unless the operational evaluator establishes that equivalence. A case reported as `Wrong Answer` or otherwise non-perfect may still award partial points. Reconstruct each case's raw value or point contribution and the official cross-case weighting and aggregation. Once established, use the final official aggregate score as the solver-comparison and promotion target; use fully passed case count only as a diagnostic or as a proven-equivalent objective. + +Keep the raw objective authoritative even when normalization remains unknown. For reactive episodes, recompute transition-local rewards and the final aggregation rather than accepting policy logs; use [reactive online decision problem solving](../reactive-online-decision-problem-solving/SKILL.md) to judge whether the policy itself estimates, plans, or explores well. + +## 6. Implement process-level interactors and simulators + +Model the same initial output, response order, state transitions, counted actions, special replies, final-answer rules, and termination conditions as the operational judge. Keep the solver unaware of simulator-only truth. + +Connect solver stdout to evaluator stdin and evaluator stdout to solver stdin through real pipes or the official runner. Direct function calls cannot expose missing flushes, buffering, premature close, EOF, deadlock, or timeout behavior. Keep response generation, transcript capture, process supervision, and strategy assertions separable. + +Test every legal hidden state for tiny sizes, boundary actions, exact budget exhaustion, early success, judge error, malformed messages, timeout, EOF, and transcript agreement with an official interactor when available. Cap deliberately exhaustive behavior to small instances; measure response complexity on large cases so the harness does not become the bottleneck. + +Use [interactive problem solving](../interactive-problem-solving/SKILL.md) for query selection and adversarial hypothesis reasoning. Use [reactive online decision problem solving](../reactive-online-decision-problem-solving/SKILL.md) for reward-aware action choice. When `testlib.h` is present, use [testlib C++ judging](../testlib-cpp-judging/SKILL.md) for concrete interactor APIs and flush/wiring requirements. + +## 7. Validate the evaluator itself + +Create an adversarial evaluator suite: + +- one hand-computed valid fixture; +- truncated, extra-token, duplicate, out-of-range, non-finite, and overflow candidates; +- one violating fixture per hard constraint; +- exact equality, contact, and other boundary cases; +- metamorphic cases whose legality or raw objective has a known relationship; +- tiny instances compared with brute enumeration; +- mutation tests that corrupt one field of a valid candidate and require rejection; +- official samples, visualizer output, official evaluator results, or remote feedback when available; +- interactive transcript and real-pipe failure cases for an interactor. + +When possible, implement the tiny brute oracle or objective calculator with a different representation from both solver and evaluator. A checker that only agrees with solver-generated fixtures is not independent evidence. + +## 8. Integrate the local loop + +Build the evaluator when source exists, then exercise its command-line or process contract before relying on it. For each challenger, run: + +``` +solver -> independent evaluator -> first failure or raw objective delta +``` + +Record only: + +``` +candidate | valid/invalid | raw objective | established score +runtime | first failure | official/local discrepancy | keep/revert +``` + +If local and official results disagree, stop solver tuning. Minimize the discrepancy, test competing contract interpretations, and change the evaluator or documented assumption before generating more solver cases under the unchanged validator. If local validation passes but official evaluation remains zero, rederive the missing acceptance condition instead of adding random tests that the same incomplete validator will accept. + +For routed evaluator work, record completion only after all applicable items are present and actually run: + +``` +independent evaluator-builder not reused as solver/challenger/reviewer, or documented sealed-pass fallback +verbatim original problem packet and evaluator contract +evaluator artifact path/hash and exact run command +source/build command when source exists; harness source/build command when required +hand-computed, malformed, boundary, mutation, and tiny-oracle fixtures +coverage table, generator source/build/full commands, validator evidence, and seed manifest when a generated campaign was needed +original and minimized failing input/output/transcript artifacts when a failure was found +real-pipe transcript tests for interactive tasks +legality and raw-objective recomputation for scored heuristic/optimization tasks +official/local comparison where an official artifact or result exists +remaining unknown score transforms or protocol assumptions +``` + +Do not report the evaluator trusted from pseudocode, a solver-authored validity function, sample-only agreement, a score estimate, or generated inputs that never passed an independent validator. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/contest-solver-engineering/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/contest-solver-engineering/SKILL.md new file mode 100644 index 000000000..1eea494a2 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/contest-solver-engineering/SKILL.md @@ -0,0 +1,202 @@ +--- +name: contest-solver-engineering +description: >- + Diagnose and harden contest or bounded algorithmic solver implementations for + compiler and toolchain mismatches, crashes, undefined behavior, overflow, + memory-layout failures, TLE or MLE, buffering, serialization, randomness, + data-structure invariants, and unsafe deadline handling. Use as an internal + algorithmic-problem-solving recovery route after evidence implicates the + implementation or target environment rather than the mathematical model, + evaluator contract, or search-method design. +--- + +# Contest Solver Engineering + +Recover implementation and environment failures after the algorithmic route is justified. Correct algorithm design can still score zero when the target compiler, protocol, serialization, numeric range, memory layout, or deadline is wrong. + +## Ownership and handoffs + +This skill owns target-toolchain reproduction, numeric and memory safety, measured runtime, low-level I/O, serialization, randomness, implementation invariants, and release hardening. Route: + +- a contradicted state, recurrence, reduction, proof, or asymptotic route to [model and route algorithms](../model-and-route-algorithms/SKILL.md); +- evaluator parsing, legality, or score uncertainty that blocks the current decision, or official/local disagreement to [checker and local evaluation](../checker-and-local-evaluation/SKILL.md); +- experiment design and champion/challenger comparison to [validation and experiments](../validation-and-experiments/SKILL.md); +- query strategy and hidden-state inference to [interactive problem solving](../interactive-problem-solving/SKILL.md); +- scored construction, representation, moves, deltas, and optimizer behavior to the shared [heuristic search](../../references/heuristic-search.md). + +## 1. Reproduce the target environment + +Read the config and build command. Record: + +``` +language standard and compiler version +optimization/debug flags and architecture flags +available headers, libraries, and runtime +time and memory limits per case/process +input/output protocol and working directory +``` + +Compile early with the actual standard. Avoid assuming optional libraries, Boost components, compiler extensions, AVX/AVX2, or native CPU flags exist. Intrinsics require matching target flags and judge hardware. Prefer portable scalar code unless the environment explicitly guarantees the feature and the gain is measured. + +Use warnings in development, for example: + +``` +-Wall -Wextra -Wshadow -Wconversion +``` + +Treat warnings involving return types, narrowing, signedness, uninitialized state, comparator requirements, and ambiguous names as correctness issues. + +## 2. Numeric safety + +Derive value bounds for every multiplication, sum, distance, count, and score. + +- Use `int64_t`/`long long` when 32-bit can overflow; use `__int128` for intermediate products when supported by the target. +- Check sentinel arithmetic. `INF + weight`, negating the minimum signed value, and evaluating a sentinel line can overflow before comparison. +- Normalize negative modular residues and prove modular inverse existence. +- Keep objective direction consistent when negating scores or costs. +- Avoid exact equality on floating point unless values are constructed to be exact. Use scale-aware predicates and one boundary convention. +- Keep scoring computations compatible with the official rounding order. + +Sanitizers (`address`, `undefined`) are valuable on development tests, but use a release build for timing and verify sanitizer availability first. + +## 3. Memory layout + +Estimate bytes, not just Big-O: + +``` +vector capacity, node padding, allocator metadata, hash load factor, +recursion stack, duplicated versions, rollback logs, per-test clearing +``` + +Avoid enormous arrays as local stack objects. Allocate large pools statically or on the heap with checked capacity. Persistent structures need a proven maximum node count and overflow guard. Prefer flat contiguous arrays when traversal is hot; reserve vectors and reuse buffers inside search loops. + +Do not clear an `O(max_size)` table per tiny test when timestamps or touched indices suffice. Release or reuse per-case data according to total memory. + +## 4. Runtime and deadlines + +Use a monotonic clock such as `steady_clock`. Set an internal limit below the official time and include parsing, construction, final validation, and output. + +``` +deadline = start + internal_budget +periodic check interval * worst move time bounds overshoot +``` + +Measure timer-call cost before checking on every tiny transition. Conversely, do not run a large uninterruptible loop or subsolve without a deadline check or work cap. Host load and measurement granularity make near-limit results flaky. + +For multiple cases under one process limit, allocate time adaptively but retain a guaranteed path to output for every remaining case. Keep the best legal state available if search stops immediately. + +Profile optimized binaries with representative data. Common hidden costs: + +- copying full state on every move; +- full rescoring instead of local delta; +- repeated allocation/free and string construction; +- maps/hashes where sorted vectors or arrays suffice; +- RNG/distributions in the hottest loop; +- excessive logging or flushing; +- pathological repair or recursion tails. + +If the measured state/transition count invalidates the selected complexity class rather than exposing an implementation cost, return to Model and Route Algorithms. + +## 5. Randomness + +Use a fast, reproducible generator appropriate for the task. During debugging, accept or log an explicit seed. Distinguish the program seed from instance and judge-noise seeds. + +- Avoid modulo bias when it can affect the method materially. +- Check empty/singleton ranges before sampling. +- Keep deterministic tie-breaking when comparing algorithms. +- Do not reseed repeatedly from low-resolution time. +- A time-derived release seed may improve diversity, but preserve a way to reproduce failures and confirm the judge permits nondeterminism. + +## 6. Input/output and serialization + +Parse the exact token grammar and sizes. For large input, use suitable fast I/O, but do not mix incompatible buffered I/O APIs carelessly. + +For batch output: + +- print exactly the required number/order of tokens or lines; +- avoid extra debug text unless comments are explicitly permitted; +- validate indices, uniqueness, counts, and bounds after conversion from the internal state; +- ensure the output buffer/string cannot grow without bound; +- retain enough time for serialization and flush. + +For interactive output: + +- read required initial data before issuing a query; +- flush after every query/action that expects a response; +- parse special termination/error responses before updating state; +- never print diagnostics to stdout; use stderr only if permitted; +- do not wait for an initial token the protocol never sends; +- avoid naming helpers so they collide with standard-library functions. + +When a concrete buffering, flush, deadlock, premature-EOF, or process-wiring symptom needs local reproduction, test the real process through pipes against a simulator. Stable coherent official interactions do not require simulator construction solely for this section; route evaluator implementation through Checker and Local Evaluation only when that diagnostic evidence exists. + +This section owns low-level process, buffering, serialization, and deadline failures. Interactive Problem Solving owns the protocol contract, hypothesis model, and query strategy. + +### Nonzero `Wrong Answer` or non-perfect feedback in query-scored tasks + +A judge label does not always identify the failing layer. If official feedback gives a nonzero score but says `Wrong Answer` or otherwise reports a non-perfect result: + +- Treat the nonzero score as evidence of partial credit, not as proof that the candidate is fully legal or correct and not as proof that every non-perfect case contributed zero. +- Do not assume `score = fully passed test cases / total test cases` unless the official evaluator or scoring rules establish that formula. A single case labeled `Wrong Answer` or otherwise not fully correct may still contribute partial points. +- If the per-case raw contribution, weights, rounding, clamps, or final aggregation is unknown or contradicted and that uncertainty blocks the current decision, recover it with [Checker and Local Evaluation](../checker-and-local-evaluation/SKILL.md). When coherent official rules or results already establish the aggregation, use them without constructing a local evaluator. Optimize the established final official score rather than the binary count of fully passed cases. +- If the public scoring formula contains a query-count term, invert it to estimate the hidden query count and compare that estimate with local query categories. +- If the score has no query-count term, do not invent one; diagnose the actual scored objective instead. +- Prioritize zero score, crash, protocol failure, malformed output, and hard-invalid candidates. Otherwise choose the next correction by its expected improvement to the established final score instead of insisting that every case become binary accepted first. + +## 7. Data structures and invariants + +Advanced structures are high-risk code. For each operation state: + +``` +precondition, mutation, lazy/tag composition, maintained aggregate, +index convention, complexity, inverse/rollback behavior +``` + +Property-test them independently. Important cases: + +- empty/single element and full range; +- repeated equal keys and duplicate insert/delete; +- overlapping lazy updates in different orders; +- split then merge identity; +- link/cut only across valid components; +- custom comparator is a strict weak ordering; +- coordinate compression maps every queried value; +- pools and queues cannot exceed capacity. + +Prefer a simpler verified structure if the advanced option does not decide the complexity bound. + +## 8. Development and release modes + +Development build can include: + +- assertions and full invariant checks; +- apply/undo and delta/full-score comparisons; +- deterministic seeds and verbose stderr traces; +- sanitizers and slow reference paths; +- operation counters and stage timers. + +Release build should: + +- retain cheap guards needed for legality; +- disable or sample expensive diagnostics; +- bound all variable work; +- use target-compatible flags only; +- contain no filesystem/network/private-data dependency unless authorized; +- produce a valid fallback under early deadline. + +Run both modes on the same regression corpus, then rerun release in a clean process or container. Reused containers may retain writable temporary state, so a clean run is part of validation. + +## 9. Final audit + +Before delivery, answer concretely: + +- Which compiler command succeeded? +- Which official or local judge command succeeded? +- What are representative and worst-observed runtime and memory? +- What margin remains below the limit? +- Which numeric products have explicit bounds? +- Does every early-exit path produce legal output or terminate protocol safely? +- Is the saved final artifact the champion? +- Are unavailable headers, CPU features, accidental debug output, and private data access absent? + +Complete the applicable release gate in Validation and Experiments. Return the root implementation/environment cause, the exact build and run commands, focused regression results, observed resource margins, and remaining target-environment assumptions. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/interactive-problem-solving/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/interactive-problem-solving/SKILL.md new file mode 100644 index 000000000..4b074f2d1 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/interactive-problem-solving/SKILL.md @@ -0,0 +1,140 @@ +--- +name: interactive-problem-solving +description: >- + Solve or debug information-acquisition tasks where queries or evaluator + feedback must reduce uncertainty until a hidden state, answer, or contract is + sufficiently determined. Use for interactive problems, output-only tasks + with a checker, and custom-judge workflows when the central difficulty is the + protocol contract, query design, hypothesis elimination, query budget, + transcript behavior, noise, or adversarial feedback. Do not use for offline + output-only or scorer-only optimization whose feedback occurs only between + development runs; use heuristic search. Use reactive-online-decision-problem-solving when + reward-bearing actions receive live feedback that changes later decisions. +--- + +# Interactive Problem Solving + +Reduce uncertainty until the answer is uniquely or sufficiently determined: + +``` +understand the interaction contract + -> design a query + -> observe and update hypotheses + -> recover the hidden state or final answer +``` + +The central question is: **What should be asked next?** The main value of an action is the information it reveals. + +## Boundary with decision-making + +Use this skill when termination means that the answer or relevant hidden state is known well enough. Use [reactive online decision problem solving](../reactive-online-decision-problem-solving/SKILL.md) when termination is a round horizon, task completion, or budget exhaustion and actions must earn immediate or future reward while controlling risk. + +For an output-only task with a checker or a custom judge, treat an evaluated candidate as a query only when its verdict, score, or diagnostic is primarily being used to infer hidden acceptance conditions, state, or evaluator behavior. If the contract is already known and the goal is offline repeated score improvement, route to [heuristic search](../../references/heuristic-search.md). Route to [Reactive](../reactive-online-decision-problem-solving/SKILL.md) only when reward-bearing actions receive live feedback that changes later decisions. + +## 1. Establish the literal contract + +Read the statement and every supplied interactor, tester, checker, scorer, runner, and configuration file relevant to the information channel. Record: + +``` +initial judge output and ordering +legal query/candidate syntax and parameter bounds +exact deterministic response function or stochastic observation model +counted queries/evaluations/actions and total budget +required flush points and process wiring +special replies: error, -1, success, EOF, timeout +final-answer syntax and whether it consumes budget +state reset or persistence across cases/rounds/submissions +correctness and score dependence on answers, queries, actions, and time +``` + +Replay sequential updates, rounding, accumulators, and state transitions literally. When prose and executable behavior differ, document both, derive their legal intersection, and use the intersection when it preserves correctness and budget. If a stronger solution depends on executable-only behavior, name and directly test that dependency. + +Only when a concrete unresolved deterministic validity or score question requires independent reconstruction, use [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) rather than treating checker calls as interactive queries. Coherent accepted official evidence does not trigger that route by itself. + +## 2. Model hypotheses and identifiability + +Formalize deterministic interaction as: + +``` +H_t hidden states consistent with the transcript through time t +R(h, q) exact response to query q in hidden state h +H_{t+1} = {h in H_t : R(h, q_t) matches the observed response} +``` + +For stochastic feedback, maintain likelihoods, posterior weights, or confidence sets rather than eliminating a hypothesis after one mismatch. + +Before coding a strategy, test identifiability: + +- Enumerate tiny hidden-state spaces and response partitions. +- Search for symmetric or globally indistinguishable states. +- Partition the hidden states into `K` final-answer classes so one same sufficient accepted answer works for every state in a class. +- If every response has at most `b > 1` outcomes, compare the budget with the necessary lower bound `ceil(log_b K)`; use `K = |H|` only when exact hidden-state recovery is required, and account for unbalanced partitions and indistinguishability. +- Prove that the planned transcript separates every pair of classes that require different final answers, not merely sampled states. + +Exact state recovery is unnecessary when every remaining hypothesis implies the same accepted answer. State that equivalence explicitly. + +## 3. Choose the next query + +Derive candidate queries from the exact response semantics. Rank them by the criterion matching the judge: + +- minimize the worst-case remaining hypothesis count; +- maximize justified expected entropy or uncertainty reduction; +- distinguish a targeted pair or equivalence class; +- exploit group testing, coding, parity, algebraic cancellation, balanced separators, or batched independent features; +- retain separation margin under noise or adversarial perturbation; +- include query cost when queries have unequal price. + +For an adversarial but consistent judge, optimize the worst response partition and preserve the complete feasible set or an equivalent invariant. For known stochastic noise, use likelihood updates, repeated measurements, sequential tests, robust estimators, or error-correcting separation. For unknown noise, reserve calibration queries and avoid brittle hard elimination. + +Do not multiply per-query success probabilities without establishing independence or a valid conditional bound. + +## 4. Isolate protocol mechanics + +Keep the solver's information logic behind a small interface: + +``` +read_initial() +ask(query) -> response +update(response) +answer(solution) +``` + +`ask` owns serialization, budget accounting, flushing, reply validation, and immediate termination on judge error or EOF. Keep diagnostics off stdout. Put buffering, deadline, numeric, and target-toolchain fixes in [contest solver engineering](../contest-solver-engineering/SKILL.md); do not mix them into the query-selection proof. + +## 5. Validate with a simulator when diagnostic + +Route to [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) when a concrete protocol symptom, disputed state transition, unexplained official response, or strategy hypothesis needs a process-level simulator/interactor, real-pipe runner, transcript contract, or independent state/reward replay. If accepted official interactions have stable protocol behavior and coherent feedback, do not build a simulator solely because this section was reached; defer it until it can expose or discriminate a problem. Once routed, use [validation and experiments](../validation-and-experiments/SKILL.md#7-interactive-and-reactive-evaluator-validation) to incorporate those artifacts into the broader validation ladder and solver comparison loop. + +When evaluator work is routed and the repository uses `testlib.h`, first keep the evaluator contract and independence rules in Checker and Local Evaluation, then follow [testlib C++ judging](../testlib-cpp-judging/SKILL.md) for the concrete interactor/checker API and build commands. + +When such a simulator or transcript campaign is justified, validate the information strategy itself with: + +- exhaustive tiny hidden states; +- exact budget exhaustion and final-answer accounting; +- a worst-case consistent response chooser; +- fixed-seed noisy transcripts when applicable; +- collision search over final remaining hypothesis classes; +- early success, error, EOF, and malformed-reply paths. + +The solver must never observe simulator-only truth. + +## 6. Safety and termination + +- Check the budget before emitting each query or candidate evaluation. +- Keep a legal final answer for every reachable belief state when partial credit or early termination exists. +- Reserve enough budget and time to serialize the answer and complete mandatory protocol steps. +- Use deterministic tie-breaking while debugging. +- Terminate as soon as the remaining hypotheses imply one sufficient answer; do not spend queries merely to identify irrelevant hidden details. + +## Diagnostics + +| Symptom | Test first | +|---|---| +| Immediate protocol failure | Initial read order, syntax, indexing, flush, special reply | +| Correct tiny cases but query blow-up | Partition balance, symmetry, repeated information | +| Decoder collision | Literal response semantics and global hypothesis enumeration | +| Works in functions but hangs remotely | Real bidirectional pipes, buffering, EOF, timeout | +| Noise causes unstable elimination | Likelihood model, dependence, calibration, robust separation | +| Nonzero feedback but non-perfect result | Whether feedback measures correctness, query count, or quality | + +Return the recovered contract, query invariant/criterion, identifiability argument, implementation, simulator/transcript commands actually run, maximum observed query count, and remaining assumptions. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/model-and-route-algorithms/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/model-and-route-algorithms/SKILL.md new file mode 100644 index 000000000..08b6c0833 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/model-and-route-algorithms/SKILL.md @@ -0,0 +1,204 @@ +--- +name: model-and-route-algorithms +description: >- + Diagnose and repair a static algorithmic problem model, state, reduction, + recurrence, invariant, or proof, then decide whether an exact, constructive, + exact-hybrid, heuristic, or scored-hybrid route fits the real bounds. Use as + an internal algorithmic-problem-solving recovery route when brute force or a + proof premise contradicts the attempted solution, or when the model is + trusted but the justified solution class remains unclear. Do not use for + hidden-state query design or reward-bearing online decisions. +--- + +# Model and Route Algorithms + +Establish a trusted static model and proof target before selecting a solution class. Enter at the earliest unresolved stage and do not reopen an upstream stage without new contradictory evidence. + +``` +operational contract -> sufficient model/state -> proof obligation +-> resource envelope -> admissible solution class -> concrete method family +``` + +## Entry modes and ownership + +Use the **model-repair entry** when the failed attempt may be solving the wrong problem or relying on an incomplete state, reduction, invariant, recurrence, or proof. Use the **route-selection entry** at [the feasibility envelope](#5-establish-the-operation-and-memory-envelope) only when the operational model and correctness obligations are already trusted. + +This skill owns static combinatorial modeling, correctness obligations, and exact/constructive/hybrid/heuristic route choice. Route other failure layers as follows: + +- evaluator parsing, legality, or objective uncertainty that blocks the current decision, or official/local disagreement: [checker and local evaluation](../checker-and-local-evaluation/SKILL.md); +- hidden-state acquisition and query semantics: [interactive problem solving](../interactive-problem-solving/SKILL.md); +- reward-bearing sequential decisions under uncertainty: [reactive online decision problem solving](../reactive-online-decision-problem-solving/SKILL.md); +- compile, numeric, memory-layout, serialization, or measured deadline failure after the route is justified: [contest solver engineering](../contest-solver-engineering/SKILL.md); +- oracle design, counterexample minimization, and controlled comparisons: [validation and experiments](../validation-and-experiments/SKILL.md). + +## 1. Normalize the operational contract + +Remove story terms and write: + +``` +Given instance I, choose x in F(I). +Hard constraints: C_j(I, x) = true for every j. +Raw objective: minimize/maximize f(I, x). +Observed score: S(I, x) = transform(f, baseline, bound, aggregation). +``` + +Derive this model from the statement and every directly relevant checker, scorer, configuration, and sample. When prose and executable behavior differ, record both interpretations, their legal intersection, and any dependency on the operational evaluator. If the evaluator itself is doubtful, stop and use Checker and Local Evaluation before treating either interpretation as trusted. + +Check common modeling traps: + +- a fine-grained constraint was replaced with an unjustifiably stronger one; +- equality, duplicates, orientation, order, or multiplicity are semantically meaningful; +- a sequential transition was replaced by a non-equivalent aggregate formula; +- the score uses a clamp, ratio, rounding, logarithm, lexicographic priority, baseline, or whole-run failure rule; +- several cases share one time, memory, state, or score budget; +- the public statement and executable evaluator disagree; +- a stochastic generation claim is a distributional promise rather than a per-instance guarantee. + +Separate legality, raw objective, and displayed score. A correct objective calculation does not prove legality, and a non-perfect score does not by itself prove invalidity. + +## 2. Define sufficient, tractable state and valid reductions + +A state must retain exactly what affects future feasibility and value. Derive it by asking which histories are equivalent for every possible continuation. + +Among sufficient states, prefer one that is compact, canonical, and efficient to encode, compare, hash, copy, and update. Its legal transitions should be easy to generate without reconstructing the full history, and the information needed for objective updates, pruning, memoization, dominance, and reconstruction should be directly available or incrementally maintained. + +Look for: + +- canonicalization under interchangeable labels, rotations, reflections, or component order; +- dominance: discard state `a` only if state `b` has no worse resources/value and at least the same continuation set; +- monotone resources enabling Pareto frontiers; +- sparse reachable states enabling maps instead of dense arrays; +- a small boundary between processed and unprocessed structure; +- reversible transitions and compact reconstruction parents. + +Falsify a proposed compression by constructing two histories with the same compressed state and searching for a continuation that treats them differently. Such a continuation disproves the state definition. + +For every reduction, define both mappings: + +``` +original feasible solution -> reduced feasible solution +reduced feasible solution -> reconstructed original solution +``` + +Show preservation of feasibility and objective in both directions. Check capacities, integrality, direction, indexing, duplicate handling, and reconstruction rather than relying on the name of a standard reduction. + +## 3. State and falsify the proof obligation + +Choose the proof shape that matches the method: + +- **Greedy:** feasibility plus an exchange, cut, or stays-ahead argument. A plausible priority rule is not a proof. +- **DP:** state meaning, base cases, exhaustive and non-overlapping transitions, induction order, optimum preservation, and reconstruction. +- **Graph reduction:** both solution mappings plus objective preservation; verify capacities, integrality, and edge direction. +- **Binary search:** monotone predicate, boundary convention, termination, and constructive witness when required. +- **Invariant-based construction:** initialization, preservation by every operation, progress, termination, and output conversion. +- **Randomized exact:** distinguish Las Vegas correctness from one-sided or two-sided error; bound error probability and runtime tails. + +Turn every critical premise into a tiny adversarial or exhaustive test where practical. When brute force disagrees, minimize the counterexample and identify the first violated premise before changing implementation details. + +## 4. Establish constructive correctness + +When many outputs are accepted but not scored, model construction as invariant-preserving decisions. Prefer representations that make duplicates, overlap, disconnectedness, or capacity violations impossible. + +Useful patterns include: + +- build a spanning, tree, path, or assignment backbone before optional structure; +- satisfy the most constrained object first; +- use Hall, cut, degree, parity, or conservation conditions to detect impossibility; +- reserve slack explicitly instead of consuming all capacity greedily; +- canonicalize symmetric choices to simplify proof and testing; +- validate final serialization independently from the internal structure. + +A promise that the input is feasible does not prove that the chosen construction reaches a solution. Prove progress and termination under every legal input. + +Do not pass the model checkpoint until the operational contract, state/reduction, and applicable proof obligation are either established or isolated as the concrete unresolved cause. + +## 5. Establish the operation and memory envelope + +Extract from the actual bounds: + +``` +number of states or objects +branching and transitions per state +number of test cases or instances +bytes per state, edge, table entry, cache, parent, and copy +I/O, initialization, clearing, reconstruction, and serialization cost +official time, memory, language, and library constraints +``` + +Use worst-case bounds for correctness and representative pilots for constants. Account for bitset word count, hash/map constants, allocator traffic, recursion, cache locality, repeated clearing, and total cost across all cases. A mathematically smaller Big-O method can lose on constants or memory bandwidth, but empirical speed does not rescue a method whose worst case violates the contract. + +For bounded exponential methods, estimate the actual remaining exponent: + +``` +raw variables -> forced decisions -> components -> symmetry classes +-> kernel size -> meet-in-the-middle split -> remaining states +``` + +Do not infer hardness from the raw Cartesian search space. Constraints may expose flow or matching structure, total unimodularity, matroids, intervals, convexity, small parameters, separators, bounded treewidth, or independent components. + +## 6. Classify admissible routes + +Use contract, proof, and resource evidence to distinguish: + +- **Exact:** a complete worst-case method fits with adequate time and memory margin. +- **Constructive:** an invariant-preserving legal witness is sufficient and no optimization is required for acceptance. +- **Exact hybrid:** decomposition, bounds, relaxations, or subsolvers preserve overall completeness. +- **Heuristic:** non-optimal output is permitted and no complete route fits with credible margin. +- **Scored hybrid:** heuristic global decisions are combined with bounded exact subproblems. + +For exact-output tasks, an incomplete search is not converted into an exact algorithm by performing well on samples. Every pruning rule, timeout, decomposition, and subsolver must preserve completeness. + +For each credible exact or exact-hybrid candidate, record: + +``` +state/object count and transition count +worst-case time with test-count and reconstruction factors +peak memory in bytes, including parents and allocator overhead +required reductions, structural promises, and proof obligations +implementation and validation risk +resource margin under the target toolchain +``` + +Reject or revise a route when completeness depends on unproved compression, unsafe dominance, unguaranteed average-case behavior, unavailable solver/library support, or a resource estimate without margin. When several exact candidates remain plausible, compare representative dominant kernels, but keep worst-case feasibility authoritative. + +## 7. Select the downstream method family + +After the admissible class is fixed, read [technique selection](../../references/technique-selection.md) for DP, graph, algebra, geometry, string, decomposition, bounded-exponential, general-solver, or data-structure families. Do not use that catalog to bypass an unresolved model or feasibility contradiction. + +When complete exact work does not fit, do not force an incomplete method if the contract requires an exact answer. Revisit missing structure or record that no justified complete route is currently known. When non-optimal output is explicitly allowed, retain hard validity and a guaranteed-valid fallback when early termination could erase all value. + +For heuristic or scored-hybrid mechanics, read the shared [heuristic search](../../references/heuristic-search.md). If a verified legal champion already meets the plateau admission gate, use [plateau escape](../plateau-escape/SKILL.md) before another tuning sequence. + +## 8. Design exact and scored hybrids + +"Hybrid" describes composition, not a relaxation of the output contract. Exact-hybrid examples include admissible bounds inside complete branch-and-bound, exact component decomposition with exact coupling, and relaxation bounds used only for certified pruning. + +When approximation is allowed, common scored hybrids include: + +``` +heuristic global order + exact local scheduling +heuristic destroy set + exact DP/matching/flow repair +exact solution on small components + heuristic coupling +candidate generation + exact selection/assignment +``` + +For each exact subproblem, quantify its size, solve cost, call frequency, cache reuse, timeout behavior, and boundary coupling. Bound variable work so one hard subsolve cannot consume the release budget. Verify that the local objective aligns with the global true score and that splicing preserves every cross-boundary constraint. + +## 9. Produce one model-and-route record + +Keep a single handoff rather than separate modeling and feasibility reports: + +``` +entry mode and decisive failure evidence +operational contract and disputed interpretation +trusted model, state, reduction, and proof obligation +smallest counterexample or remaining unproved premise +Candidate A: O(...), ... MB; accepted/rejected because ... +Candidate B: O(...), ... MB; accepted/rejected because ... +chosen route: exact / constructive / exact hybrid / heuristic / scored hybrid +critical structural, evaluator, and resource assumptions +oracle, bound, compiler pilot, or evaluator evidence actually used +next routed document/sub-skill and its exact question +``` + +Proceed only from the established checkpoint. Reopen the model when new correctness evidence contradicts it; reopen route choice when a bound, environment fact, or method premise changes materially. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/plateau-escape/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/plateau-escape/SKILL.md new file mode 100644 index 000000000..51a124f31 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/plateau-escape/SKILL.md @@ -0,0 +1,248 @@ +--- +name: plateau-escape +description: >- + Escape a root-verified measured plateau or severe score gap in a scored + algorithmic, heuristic, or hybrid solver through independent review, method + research, and an evidence-gated structural prototype. Use only when the root + router admits a verified legal champion. For an interactive or scored + heuristic/optimization task, including AHC, first implement and validate its + mandatory independent local interactor/checker/scorer under + checker-and-local-evaluation, delegating missing implementation to a separate + fresh-context evaluator builder when available. Do not use for an isolated + low score without the root severe-gap evidence, invalid or zero output, + checker disagreement, crash, TLE/MLE, or interactive protocol failure alone. +--- + +# Plateau Escape + +Protect the verified champion, challenge the current problem representation, and test a structurally different method before resuming scalar tuning. + +## Entry contract + +Enter only when the root [scored-recovery escalation](../../SKILL.md#escalate-weak-scored-recovery) routes here. Treat that routing decision as authoritative instead of re-deriving plateau thresholds in this sub-skill. The handoff must identify the measured-plateau or severe-score-gap signal and include the champion-legality, resource, and controlled-attempt evidence used by the root; if the escalation signal is absent, return to the root router. For an interactive or scored heuristic/optimization task, missing or unvalidated local-evaluator evidence is not a reason to proceed or return only advice: complete the mandatory evaluator gate below first. + +## Freeze the champion and separate facts from conclusions + +Do not mutate the champion during diagnosis. Preserve the original problem artifacts rather than replacing them with a summary. Build two records: + +``` +primary problem packet + verbatim original statement content/title/URL and exact I/O or protocol + original constraints, objective, scoring and aggregation rules + official evaluator/configuration, samples, public instances, benchmark or dataset + target toolchain, resource limits, submission interface and remaining budget + +verified fact ledger + necessary fact | exact derivation/source/command | reproducing artifact + champion identity, legality, final score/runtime and evaluator evidence + local evaluator identity/hash, build/run commands, fixtures and validation evidence + saved counterexamples, profiles and fixed cases/seeds +``` + +A derived fact is admissible only when it follows from a checked proof, an official primary artifact, or a reproducible command or test whose artifact is included. Keep prior diagnoses, method preferences, estimated upside, risk labels, and reported results without raw evidence out of the fact ledger. Preserve an unresolved discrepancy as a question with its primary artifacts; never silently turn one interpretation into a fact. + +Give every sub-agent the verbatim primary problem packet rather than only a main-agent summary. A summary may index original artifacts but may not replace them. Pass main-agent reasoning, diagnoses, or prior results only when a later role explicitly requires them, and label claims that lack primary or reproducible evidence as unverified. + +Keep the prior attempt log separately as claims plus observations: + +``` +claimed hypothesis -> exact material change -> command/submission -> observed result +``` + +Stop repeated changes to weights, thresholds, temperatures, tie-breaks, seeds, or runtime allocation. The next hypothesis must concern representation, objective/value model, neighborhood reachability, planning horizon, decomposition, relaxation, or algorithm family. + +## Complete the phase-local mandatory evaluator gate + +For every interactive task and every scored heuristic or optimization task, including AHC, read and execute [Checker and Local Evaluation](../checker-and-local-evaluation/SKILL.md) in full, then verify that an executable independent local interactor, checker, scorer, simulator, or required combination satisfies it. This is a Plateau Escape phase requirement activated by the verified plateau or severe score gap; it is not a global requirement after the first non-full candidate. An official score or solver-authored validity function alone does not satisfy this prerequisite. Do not start the trajectory critic, method researcher, challenger comparison, another submission, or final delivery until the gate is complete. + +If the artifact is missing or insufficient, Plateau Escape owns implementing and validating it. Do not defer it to the main agent as a recommendation or mark the structural search complete without it. When delegation is available, MUST assign implementation to a dedicated fresh-context evaluator-builder sub-agent that is distinct from the solver/challenger owner, trajectory critic, and method researcher. That builder must read the complete Checker and Local Evaluation sub-skill and own the executable artifact and its validation, not merely audit another owner's evaluator. Give that evaluator builder only: + +``` +complete Checker and Local Evaluation sub-skill as the implementation contract +verbatim primary problem packet and official original files +only necessary verified toolchain, process, resource, and filesystem facts +an isolated writable path and role/output handoff contract only, + with no main-agent-derived evaluator semantics +``` + +Initially withhold the champion, solver source, attempt log, current methods, main-agent reasoning, diagnoses, expected results, claimed scores, and unverified summaries. Require the evaluator builder to freeze its evaluator contract, implementation, and initial fixtures before receiving raw candidate outputs, transcripts, or raw official responses for differential validation. Never replace a raw artifact with the main agent's interpretation of it. + +Require the evaluator-builder handoff to contain: + +``` +evaluator source/executable paths and content hashes +exact end-to-end run and applicable build commands +valid, invalid, boundary, mutation, and tiny hand-computed fixtures +coverage table, parameterized generator and validator commands, and seed manifest when generated cases are needed +original and minimized failing cases with full provenance when a failure is found +real-pipe protocol and transcript tests when interactive +independent legality and raw-objective recomputation when scored +official/local comparison evidence and unresolved discrepancies +``` + +Run those commands and adversarial fixtures before registering the evaluator in the verified fact ledger. If local and official evidence disagree, stop challenger evaluation and minimize that discrepancy first. The solver owner may execute the frozen evaluator but may not alter it to accept a candidate; evaluator changes require official evidence, a hand-computed counterexample, or a failing evaluator test. If delegation is unavailable, perform a sealed evaluator pass with separate files, representations, derivations, and fixtures before resuming the two review roles. + +## Run two independent fresh-context reviews + +Use two fresh isolated sub-agents when the environment supports delegation. Prefer the least inherited conversation context and give each a distinct writable path that does not yet exist. Do not let either reviewer see the other's preliminary conclusions, and do not select the next route before both results return. + +Give both reviewers the complete verbatim primary problem packet and the frozen evaluator interface. Do not substitute a problem summary or solver narrative for original artifacts. Keep the main agent's reasoning and unreliable results out of the initial review packets; reveal only the role-specific verified evidence described below and only in the stated order. + +Give each reviewer the primary problem packet, only the role-specific projection of the verified fact ledger described below, and this write and evaluation contract: + +``` +You may create and edit files only under: +Do not modify, delete, rename, or overwrite any pre-existing file. +Treat every pre-existing artifact, including the champion, as read-only. +You may build, run, evaluate, and, within the delegated quota, submit challengers +created in your isolated path. The quota allocation is the submission authorization; +do not request main-agent approval again for each in-quota submission. +Return the path of every file you create. +``` + +Do not place either reviewer in a read-only environment or globally forbid file creation. If the user or enclosing workflow authorizes remote submissions and at least one submission remains, reserve an explicit quota for sub-agent evaluation; allocate at least one submission to the method researcher instead of retaining the entire budget for the main agent. When authorization and remaining budget permit multiple submissions, give each selected rank-1 track a multi-submission iteration quota before funding lower-ranked methods or retaining unused budget for the main agent. State the evaluator, final metric, quota, deadline, and whether results are synchronous. A sub-agent may not exceed that quota or infer broader authorization. If only the main agent can access the submission transport, it must relay the sub-agent's exact artifact unchanged for each iteration and return the raw evaluator response; it may not substitute a different candidate or make a method decision at that boundary. + +Both reviewers should default to returning a self-authored executable challenger and owning its implementation, compile, local validation, comparison, repair/improvement iterations, and authorized official submissions through the terminal result or quota. Prefer reviewer-owned evaluation over handing code or an unevaluated proposal back to the main agent for subjective screening. If no faithful prototype fits the assigned resources or no submission is authorized, record the exact blocker and return the strongest executable partial artifact; do not silently transfer implementation or method judgment to the main agent. + +Require compilable source in the target language, exact build/run commands, and portable C++17 when the target does not specify a language standard. Any new checker, scorer, interactor, simulator, or evaluation harness must follow [Checker and Local Evaluation](../checker-and-local-evaluation/SKILL.md), should be C++ unless the verified interface makes that genuinely impractical, and must remain owned independently from the challenger it evaluates. + +For both roles, freeze an ordered remedy ranking before implementation. Rank by the evidence-weighted expected terminal final official aggregate score achievable under the full verified remaining hard time, compute, and submission budget, using source evidence, structural fit, comparable measurements, and uncertainty. *Applicable* means compatible with the verified contract and final judge limits; it does not mean easy to code or likely to fit in one sub-agent turn. Implementation difficulty, patch or code size, rewrite scope, familiarity, prototype speed, and current-turn convenience MUST NOT affect a method's rank. Small or quick tests may order diagnostics within a method, never the methods themselves. + +Explore aggressively across high-upside representations, objectives, decompositions, neighborhoods, planning horizons, relaxations, and algorithm families, including routes that require a substantial rewrite. Aggressive exploration means ambitious method choice plus persistent evidence-driven iteration; it never waives legality, evaluator fidelity, final judge limits, or submission authorization. *Assigned/delegated resources* means explicit user scope, hard deadline, compute, filesystem, and authorized submission quota, never the length of one sub-agent turn. When follow-up turns are supported, continue the same isolated rank-1 owner and writable path instead of treating a turn boundary as a blocker. + +Phrase each rank-1 assignment as: + +``` +Implement the rank-1 applicable method by expected terminal official score. +Start with its smallest faithful structural prototype, preserve its defining +mechanism, then iteratively repair and improve it through the delegated +evaluation and submission quota. +``` + +`Smallest` constrains only the first experiment, never the method ranking. If the rank-1 method cannot be completed within the explicit hard delegated resources after available follow-up turns, preserve its rank, record the exact blocker and minimum additional requirement, and return its strongest faithful partial artifact. Do not silently substitute an easier lower-ranked method; move lower only after the recorded-elimination gate is satisfied. + +For either role, use this protected rank-1 loop within the authorized quota: + +``` +raw local/official result -> diagnose defining weakness from evidence +-> method-faithful repair, completion, or structural refinement +-> local legality/objective validation -> next authorized submission -> repeat +``` + +If remote submission is not authorized, run the same loop with local evaluation only. Stop the track only when its explicit quota or deadline is exhausted, its predeclared defining hypothesis is falsified, a hard incompatibility or invalidity remains after the required focused repair, or evidence shows that further method-faithful iterations cannot materially improve the official metric. The first compiling prototype, first official score, or one under-tuned or non-improving submission is not terminal rejection evidence. + +Return executable code and fixtures plus a completed ordered iteration record, not a recommendation for the main agent to judge. Label each completed official response `EXACT_OFFICIAL` and each reproducible local result under an identified evaluator `EXACT_LOCAL`, while distinguishing an iteration result from the track's terminal status. Otherwise report `NOT_EVALUATED` and the concrete unavailable authorization, quota, interface, or resource; never replace a missing score with an estimate. + +The iteration record for each role must include: + +``` +rank-1 track identity, expected-score rationale, quota and stop conditions +for every iteration: candidate path/hash, diagnosis, and material change +exact build, run, validation and submission commands per iteration +evaluator/version, instance set, seeds and resource budget +submission ids, timestamps and raw responses for every remote iteration +per-iteration validity/verdict, raw objective, final score, runtime and memory +per-case data and comparison with the frozen champion under the same final metric +``` + +Assign distinct roles: + +### Trajectory critic + +First reconstruct the objective, bottleneck, and likely failure layer from the primary problem packet and verified fact ledger. Freeze that reconstruction before receiving the separate attempt log. Then audit: + +- model and evaluator fidelity; +- representation and invariants; +- neighborhood reachability and proxy alignment; +- assumptions rejected without evidence; +- experiment design, variance, and overfitting. + +Return ranked root causes, the strongest competing explanation, and `3-5` structurally different remedies. Rank the remedies under the shared expected-score rule before designing their experiments. For each leading remedy, specify the smallest falsifying experiment, success threshold, and abort condition. Implement the rank-1 applicable remedy in the assigned path, then repair and improve it through its delegated evaluation/submission quota rather than replacing it because the first faithful prototype is difficult or initially weak. + +### Method researcher + +Research the original problem independently, not the incumbent solver or the main agent's diagnosis. Initially receive the complete primary problem packet and only verified facts necessary to understand the contract, hard budgets, target, and available evaluation interface. Do not receive the champion identity or score, attempt log, current method labels, rejected-method rationales, trajectory critic conclusions, or any unverified summary before freezing an independent research map. After that map is saved, the exact champion interface and score plus reproducible negative results may be revealed only to prevent duplicate implementation and to support a direct comparison. + +Search from the problem outward: + +- exact title, distinctive statement phrases, source contest, editorials, leaderboards, and strong solution write-ups; +- canonical problem names, aliases, mathematical structure, objective, constraints, and scale; +- primary papers, surveys, references and citation chains, including adjacent fields with the same optimization structure; +- relevant benchmarks, public datasets, instance generators, baselines, winning methods, and evaluation protocols; +- maintained implementations and repositories that clarify a method's defining mechanism. + +Do not stop at the current solver family. Continue targeted searches until new queries only repeat already-mapped method families or the explicit research budget ends. Keep a source/evidence index; a concise decision memo may link to that index and must not discard useful sources merely to satisfy an arbitrary link cap. + +Map every candidate to the exact contract. Distinguish source-backed facts, independently derived facts, measured results, and unresolved hypotheses. Freeze an ordered ranking under the shared expected-score rule before prototyping. Implement the rank-1 applicable method first: begin with its smallest faithful prototype, then iterate toward the full scoring implementation. Select a lower-ranked method only after every higher-ranked method has one of these recorded in the ordered candidate ledger: + +- a minimally faithful prototype that still fails its defining hypothesis after the predeclared repair/iteration budget, with its commands, artifacts, and results; +- a demonstrated incompatibility with the verified contract; +- a measured runtime or memory failure against a hard budget; +- independently verified implementation invalidity that remains after the required focused repair. + +Implementation convenience, familiarity, code size, or a main-agent preference is not rejection evidence and does not permit skipping a higher-ranked method. For the current highest-ranked eligible method, own the full track end to end: + +``` +research -> faithful implementation -> compile -> local legality/objective checks +-> champion comparison -> authorized official submission -> diagnose and iterate +-> terminal evaluator result or exhausted delegated quota +``` + +If delegation is unavailable, perform the two roles as explicitly separate passes: write and seal the trajectory audit before starting a source-backed method search, and do not rewrite the first report to match the second. Preserve the same role-specific information barrier. + +## Apply the independent results + +After both reviews return: + +1. Register every leading proposal and preserve each reviewer's ranking and evidence; do not rewrite the reports into a main-agent ranking before disposition. +2. Accept an already terminally evaluated sub-agent result as evidence. Do not send it back to the main agent for a second subjective method judgment. +3. Process both frozen rankings in order and give each role's rank-1 applicable unresolved method a protected iterative implementation/evaluation slot. If scope leaves only one slot, choose between the two rank-1 methods by expected terminal official score evidence, not implementation ease. Do not select a lower-ranked method until every higher-ranked method satisfies the recorded-elimination gate above. +4. Give every leading proposal exactly one disposition: `EVALUATED`, `TEST_REQUIRED`, `REJECTED_WITH_EVIDENCE`, or `BLOCKED_BY_SCOPE`. A label without its command, artifact, measurement, proof, primary contract citation, or exact immutable scope boundary is incomplete. +5. Permit at most one or two already-defined cheap same-family scalar experiments to finish before the protected rank-1 structural tracks. Do not begin another tuning sequence. + +A renamed version of the current solver is not a structural prototype. + +`BLOCKED_BY_SCOPE` may stop the workflow, but it does not eliminate a higher-ranked method or authorize selection of a lower-ranked one. + +For this gate, *leading* includes each reviewer's rank-1 method and every challenger already returned as executable code or with a terminal evaluation. The main agent may add candidates but may not remove these mandatory dispositions. When evidence conflicts, prefer a comparable terminal official result, then a reproducible independent local result, then a checked proof or measured resource bound, then source-backed applicability evidence; an opinion never overrides a higher tier. + +Use [model and route algorithms](../model-and-route-algorithms/SKILL.md#5-establish-the-operation-and-memory-envelope) to test whether a proposed exact or hybrid class fits the contract and resource envelope, [technique selection](../../references/technique-selection.md) to compare concrete exact or data-structure families, and [heuristic search](../../references/heuristic-search.md) to instantiate and measure scored-search alternatives. The reviews choose hypotheses; the model-and-route sub-skill supplies the feasibility gate, while the shared references supply algorithm-family and search mechanics. + +## Bind main-agent discretion to evidence + +Reject a researched method only when concrete evidence shows one of these: + +- a cited assumption contradicts the verified original contract; +- a minimally faithful prototype still fails its predeclared defining hypothesis after its focused repair/iteration budget; +- a derived bound or measured runtime or memory exceeds a hard budget without credible optimization margin; +- an identified reconstruction, serialization, or hard-validity condition fails after one focused repair cycle; +- after the protected rank-1 loop reaches one of the shared stop conditions above, the track's best exact result is no better than a competing alternative under the same final evaluator and comparable total iteration/submission budgets. + +`Risk too high`, unfamiliarity, restructuring cost, implementation size, disagreement with the current trajectory, or an unmeasured pessimistic estimate is not rejection evidence. Quantify any claimed risk as a violated hard constraint, a derived bound, a failed artifact, or a measured result. Repair trivial build or integration defects for one focused cycle before judging the method. If the main agent believes a prototype is unfaithful, it must name the missing defining mechanism and allow one focused repair instead of discarding the family. + +When a user-imposed authorization, submission quota, deadline, language, or artifact boundary genuinely makes evaluation impossible, record `BLOCKED_BY_SCOPE` with the exact remaining budget and the measured or derived minimum requirement. This is not evidence that the method is weak and must not be rewritten as `REJECTED_WITH_EVIDENCE`. + +When a reviewer returns code, keep it as a separate challenger and run: + +``` +inspect -> compile -> repair trivial integration defects once +-> legality/objective/runtime validation -> sub-agent-owned authorized submission +-> diagnose -> repair or improve -> repeat within delegated quota +-> record terminal result -> promote or reject with evidence +``` + +The sub-agent owns the ordered evaluation record and terminal score of its rank-1 track within the delegated quota; the main agent owns only champion-file mutation and mechanical promotion. Under the same established final evaluator, automatically promote a legal iteration whose exact final score is better and whose hard resource/protocol constraints pass, but do not end the protected track merely because an intermediate champion was promoted. Withhold promotion only for a concrete failing artifact or an explicit user-imposed release condition, and record it. Retain the champion when an iteration is invalid, worse, tied without another predeclared benefit, or not evaluated. Opinion cannot override an exact comparable result. + +## Exit condition + +End this escape cycle only after every selected rank-1 track has reached one of the shared stop conditions above and every other leading structural alternative has a terminal evaluation, concrete rejection evidence, or an immutable user-scope block with quantified requirements. Do not stop a selected rank-1 track after its first legal or improved score while delegated iterations remain and evidence predicts material gain. `Not evaluated`, `risky`, and an unevidenced main-agent veto do not satisfy the exit condition. Resume focused tuning only on the selected final method. Run this workflow again only after that materially new family also reaches a new plateau or new evidence invalidates the reviews. + +Return: + +``` +plateau evidence +role-specific input packets and reviewer rankings +source/evidence index, prototype and exact commands +submission ids and terminal champion-versus-challenger scores +promotion or evidence-backed rejection record +``` diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/reactive-online-decision-problem-solving/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/reactive-online-decision-problem-solving/SKILL.md new file mode 100644 index 000000000..7e9fbb722 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/reactive-online-decision-problem-solving/SKILL.md @@ -0,0 +1,148 @@ +--- +name: reactive-online-decision-problem-solving +description: >- + Design, debug, and improve decision policies that must earn immediate or + future reward under uncertainty, including reactive or online control, + repeated-round learning, exploration, and risk-aware action selection. Use + when actions occur in a live sequential process, their feedback changes later + choices in the same run or environment, and termination is a horizon, task + completion, or decision budget. Do not use for offline output-only AHC or + scorer-only optimization across development runs; use heuristic search. Use + interactive-problem-solving for pure query design or protocol mechanics. +--- + +# Reactive Online Decision Problem Solving + +Make good decisions under uncertainty. The central question is: **What should be done now?** An action's value combines immediate reward, future opportunity, information, and risk. + +``` +observe -> update belief/model -> generate actions -> legal/risk filter + -> choose and act -> receive feedback -> repeat +``` + +This includes online or AHC tasks only when execution contains a live reward-bearing sequence in which earlier actions change the state, observations, rewards, or later choices. It excludes offline output-only AHC and scorer-only optimization whose complete input is known and whose evaluator is called only between development runs; route those tasks to [heuristic search](../../references/heuristic-search.md) and [validation and experiments](../validation-and-experiments/SKILL.md). Repeated evaluation alone does not make a task reactive. + +## Boundary with information acquisition + +Use [interactive problem solving](../interactive-problem-solving/SKILL.md) when actions exist primarily to eliminate hidden hypotheses and the process stops once an answer is determined. Use this skill when live sequential actions must earn reward under current uncertainty and the process stops because the horizon, task, time, or decision budget ends. + +If a reactive task contains a separable calibration or hidden-state identification stage, use the Interactive skill for that subproblem while keeping overall action choice here. + +## 1. Define the decision contract + +Record the operational decision process: + +``` +observable state/history and latent uncertainty +legal actions or candidate-output representation +state transition and feedback timing +round, time, action, or evaluator budget +raw reward/cost, direction, transform, and aggregation +risk constraints and whole-run failure conditions +reset/persistence across rounds, cases, and evaluations +terminal conditions and mandatory completion requirements +``` + +Separate hard legality from quality. Replay state transitions and reward calculations literally. If local and official scoring disagree, route to [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) before changing the policy. + +## 2. Separate estimator, planner, and explorer + +Use only the layers the task needs, but keep their contracts distinct: + +1. **Estimator:** infer decision-relevant latent quantities and uncertainty from feedback. +2. **Planner:** choose the best feasible action under the current model and horizon. +3. **Explorer:** trade immediate cost for information that can improve later decisions. + +Test each layer independently: synthetic truth for the estimator, known parameters for the planner, and controlled horizons for the exploration rule. A strong score should not be the only evidence that all three are correct. + +Choose the simplest identifiable model that supports decisions: + +- per-item or per-edge estimates with shrinkage; +- feature, segment, group, or low-rank models for aggregate feedback; +- posterior distributions or confidence intervals when uncertainty changes actions; +- robust updates for heavy-tailed or corrupted observations; +- recency weighting or change detection for drift; +- censored or delayed-feedback models when outcomes are partial. + +Aggregate feedback may not identify every latent parameter. Optimize prediction of decision-relevant totals instead of pretending all components are known; check rank, conditioning, calibration, and regularization. + +## 3. Choose actions by utility, not uncertainty alone + +Define every outcome as a higher-is-better utility `U`; for a minimization objective, negate or otherwise reorient the cost first. For a separable information-gathering action `a`, feedback outcome `y`, and later decision `d`, a conceptual value-of-information test is: + +``` +VOI(a) = E_y[max_d E[U(d) | y, a]] + - max_d E[U(d)] + - immediate_and_opportunity_cost(a) +``` + +Exact VOI is often too expensive. Use a justified proxy such as uncertainty along likely decisions, disagreement between plausible models, confidence-bound optimism, posterior sampling, or expected regret reduction. Do not apply the pure-information expression when the action also changes state or reward; compare full action values under the real transition model instead. Do not explore uncertain regions that cannot affect later choices. + +Explore more when useful future decisions remain; exploit more near the end or in heavily weighted rounds. Base the schedule on confidence, remaining opportunity, and risk rather than only a hard-coded round number. + +When an action changes future feasibility, plan with the real transition and reserve recovery slack. A myopic reward can destroy reachability or mandatory completion. + +## 4. Build the planner and search policy + +Use exact combinatorial optimization inside each decision when it fits: shortest path, matching, flow, scheduling, DP, or a bounded exact local solve. Useful hybrids include learned costs plus exact routing, learned skills plus matching, posterior samples plus robust optimization, and rolling-horizon construction with bounded repair. + +For scored construction, neighborhoods, incremental deltas, SA, beam search, LNS, portfolios, time allocation, and other concrete optimization mechanisms, read [heuristic search](../../references/heuristic-search.md). That reference owns search-method details; this skill owns the uncertainty-aware policy wrapped around them. + +Maintain: + +``` +legal fallback policy or output +current model/policy state +best independently validated legal champion +experimental challenger +``` + +Never replace the champion with a seed-sensitive or invalid challenger. If controlled structural alternatives have stopped improving a verified champion, route to [plateau escape](../plateau-escape/SKILL.md). + +## 5. Evaluate repeated feedback correctly + +When a concrete legality, transition, reward, protocol, or official-feedback uncertainty needs local diagnosis, route to [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) to implement an independent episode replayer, scorer, interactor, or process harness. Coherent accepted online episodes do not require this route solely because the policy is reactive; local replay may be deferred while the open question is policy quality. Once routed, that sub-skill owns evaluator isolation, executable contracts, and adversarial evaluator tests; this skill owns the estimator, planner, explorer, and policy-level diagnosis. When the evaluator uses `testlib.h`, follow the concrete Testlib route linked there. + +Use [validation and experiments](../validation-and-experiments/SKILL.md) for champion/challenger discipline, paired cases/seeds, holdouts, and release checks. + +Separate randomness sources: + +``` +instance seed | transition/judge noise | solver RNG | host/runtime noise +``` + +Track per-round and per-instance evidence, not only one aggregate score: + +- cumulative and late-horizon reward; +- prediction residuals and calibration by uncertainty bucket; +- exploration cost versus measured later benefit; +- invalid action count and fallback use; +- score and runtime by instance features; +- median, quantiles, and lower-tail failures as well as mean. + +Pair policies on the same allowed inputs and noise seeds. One higher noisy score is not proof of a stronger decision rule. + +## 6. Safety and completion + +- Filter every proposed action through legality and risk constraints. +- Keep a conservative legal action when inference or planning fails. +- Bound variable-time replanning, repair, and exact subsolves. +- Reserve actions/time needed for coverage, return-to-start, final output, or other mandatory completion. +- Checkpoint the last-known-valid model and champion. +- Make tie-breaking and seeds reproducible while diagnosing. + +Do not add flush/query machinery to an offline scorer-only workflow. If a live judge protocol actually exists, route those mechanics to Interactive and [contest solver engineering](../contest-solver-engineering/SKILL.md). + +## Diagnostics + +| Symptom | Test first | +|---|---| +| Good prediction error, poor score | Planner objective, feasibility, horizon, proxy alignment | +| Early reward good, late reward poor | Update bias, drift, exploration schedule | +| High uncertainty persists | Identifiability, observation design, regularization | +| Exploration costs more than it returns | Decision relevance, remaining horizon, opportunity cost | +| Seeds diverge sharply | Fragile early updates, heavy tails, missing fallback | +| Runtime spikes | Replanning complexity, allocation, unbounded repair/subsolve | +| Local score rises, official score does not | Score transform, instance shift, overfit, evaluator mismatch | + +Return the decision contract, estimator/planner/explorer design, legality and fallback invariants, implementation, evaluations actually run, champion comparison, and residual uncertainty or risk. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/SKILL.md new file mode 100644 index 000000000..8eb0f4980 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/SKILL.md @@ -0,0 +1,146 @@ +--- +name: testlib-cpp-judging +description: "Guide C++ competitive-programming judging with the bundled testlib.h. Use when the user asks to author special or scored checkers, strict validators, deterministic generators, basic interactors, or a minimal local solution-checker workflow." +--- + +# Testlib C++ Judging + +## Scope + +Use this skill when a task involves `testlib.h`, a special judge, a custom +checker or scorer, an input validator, a deterministic test generator, or a +testlib interactor. It is intentionally C++-only and header-only. Do not create +a Python harness for the ordinary local judging workflow. + +This skill owns concrete Testlib APIs, streams, verdicts, command lines, and +templates. When evaluator-facing evidence has triggered independent contract or +oracle work, use [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) +to choose checker/scorer/interactor architecture, reconstruct legality and +objectives, and validate the evaluator itself. Do not enter that route merely +because a previously specified generator or validator is being implemented. + +## Start Here + +1. Read [the testlib usage guide](references/testlib-usage.md) before writing + checker, validator, generator, or interactor code. +2. Copy the bundled [testlib.h](scripts/testlib.h) into the working directory + beside the C++ sources, or add its directory to the compiler include path. +3. Put `#include "testlib.h"` before other includes. +4. Select exactly one registration function for each executable. +5. For ordinary offline judging, use the minimal commands below rather than + building a separate evaluation framework. + +## Choose the Executable Role + +| Role | Registration | Main testlib interfaces | Purpose | +| --- | --- | --- | --- | +| Checker | `registerTestlibCmd(argc, argv)` | `inf`, `ouf`, `ans`, `quitf` | Judge contestant output against the input and reference answer | +| Validator | `registerValidation(argc, argv)` | strict `inf`, `ensuref` | Reject malformed or out-of-constraint input | +| Generator | `registerGen(argc, argv, 1)` | `rnd`, `opt`, `println` | Produce deterministic tests from command-line parameters | +| Interactor | `registerInteraction(argc, argv)` | `inf`, `ouf`, `tout`, stdout | Exchange a protocol with an interactive solution | + +## Generate Data as Reproducible Code + +Testlib includes input-generation support; it is not limited to checkers and validators. When a local problem-finding campaign needs randomized, batch, or maximum-scale data, implement a problem-specific `generator.cpp` instead of hand-authoring large inputs or copying many variants. Parameterize the relevant case family, size, density/bias, structure, and seed tag with `opt`; use `rnd` for all randomness. The full command line determines Testlib's deterministic seed, so preserve that command exactly. + +Define the coverage families and their expected oracle, invariant, or failure target before generating volume. Use one deterministic generator invocation per case, or a small coded batch driver that records every invocation in a manifest. Validate every intended-valid generated case with an independent validator before running the solution, and retain the generator command, input hash, validator result, and any failing seed and artifacts. Follow [Checker and Local Evaluation](../checker-and-local-evaluation/SKILL.md#build-problem-finding-data-end-to-end) for the complete coverage-to-retention workflow. + +## Critical Checker Contract + +Always invoke a checker in this exact order: + +``` +checker +``` + +After registration, the mapping is: + +- `inf`: input file +- `ouf`: contestant or participant output +- `ans`: standard or jury answer + +Never swap `ouf` and `ans`. Even a checker that does not use the standard +answer still needs the third file argument; pass an empty placeholder file if +necessary. + +## Minimal Local Judging Flow + +Run these commands from a directory containing `testlib.h`, `solution.cpp`, +`checker.cpp`, `case.in`, and `case.ans`: + +```bash +g++ -std=c++17 -O2 solution.cpp -o solution +g++ -std=c++17 -O2 -I. checker.cpp -o checker + +timeout 2s ./solution < case.in > case.out +solver_status=$? +if [ "$solver_status" -ne 0 ]; then + echo "solution failed with status $solver_status" >&2 + exit "$solver_status" +fi +./checker case.in case.out case.ans +checker_status=$? +echo "$checker_status" +exit "$checker_status" +``` + +If the problem has `validator.cpp`, compile it and validate the input before +running the solution: + +```bash +g++ -std=c++17 -O2 -I. validator.cpp -o validator +./validator < case.in || exit 1 +``` + +Capture `solver_status` before running the checker; a timeout, signal, or other +nonzero solution exit is a solver failure and must not be relabeled as an output +verdict. Capture `checker_status` immediately after the checker. For a +verdict-only checker under the default local Testlib configuration, status `0` +means accepted and nonzero means rejection or judge failure. A points checker +using `quitp` instead returns a partial-points status (default `7`) that a +points-aware runner must parse separately. Preserve checker diagnostics and any +reported points with the status. + +## Authoring Rules + +- Use testlib readers instead of raw parsing for judged files. +- Return `_wa` for a semantically wrong contestant answer, `_pe` for malformed + contestant output when you detect it explicitly, `_fail` for a broken jury + answer or checker invariant, and `_ok` only after all required checks pass. +- In validators, describe the exact grammar with `readSpace`, `readEoln`, and + `readEof`; use bounded reads and `ensuref` for semantic constraints. +- In generators, use `rnd` rather than `rand`, `srand`, or + `random_shuffle`; identical command lines should reproduce identical data. +- Implement batch and large-case generation in generator code; do not maintain + hand-edited large input files as the source of truth. +- Run every intended-valid generated case through the independent validator and + preserve its full generator command and seed tag. +- Flush every interactor query with `std::endl` or an explicit flush. +- Keep `solution.cpp`, checker logic, validator logic, and answer generation as + separate concerns. + +## Boundaries + +- This skill explains the public `testlib.h` workflow, not development of the + testlib repository itself. +- For evaluator roles, this skill implements a previously derived contract. Use + [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) only + when contract, reconstruction, score-transform, or fidelity uncertainty is + evidence-triggered and blocks the current decision, or when official and local + behavior disagree. +- This skill owns concrete testlib implementation. Use + [interactive problem solving](../interactive-problem-solving/SKILL.md) for + protocol modeling, hidden hypotheses, query design, and adversarial strategy. +- It does not bundle a Python evaluator, a build system, repository tests, or + CI configuration. +- The simple three-file flow is for non-interactive judging. Interactive + solutions require a bidirectional process runner supplied by the judge. +- Partial scoring needs a runner that understands testlib points verdicts; do + not interpret every nonzero checker status as ordinary wrong answer in that + mode. + +## Troubleshooting + +- Read [troubleshooting](references/troubleshooting.md) when compilation, + checker arguments, strict whitespace, exit status, generator reproducibility, + or interactor flushing causes a failure. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/testlib-usage.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/testlib-usage.md new file mode 100644 index 000000000..bcaca5eee --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/testlib-usage.md @@ -0,0 +1,303 @@ +# testlib.h Usage Guide + +## Purpose + +Use this reference to author small, self-contained C++ programs around the +bundled `testlib.h`. The primary workflow is an offline checker with an +optional validator. Generator and interactor coverage is included because they +use the same header, but they do not change the minimal checker command line in +the root skill. + +The API facts and examples here were distilled from testlib 0.9.45, its bundled +header, and the repository's C++ examples. + +## Header Setup + +Testlib is a single header. Copy the bundled header from this skill's +`scripts/` directory into the working directory and include it first: + +```cpp +#include "testlib.h" + +#include +``` + +No library link flag is required. Compile each checker, validator, generator, +or interactor as its own executable. The `-I.` flag works when `testlib.h` is +in the current directory; otherwise point `-I` at the directory that contains +the bundled header. + +Including testlib first also lets it replace compiler-specific random helpers. +For generator code, use `rnd` and testlib's `shuffle` instead of `rand`, +`srand`, or `random_shuffle`. + +## Checker + +### Minimal checker template + +This checker compares one signed 64-bit integer from the standard answer with +one integer from the contestant output: + +```cpp +#include "testlib.h" + +int main(int argc, char* argv[]) { + setName("compare one signed 64-bit integer"); + registerTestlibCmd(argc, argv); + + long long expected = ans.readLong(); + long long actual = ouf.readLong(); + + if (actual != expected) { + quitf(_wa, "expected %lld, found %lld", expected, actual); + } + + quitf(_ok, "answer is %lld", actual); +} +``` + +Compile it with the command in the root skill and run it with three file paths +in this exact semantic order: input, contestant output, standard answer. + +### Streams + +| Stream | Source | Typical use | +| --- | --- | --- | +| `inf` | First checker file argument | Read problem input, constraints, dimensions, or instance data | +| `ouf` | Second checker file argument | Parse and validate contestant output | +| `ans` | Third checker file argument | Read the jury answer, reference objective, or expected tokens | + +For non-unique or constructive answers, validate `ouf` against the instance in +`inf`. Use `ans` only for data that is genuinely needed, such as an optimal +objective value. Do not reject a valid construction merely because its text is +different from the jury construction. + +### Common readers + +| Need | Typical call | +| --- | --- | +| 32-bit integer | `readInt()` or `readInt(lo, hi, "name")` | +| 64-bit integer | `readLong()` or `readLong(lo, hi, "name")` | +| Floating-point value | `readDouble()` or `readDouble(lo, hi, "name")` | +| Whitespace-delimited token | `readToken()` | +| Token matching a pattern | `readToken("[a-z]+", "word")` | +| Whole line | `readLine()` or `readString()` | +| Several bounded values | `readInts`, `readLongs`, or `readDoubles` | +| End probe ignoring trailing blanks | `seekEof()` | + +Bound contestant values while reading when the range is part of the output +contract. Testlib turns malformed or out-of-range judged output into an +appropriate non-accepted result. A successful `quitf(_ok, ...)` also checks +for extra non-whitespace data remaining in `ouf`. + +`seekEof()` is a Boolean probe. Calling it without checking the returned value +does not reject trailing output. Test the result explicitly when probing early, +or rely on the successful `_ok` termination check after consuming all permitted +output. + +### Verdicts + +| Result | Meaning | Default local exit code | +| --- | --- | --- | +| `_ok` | Contestant output is accepted | `0` | +| `_wa` | Output is well-formed but semantically wrong | `1` | +| `_pe` | Contestant output format is invalid | `2` | +| `_fail` | Checker, input, or jury-answer failure | `3` | +| `quitp(points, ...)` | Points or partial score | `7` | + +These codes are defaults. Judge-specific compile-time macros can remap them, +so portable orchestration should distinguish accepted from non-accepted and +preserve the diagnostic rather than assuming every platform uses the same +number. + +Use `_fail`, not `_wa`, when the official answer is malformed or an internal +checker invariant is false. This prevents a judge-data problem from being +reported as a contestant mistake. + +### Checker design sequence + +1. Call `setName` and `registerTestlibCmd` near the start of `main`. +2. Read the instance data required from `inf`. +3. Read and sanity-check jury data from `ans` if the checker needs it. +4. Parse the contestant result from `ouf` with explicit types and bounds. +5. Validate syntax, feasibility, and objective value in that order. +6. Call `quitf` with a short diagnostic that includes the first useful + mismatch or violated condition. +7. Call `_ok` only when the entire required output has been consumed and all + semantic checks pass. + +For multi-case output, call `setTestCase(i)` while checking each case so error +messages identify the failing case. + +## Validator + +### Strict validator template + +This template accepts an integer `n`, then exactly `n` space-separated values +on the next line, and applies an example semantic rule that their sum must be +non-negative: + +```cpp +#include "testlib.h" + +int main(int argc, char* argv[]) { + registerValidation(argc, argv); + + int n = inf.readInt(1, 200000, "n"); + inf.readEoln(); + + long long sum = 0; + for (int i = 0; i < n; ++i) { + if (i > 0) { + inf.readSpace(); + } + sum += inf.readLong(-1000000000LL, 1000000000LL, "a_i"); + } + inf.readEoln(); + inf.readEof(); + + ensuref(sum >= 0, "sum must be non-negative"); +} +``` + +`registerValidation` maps standard input to strict `inf`. In strict mode, +format is part of validity: read spaces, line endings, and EOF explicitly. +Always finish a complete validator with `inf.readEof()`; returning without it +is treated as an incomplete validator. + +Use names such as `"n"` and `"a_i"` on bounded reads. They improve error +messages and validator metadata. Use `ensuref(condition, ...)` for constraints +that span multiple values, such as uniqueness, graph simplicity, +connectivity, or a total-sum limit. + +The root skill shows the compile command and the stdin invocation. Run the +validator before executing the solution; stop the case immediately when the +validator returns nonzero. + +## Generator + +### Minimal deterministic generator + +```cpp +#include "testlib.h" + +int main(int argc, char* argv[]) { + registerGen(argc, argv, 1); + + int n = opt(1); + println(n); + println(rnd.perm(n, 1)); +} +``` + +Compile and run it as follows after placing `testlib.h` beside the source: + +```bash +g++ -std=c++17 -O2 -I. generator.cpp -o generator +./generator 10 > case.in +``` + +`registerGen(argc, argv, 1)` seeds `rnd` from the command line. Repeating the +same executable with the same arguments reproduces the same test. Change the +arguments to change the seed. + +Useful primitives include: + +- `rnd.next(lo, hi)` for a uniform inclusive integer range. +- `rnd.next("[a-z]{1,20}")` for a token generated from a testlib pattern. +- `rnd.wnext(lo, hi, bias)` for a biased distribution. +- `rnd.perm(n, first)` for a permutation. +- `rnd.distinct(count, lo, hi)` for distinct values. +- `rnd.partition(count, sum, minPart)` for an integer partition. +- `opt(1)` for a positional argument and `opt("n")` for `-n` or + `--n` style options. +- `println` for stable whitespace-separated lines; use standard C++ output when + custom no-newline formatting is required. + +After generation, validate every intended-valid produced file with the problem +validator. If a routed randomized, batch, or large-data campaign has no +validator yet, implement the independent validator before relying on the +generated cases. + +### Parameterized campaigns and large cases + +Treat a generator as executable provenance for a coverage family. For randomized, batch, or large cases, expose the family and relevant structural parameters explicitly rather than hand-editing the emitted file: + +```cpp +#include "testlib.h" + +int main(int argc, char* argv[]) { + registerGen(argc, argv, 1); + + std::string family = opt("family"); + int n = opt("n"); + int bias = opt("bias"); + long long seedTag = opt("seed"); + + // The complete command line, including seedTag, determines rnd's seed. + // Implement each problem-specific family and print exactly one valid case. + ensuref(n > 0, "n must be positive"); + (void)family; + (void)bias; + (void)seedTag; + println(n); +} +``` + +A case can then be reproduced from its full command: + +```bash +./generator --family=random --n=200000 --bias=2 --seed=104729 > case.in +./validator < case.in || exit 1 +``` + +Implement the actual family semantics in C++; the placeholder above only demonstrates parameter and seed provenance. Use a small coded driver to enumerate a batch parameter matrix, invoke the generator once per case, validate each emitted file, and record a manifest. Keep at least the case id, family, full command, generator hash/version, input hash, and validator result. Preserve failing commands and seeds with the corresponding input, solver output, evaluator diagnostic, and transcript. Do not use manually assembled maximum-size files or unlabeled random dumps as reproducible test sources. + +## Interactor + +An interactor has different process wiring from an offline checker: + +```cpp +#include "testlib.h" + +#include + +int main(int argc, char* argv[]) { + registerInteraction(argc, argv); + + long long a = inf.readLong(); + long long b = inf.readLong(); + std::cout << a << ' ' << b << std::endl; + + long long reply = ouf.readLong(); + tout << reply << std::endl; + + if (reply != a + b) { + quitf(_wa, "expected %lld, found %lld", a + b, reply); + } + quitf(_ok, "interaction completed"); +} +``` + +The interactor reads test data from `inf`, reads the participant process from +`ouf` (the interactor's stdin), writes protocol messages to stdout, and writes +a transcript or checker input to `tout`. Every query must be flushed. A judge +or dedicated interactive runner must connect the two processes; the root +skill's `solution < case.in > case.out` command is not an interactive runner. + +## Minimal File Layout + +``` +work/ + testlib.h + solution.cpp + checker.cpp + validator.cpp # optional + generator.cpp # optional + case.in + case.ans + case.out # produced by the solution +``` + +Keep the header copied with the judging sources so the compile commands are +portable and do not depend on the original testlib repository checkout. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/troubleshooting.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/troubleshooting.md new file mode 100644 index 000000000..bcb2d1ebf --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/references/troubleshooting.md @@ -0,0 +1,73 @@ +# Troubleshooting + +## Header Not Found + +Symptom: compilation reports `fatal error: testlib.h: No such file or directory`. + +Copy the bundled `scripts/testlib.h` from this skill into the working directory +and compile with `-I.`, or pass `-I` the directory that actually contains the +header. Testlib is header-only; no library link flag is needed. + +## Checker Rejects Its Command Line + +Symptom: the checker says it must be run with input, output, and answer files. + +Use exactly: + +``` +./checker case.in case.out case.ans +``` + +The second path is contestant output and the third is the standard answer. If +the checker ignores `ans`, still provide an existing placeholder file. + +## Checker Gives Surprising Results + +- Confirm that `ouf` is read as contestant output and `ans` as jury output. +- A default exit code of `1` is wrong answer, `2` is presentation/format error, + and `3` is a checker or judge-data failure. +- `quitf(_ok, ...)` checks for extra non-whitespace contestant output. Consume + all permitted output before accepting. +- Preserve stderr: it contains the checker verdict and diagnostic. +- Run `echo $?` immediately after the checker, before any other command changes + the shell status. + +## Validator Rejects Apparently Valid Input + +Validators are strict. `readSpace()` expects the required separator, +`readEoln()` expects the line ending at that point, and `readEof()` expects no +remaining bytes. Compare the file's exact whitespace with the validator's +grammar. Do not replace strict reads with loose parsing merely to accept a +malformed generated test. + +## Solution Timeout or Crash Is Confused with Checker Status + +GNU `timeout` commonly returns `124` when the time limit expires. That status +belongs to the solution command, while the later status belongs to the +checker. In a more defensive shell loop, capture each status immediately and +report whether the failure came from solution execution or output checking. + +## Generator Is Not Reproducible + +- Use `registerGen(argc, argv, 1)`. +- Use `rnd` and testlib's `shuffle`, not standard `rand`, `srand`, or + `random_shuffle`. +- Keep the full command line and bundled header version unchanged. +- Remember that changing any generator argument changes the seed. + +## Interactor Hangs + +- Flush every query with `std::endl` or `std::flush`. +- Confirm that a bidirectional runner connects solution stdout to interactor + stdin and interactor stdout to solution stdin. +- Do not run an interactive solution with the ordinary redirected offline + command. +- Treat unexpected EOF as a protocol or process failure and include the last + completed protocol step in the diagnostic. + +## Partial Scoring Looks Like Failure + +`quitp(points, ...)` uses the points result and normally exits with code `7`, +not `0`. A simple zero/nonzero shell policy is appropriate only for accepted +versus non-accepted checking. A scored task needs an orchestrator that parses +and aggregates the points verdict. diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/scripts/testlib.h b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/scripts/testlib.h new file mode 100644 index 000000000..349b839ff --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/testlib-cpp-judging/scripts/testlib.h @@ -0,0 +1,6252 @@ +/* + * It is strictly recommended to include "testlib.h" before any other include + * in your code. In this case testlib overrides compiler specific "random()". + * + * If you can't compile your code and compiler outputs something about + * ambiguous call of "random_shuffle", "rand" or "srand" it means that + * you shouldn't use them. Use "shuffle", and "rnd.next()" instead of them + * because these calls produce stable result for any C++ compiler. Read + * sample generator sources for clarification. + * + * Please read the documentation for class "random_t" and use "rnd" instance in + * generators. Probably, these sample calls will be useful for you: + * rnd.next(); rnd.next(100); rnd.next(1, 2); + * rnd.next(3.14); rnd.next("[a-z]{1,100}"). + * + * Also read about wnext() to generate off-center random distribution. + * + * See https://github.com/MikeMirzayanov/testlib/ to get latest version or bug tracker. + */ + +#ifndef _TESTLIB_H_ +#define _TESTLIB_H_ + +/* + * Copyright (c) 2005-2025 + */ + +#define VERSION "0.9.45" + +/* + * Mike Mirzayanov + * + * This material is provided "as is", with absolutely no warranty expressed + * or implied. Any use is at your own risk. + * + * Permission to use or copy this software for any purpose is hereby granted + * without fee, provided the above notices are retained on all copies. + * Permission to modify the code and to distribute modified code is granted, + * provided the above notices are retained, and a notice that the code was + * modified is included with the above copyright notice. + * + */ + +/* NOTE: This file contains testlib library for C++. + * + * Check, using testlib running format: + * check.exe [ [-appes]], + * If result file is specified it will contain results. + * + * Validator, using testlib running format: + * validator.exe < input.txt, + * It will return non-zero exit code and writes message to standard output. + * + * Generator, using testlib running format: + * gen.exe [parameter-1] [parameter-2] [... paramerter-n] + * You can write generated test(s) into standard output or into the file(s). + * + * Interactor, using testlib running format: + * interactor.exe [ [ [-appes]]], + * Reads test from inf (mapped to args[1]), writes result to tout (mapped to argv[2], + * can be judged by checker later), reads program output from ouf (mapped to stdin), + * writes output to program via stdout (use cout, printf, etc). + */ + +const char *latestFeatures[] = { + "Remove incorrect const attributes", + "Added ConstantBoundsLog, VariablesLog to validator testOverviewLogFile", + "Use setAppesModeEncoding to change xml encoding from windows-1251 to other", + "rnd.any/wany use distance/advance instead of -/+: now they support sets/multisets", + "Use syntax `int t = inf.readInt(1, 3, \"~t\");` to skip the lower bound check. Tildes can be used on either side or both: ~t, t~, ~t~", + "Supported EJUDGE support in registerTestlibCmd", + "Supported '--testMarkupFileName fn' and '--testCase tc/--testCaseFileName fn' for validators", + "Added opt defaults via opt(key/index, default_val); check unused opts when using has_opt or default opt (turn off this check with suppressEnsureNoUnusedOpt()).", + "For checker added --group and --testset command line params (like for validator), use checker.group() or checker.testset() to get values", + "Added quitpi(points_info, message) function to return with _points exit code 7 and given points_info", + "rnd.partition(size, sum[, min_part=1]) returns random (unsorted) partition which is a representation of the given `sum` as a sum of `size` positive integers (or >=min_part if specified)", + "rnd.distinct(size, n) and rnd.distinct(size, from, to)", + "opt(\"some_missing_key\") returns false now", + "has_opt(key)", + "Abort validator on validator.testset()/validator.group() if registered without using command line", + "Print integer range violations in a human readable way like `violates the range [1, 10^9]`", + "Opts supported: use them like n = opt(\"n\"), in a command line you can use an exponential notation", + "Reformatted", + "Use setTestCase(i) or unsetTestCase() to support test cases (you can use it in any type of program: generator, interactor, validator or checker)", + "Fixed issue #87: readStrictDouble accepts \"-0.00\"", + "Fixed issue #83: added InStream::quitif(condition, ...)", + "Fixed issue #79: fixed missed guard against repeated header include", + "Fixed issue #80: fixed UB in case of huge quitf message", + "Fixed issue #84: added readXs(size, indexBase = 1)", + "Fixed stringstream repeated usage issue", + "Fixed compilation in g++ (for std=c++03)", + "Batch of println functions (support collections, iterator ranges)", + "Introduced rnd.perm(size, first = 0) to generate a `first`-indexed permutation", + "Allow any whitespace in readInts-like functions for non-validators", + "Ignore 4+ command line arguments ifdef EJUDGE", + "Speed up of vtos", + "Show line number in validators in case of incorrect format", + "Truncate huge checker/validator/interactor message", + "Fixed issue with readTokenTo of very long tokens, now aborts with _pe/_fail depending of a stream type", + "Introduced InStream::ensure/ensuref checking a condition, returns wa/fail depending of a stream type", + "Fixed compilation in VS 2015+", + "Introduced space-separated read functions: readWords/readTokens, multilines read functions: readStrings/readLines", + "Introduced space-separated read functions: readInts/readIntegers/readLongs/readUnsignedLongs/readDoubles/readReals/readStrictDoubles/readStrictReals", + "Introduced split/tokenize functions to separate string by given char", + "Introduced InStream::readUnsignedLong and InStream::readLong with unsigned long long parameters", + "Supported --testOverviewLogFileName for validator: bounds hits + features", + "Fixed UB (sequence points) in random_t", + "POINTS_EXIT_CODE returned back to 7 (instead of 0)", + "Removed disable buffers for interactive problems, because it works unexpectedly in wine", + "InStream over string: constructor of InStream from base InStream to inherit policies and std::string", + "Added expectedButFound quit function, examples: expectedButFound(_wa, 10, 20), expectedButFound(_fail, ja, pa, \"[n=%d,m=%d]\", n, m)", + "Fixed incorrect interval parsing in patterns", + "Use registerGen(argc, argv, 1) to develop new generator, use registerGen(argc, argv, 0) to compile old generators (originally created for testlib under 0.8.7)", + "Introduced disableFinalizeGuard() to switch off finalization checkings", + "Use join() functions to format a range of items as a single string (separated by spaces or other separators)", + "Use -DENABLE_UNEXPECTED_EOF to enable special exit code (by default, 8) in case of unexpected eof. It is good idea to use it in interactors", + "Use -DUSE_RND_AS_BEFORE_087 to compile in compatibility mode with random behavior of versions before 0.8.7", + "Fixed bug with nan in stringToDouble", + "Fixed issue around overloads for size_t on x64", + "Added attribute 'points' to the XML output in case of result=_points", + "Exit codes can be customized via macros, e.g. -DPE_EXIT_CODE=14", + "Introduced InStream function readWordTo/readTokenTo/readStringTo/readLineTo for faster reading", + "Introduced global functions: format(), englishEnding(), upperCase(), lowerCase(), compress()", + "Manual buffer in InStreams, some IO speed improvements", + "Introduced quitif(bool, const char* pattern, ...) which delegates to quitf() in case of first argument is true", + "Introduced guard against missed quitf() in checker or readEof() in validators", + "Supported readStrictReal/readStrictDouble - to use in validators to check strictly float numbers", + "Supported registerInteraction(argc, argv)", + "Print checker message to the stderr instead of stdout", + "Supported TResult _points to output calculated score, use quitp(...) functions", + "Fixed to be compilable on Mac", + "PC_BASE_EXIT_CODE=50 in case of defined TESTSYS", + "Fixed issues 19-21, added __attribute__ format printf", + "Some bug fixes", + "ouf.readInt(1, 100) and similar calls return WA", + "Modified random_t to avoid integer overflow", + "Truncated checker output [patch by Stepan Gatilov]", + "Renamed class random -> class random_t", + "Supported name parameter for read-and-validation methods, like readInt(1, 2, \"n\")", + "Fixed bug in readDouble()", + "Improved ensuref(), fixed nextLine to work in case of EOF, added startTest()", + "Supported \"partially correct\", example: quitf(_pc(13), \"result=%d\", result)", + "Added shuffle(begin, end), use it instead of random_shuffle(begin, end)", + "Added readLine(const string& ptrn), fixed the logic of readLine() in the validation mode", + "Package extended with samples of generators and validators", + "Written the documentation for classes and public methods in testlib.h", + "Implemented random routine to support generators, use registerGen() to switch it on", + "Implemented strict mode to validate tests, use registerValidation() to switch it on", + "Now ncmp.cpp and wcmp.cpp are return WA if answer is suffix or prefix of the output", + "Added InStream::readLong() and removed InStream::readLongint()", + "Now no footer added to each report by default (use directive FOOTER to switch on)", + "Now every checker has a name, use setName(const char* format, ...) to set it", + "Now it is compatible with TTS (by Kittens Computing)", + "Added \'ensure(condition, message = \"\")\' feature, it works like assert()", + "Fixed compatibility with MS C++ 7.1", + "Added footer with exit code information", + "Added compatibility with EJUDGE (compile with EJUDGE directive)", + "Added compatibility with Contester (compile with CONTESTER directive)" +}; + +#ifdef _MSC_VER +#define _CRT_SECURE_NO_DEPRECATE +#define _CRT_SECURE_NO_WARNINGS +#define _CRT_NO_VA_START_VALIDATION +#endif + +/* Overrides random() for Borland C++. */ +#define random __random_deprecated +#include +#include +#include +#include +#undef random + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef TESTLIB_THROW_EXIT_EXCEPTION_INSTEAD_OF_EXIT +# include +#endif + +#if (_WIN32 || __WIN32__ || __WIN32 || _WIN64 || __WIN64__ || __WIN64 || WINNT || __WINNT || __WINNT__ || __CYGWIN__) +# if !defined(_MSC_VER) || _MSC_VER > 1400 +# define NOMINMAX 1 +# include +# else +# define WORD unsigned short +# include +# endif +# include +# define ON_WINDOWS +# if defined(_MSC_VER) && _MSC_VER > 1400 +# pragma warning( disable : 4127 ) +# pragma warning( disable : 4146 ) +# pragma warning( disable : 4458 ) +# endif +#else +# define WORD unsigned short +# include +#endif + +#if defined(FOR_WINDOWS) && defined(FOR_LINUX) +#error Only one target system is allowed +#endif + +#ifndef LLONG_MIN +#define LLONG_MIN (-9223372036854775807LL - 1) +#endif + +#ifndef ULLONG_MAX +#define ULLONG_MAX (18446744073709551615) +#endif + +#define LF ((char)10) +#define CR ((char)13) +#define TAB ((char)9) +#define SPACE ((char)' ') +#define EOFC (255) + +#ifndef OK_EXIT_CODE +# ifdef CONTESTER +# define OK_EXIT_CODE 0xAC +# else +# define OK_EXIT_CODE 0 +# endif +#endif + +#ifndef WA_EXIT_CODE +# ifdef EJUDGE +# define WA_EXIT_CODE 5 +# elif defined(CONTESTER) +# define WA_EXIT_CODE 0xAB +# else +# define WA_EXIT_CODE 1 +# endif +#endif + +#ifndef PE_EXIT_CODE +# ifdef EJUDGE +# define PE_EXIT_CODE 4 +# elif defined(CONTESTER) +# define PE_EXIT_CODE 0xAA +# else +# define PE_EXIT_CODE 2 +# endif +#endif + +#ifndef FAIL_EXIT_CODE +# ifdef EJUDGE +# define FAIL_EXIT_CODE 6 +# elif defined(CONTESTER) +# define FAIL_EXIT_CODE 0xA3 +# else +# define FAIL_EXIT_CODE 3 +# endif +#endif + +#ifndef DIRT_EXIT_CODE +# ifdef EJUDGE +# define DIRT_EXIT_CODE 6 +# else +# define DIRT_EXIT_CODE 4 +# endif +#endif + +#ifndef POINTS_EXIT_CODE +# define POINTS_EXIT_CODE 7 +#endif + +#ifndef UNEXPECTED_EOF_EXIT_CODE +# define UNEXPECTED_EOF_EXIT_CODE 8 +#endif + +#ifndef PC_BASE_EXIT_CODE +# ifdef TESTSYS +# define PC_BASE_EXIT_CODE 50 +# else +# define PC_BASE_EXIT_CODE 0 +# endif +#endif + +#ifdef __GNUC__ +# define __TESTLIB_STATIC_ASSERT(condition) typedef void* __testlib_static_assert_type[(condition) ? 1 : -1] __attribute__((unused)) +#else +# define __TESTLIB_STATIC_ASSERT(condition) typedef void* __testlib_static_assert_type[(condition) ? 1 : -1] +#endif + +#ifdef ON_WINDOWS +#define I64 "%I64d" +#define U64 "%I64u" +#else +#define I64 "%lld" +#define U64 "%llu" +#endif + +#ifdef _MSC_VER +# define NORETURN __declspec(noreturn) +#elif defined __GNUC__ +# define NORETURN __attribute__ ((noreturn)) +#else +# define NORETURN +#endif + +static char __testlib_format_buffer[16777216]; +static int __testlib_format_buffer_usage_count = 0; + +#define FMT_TO_RESULT(fmt, cstr, result) std::string result; \ + if (__testlib_format_buffer_usage_count != 0) \ + __testlib_fail("FMT_TO_RESULT::__testlib_format_buffer_usage_count != 0"); \ + __testlib_format_buffer_usage_count++; \ + va_list ap; \ + va_start(ap, fmt); \ + vsnprintf(__testlib_format_buffer, sizeof(__testlib_format_buffer), cstr, ap); \ + va_end(ap); \ + __testlib_format_buffer[sizeof(__testlib_format_buffer) - 1] = 0; \ + result = std::string(__testlib_format_buffer); \ + __testlib_format_buffer_usage_count--; \ + +#ifdef __GNUC__ +__attribute__ ((format (printf, 1, 2))) +#endif +std::string testlib_format_(const char *fmt, ...); +std::string testlib_format_(const std::string fmt, ...); + +const long long __TESTLIB_LONGLONG_MAX = 9223372036854775807LL; +const int __TESTLIB_MAX_TEST_CASE = 1073741823; + +int __testlib_exitCode; + +bool __testlib_hasTestCase; +int __testlib_testCase = -1; + +void setTestCase(int testCase); + +void unsetTestCase() { + __testlib_hasTestCase = false; + __testlib_testCase = -1; +} + +NORETURN static void __testlib_fail(const std::string &message); + +template +#ifdef __GNUC__ +__attribute__((const)) +#endif +static inline T __testlib_abs(const T &x) { + return x > 0 ? x : -x; +} + +template +#ifdef __GNUC__ +__attribute__((const)) +#endif +static inline T __testlib_min(const T &a, const T &b) { + return a < b ? a : b; +} + +template +#ifdef __GNUC__ +__attribute__((const)) +#endif +static inline T __testlib_max(const T &a, const T &b) { + return a > b ? a : b; +} + +template +#ifdef __GNUC__ +__attribute__((const)) +#endif +static inline T __testlib_crop(T value, T a, T b) { + return __testlib_min(__testlib_max(value, a), --b); +} + +#ifdef __GNUC__ +__attribute__((const)) +#endif +static inline double __testlib_crop(double value, double a, double b) { + value = __testlib_min(__testlib_max(value, a), b); + if (value >= b) + value = std::nexttoward(b, a); + return value; +} + +static bool __testlib_prelimIsNaN(double r) { + volatile double ra = r; +#ifndef __BORLANDC__ + return ((ra != ra) == true) && ((ra == ra) == false) && ((1.0 > ra) == false) && ((1.0 < ra) == false); +#else + return std::_isnan(ra); +#endif +} + +static std::string removeDoubleTrailingZeroes(std::string value) { + while (!value.empty() && value[value.length() - 1] == '0' && value.find('.') != std::string::npos) + value = value.substr(0, value.length() - 1); + if (!value.empty() && value[value.length() - 1] == '.') + return value + '0'; + else + return value; +} + +inline std::string upperCase(std::string s) { + for (size_t i = 0; i < s.length(); i++) + if ('a' <= s[i] && s[i] <= 'z') + s[i] = char(s[i] - 'a' + 'A'); + return s; +} + +inline std::string lowerCase(std::string s) { + for (size_t i = 0; i < s.length(); i++) + if ('A' <= s[i] && s[i] <= 'Z') + s[i] = char(s[i] - 'A' + 'a'); + return s; +} + +static std::string __testlib_part(const std::string &s); + +static bool __testlib_isNaN(double r) { + __TESTLIB_STATIC_ASSERT(sizeof(double) == sizeof(long long)); + volatile double ra = r; + long long llr1, llr2; + std::memcpy((void *) &llr1, (void *) &ra, sizeof(double)); + ra = -ra; + std::memcpy((void *) &llr2, (void *) &ra, sizeof(double)); + long long llnan = 0xFFF8000000000000LL; + return __testlib_prelimIsNaN(r) || llnan == llr1 || llnan == llr2; +} + +static double __testlib_nan() { + __TESTLIB_STATIC_ASSERT(sizeof(double) == sizeof(long long)); +#ifndef NAN + long long llnan = 0xFFF8000000000000LL; + double nan; + std::memcpy(&nan, &llnan, sizeof(double)); + return nan; +#else + return NAN; +#endif +} + +static bool __testlib_isInfinite(double r) { + volatile double ra = r; + return (ra > 1E300 || ra < -1E300); +} + +#ifdef __GNUC__ +__attribute__((const)) +#endif +inline bool doubleCompare(double expected, double result, double MAX_DOUBLE_ERROR) { + MAX_DOUBLE_ERROR += 1E-15; + if (__testlib_isNaN(expected)) { + return __testlib_isNaN(result); + } else if (__testlib_isInfinite(expected)) { + if (expected > 0) { + return result > 0 && __testlib_isInfinite(result); + } else { + return result < 0 && __testlib_isInfinite(result); + } + } else if (__testlib_isNaN(result) || __testlib_isInfinite(result)) { + return false; + } else if (__testlib_abs(result - expected) <= MAX_DOUBLE_ERROR) { + return true; + } else { + double minv = __testlib_min(expected * (1.0 - MAX_DOUBLE_ERROR), + expected * (1.0 + MAX_DOUBLE_ERROR)); + double maxv = __testlib_max(expected * (1.0 - MAX_DOUBLE_ERROR), + expected * (1.0 + MAX_DOUBLE_ERROR)); + return result >= minv && result <= maxv; + } +} + +#ifdef __GNUC__ +__attribute__((const)) +#endif +inline double doubleDelta(double expected, double result) { + double absolute = __testlib_abs(result - expected); + + if (__testlib_abs(expected) > 1E-9) { + double relative = __testlib_abs(absolute / expected); + return __testlib_min(absolute, relative); + } else + return absolute; +} + +/** It does nothing on non-windows and files differ from stdin/stdout/stderr. */ +static void __testlib_set_binary(std::FILE *file) { + if (NULL != file) { +#ifdef ON_WINDOWS +# ifdef _O_BINARY + if (stdin == file) +# ifdef STDIN_FILENO + return void(_setmode(STDIN_FILENO, _O_BINARY)); +# else + return void(_setmode(_fileno(stdin), _O_BINARY)); +# endif + if (stdout == file) +# ifdef STDOUT_FILENO + return void(_setmode(STDOUT_FILENO, _O_BINARY)); +# else + return void(_setmode(_fileno(stdout), _O_BINARY)); +# endif + if (stderr == file) +# ifdef STDERR_FILENO + return void(_setmode(STDERR_FILENO, _O_BINARY)); +# else + return void(_setmode(_fileno(stderr), _O_BINARY)); +# endif +# elif O_BINARY + if (stdin == file) +# ifdef STDIN_FILENO + return void(setmode(STDIN_FILENO, O_BINARY)); +# else + return void(setmode(fileno(stdin), O_BINARY)); +# endif + if (stdout == file) +# ifdef STDOUT_FILENO + return void(setmode(STDOUT_FILENO, O_BINARY)); +# else + return void(setmode(fileno(stdout), O_BINARY)); +# endif + if (stderr == file) +# ifdef STDERR_FILENO + return void(setmode(STDERR_FILENO, O_BINARY)); +# else + return void(setmode(fileno(stderr), O_BINARY)); +# endif +# endif +#endif + } +} + +#if __cplusplus > 199711L || defined(_MSC_VER) +template +static std::string vtos(const T &t, std::true_type) { + if (t == 0) + return "0"; + else { + T n(t); + bool negative = n < 0; + std::string s; + while (n != 0) { + T digit = n % 10; + if (digit < 0) + digit = -digit; + s += char('0' + digit); + n /= 10; + } + std::reverse(s.begin(), s.end()); + return negative ? "-" + s : s; + } +} + +template +static std::string vtos(const T &t, std::false_type) { + std::string s; + static std::stringstream ss; + ss.str(std::string()); + ss.clear(); + ss << t; + ss >> s; + return s; +} + +template +static std::string vtos(const T &t) { + return vtos(t, std::is_integral()); +} + +/* signed case. */ +template +static std::string toHumanReadableString(const T &n, std::false_type) { + if (n == 0) + return vtos(n); + int trailingZeroCount = 0; + T n_ = n; + while (n_ % 10 == 0) + n_ /= 10, trailingZeroCount++; + if (trailingZeroCount >= 7) { + if (n_ == 1) + return "10^" + vtos(trailingZeroCount); + else if (n_ == -1) + return "-10^" + vtos(trailingZeroCount); + else + return vtos(n_) + "*10^" + vtos(trailingZeroCount); + } else + return vtos(n); +} + +/* unsigned case. */ +template +static std::string toHumanReadableString(const T &n, std::true_type) { + if (n == 0) + return vtos(n); + int trailingZeroCount = 0; + T n_ = n; + while (n_ % 10 == 0) + n_ /= 10, trailingZeroCount++; + if (trailingZeroCount >= 7) { + if (n_ == 1) + return "10^" + vtos(trailingZeroCount); + else + return vtos(n_) + "*10^" + vtos(trailingZeroCount); + } else + return vtos(n); +} + +template +static std::string toHumanReadableString(const T &n) { + return toHumanReadableString(n, std::is_unsigned()); +} +#else +template +static std::string vtos(const T& t) +{ + std::string s; + static std::stringstream ss; + ss.str(std::string()); + ss.clear(); + ss << t; + ss >> s; + return s; +} + +template +static std::string toHumanReadableString(const T &n) { + return vtos(n); +} +#endif + +template +static std::string toString(const T &t) { + return vtos(t); +} + +#if __cplusplus > 199711L || defined(_MSC_VER) +/* opts */ +void prepareOpts(int argc, char* argv[]); +#endif + +FILE* testlib_fopen_(const char* path, const char* mode) { +#ifdef _MSC_VER + FILE* result = NULL; + if (fopen_s(&result, path, mode) != 0) + return NULL; + else + return result; +#else + return std::fopen(path, mode); +#endif +} + +FILE* testlib_freopen_(const char* path, const char* mode, FILE* file) { +#ifdef _MSC_VER + FILE* result = NULL; + if (freopen_s(&result, path, mode, file) != 0) + return NULL; + else + return result; +#else + return std::freopen(path, mode, file); +#endif +} + +/* + * Very simple regex-like pattern. + * It used for two purposes: validation and generation. + * + * For example, pattern("[a-z]{1,5}").next(rnd) will return + * random string from lowercase latin letters with length + * from 1 to 5. It is easier to call rnd.next("[a-z]{1,5}") + * for the same effect. + * + * Another samples: + * "mike|john" will generate (match) "mike" or "john"; + * "-?[1-9][0-9]{0,3}" will generate (match) non-zero integers from -9999 to 9999; + * "id-([ac]|b{2})" will generate (match) "id-a", "id-bb", "id-c"; + * "[^0-9]*" will match sequences (empty or non-empty) without digits, you can't + * use it for generations. + * + * You can't use pattern for generation if it contains meta-symbol '*'. Also it + * is not recommended to use it for char-sets with meta-symbol '^' like [^a-z]. + * + * For matching very simple greedy algorithm is used. For example, pattern + * "[0-9]?1" will not match "1", because of greedy nature of matching. + * Alternations (meta-symbols "|") are processed with brute-force algorithm, so + * do not use many alternations in one expression. + * + * If you want to use one expression many times it is better to compile it into + * a single pattern like "pattern p("[a-z]+")". Later you can use + * "p.matches(std::string s)" or "p.next(random_t& rd)" to check matching or generate + * new string by pattern. + * + * Simpler way to read token and check it for pattern matching is "inf.readToken("[a-z]+")". + * + * All spaces are ignored in regex, unless escaped with \. For example, ouf.readLine("NO SOLUTION") + * will expect "NOSOLUTION", the correct call should be ouf.readLine("NO\\ SOLUTION") or + * ouf.readLine(R"(NO\ SOLUTION)") if you prefer raw string literals from C++11. + */ +class random_t; + +class pattern { +public: + /* Create pattern instance by string. */ + pattern(std::string s); + + /* Generate new string by pattern and given random_t. */ + std::string next(random_t &rnd) const; + + /* Checks if given string match the pattern. */ + bool matches(const std::string &s) const; + + /* Returns source string of the pattern. */ + std::string src() const; + +private: + bool matches(const std::string &s, size_t pos) const; + + std::string s; + std::vector children; + std::vector chars; + int from; + int to; +}; + +/* + * Use random_t instances to generate random values. It is preferred + * way to use randoms instead of rand() function or self-written + * randoms. + * + * Testlib defines global variable "rnd" of random_t class. + * Use registerGen(argc, argv, 1) to setup random_t seed be command + * line (to use latest random generator version). + * + * Random generates uniformly distributed values if another strategy is + * not specified explicitly. + */ +class random_t { +private: + unsigned long long seed; + static const unsigned long long multiplier; + static const unsigned long long addend; + static const unsigned long long mask; + static const int lim; + + long long nextBits(int bits) { + if (bits <= 48) { + seed = (seed * multiplier + addend) & mask; + return (long long) (seed >> (48 - bits)); + } else { + if (bits > 63) + __testlib_fail("random_t::nextBits(int bits): n must be less than 64"); + + int lowerBitCount = (random_t::version == 0 ? 31 : 32); + + long long left = (nextBits(31) << 32); + long long right = nextBits(lowerBitCount); + + return left ^ right; + } + } + +public: + static int version; + + /* New random_t with fixed seed. */ + random_t() + : seed(3905348978240129619LL) { + } + + /* Sets seed by command line. */ + void setSeed(int argc, char *argv[]) { + random_t p; + + seed = 3905348978240129619LL; + for (int i = 1; i < argc; i++) { + std::size_t le = std::strlen(argv[i]); + for (std::size_t j = 0; j < le; j++) + seed = seed * multiplier + (unsigned int) (argv[i][j]) + addend; + seed += multiplier / addend; + } + + seed = seed & mask; + } + + /* Sets seed by given value. */ + void setSeed(long long _seed) { + seed = (unsigned long long) _seed; + seed = (seed ^ multiplier) & mask; + } + +#ifndef __BORLANDC__ + + /* Random string value by given pattern (see pattern documentation). */ + std::string next(const std::string &ptrn) { + pattern p(ptrn); + return p.next(*this); + } + +#else + /* Random string value by given pattern (see pattern documentation). */ + std::string next(std::string ptrn) + { + pattern p(ptrn); + return p.next(*this); + } +#endif + + /* Random value in range [0, n-1]. */ + int next(int n) { + if (n <= 0) + __testlib_fail("random_t::next(int n): n must be positive"); + + if ((n & -n) == n) // n is a power of 2 + return (int) ((n * (long long) nextBits(31)) >> 31); + + const long long limit = INT_MAX / n * n; + + long long bits; + do { + bits = nextBits(31); + } while (bits >= limit); + + return int(bits % n); + } + + /* Random value in range [0, n-1]. */ + unsigned int next(unsigned int n) { + if (n >= INT_MAX) + __testlib_fail("random_t::next(unsigned int n): n must be less INT_MAX"); + return (unsigned int) next(int(n)); + } + + /* Random value in range [0, n-1]. */ + long long next(long long n) { + if (n <= 0) + __testlib_fail("random_t::next(long long n): n must be positive"); + + const long long limit = __TESTLIB_LONGLONG_MAX / n * n; + + long long bits; + do { + bits = nextBits(63); + } while (bits >= limit); + + return bits % n; + } + + /* Random value in range [0, n-1]. */ + unsigned long long next(unsigned long long n) { + if (n >= (unsigned long long) (__TESTLIB_LONGLONG_MAX)) + __testlib_fail("random_t::next(unsigned long long n): n must be less LONGLONG_MAX"); + return (unsigned long long) next((long long) (n)); + } + + /* Random value in range [0, n-1]. */ + long next(long n) { + return (long) next((long long) (n)); + } + + /* Random value in range [0, n-1]. */ + unsigned long next(unsigned long n) { + if (n >= (unsigned long) (LONG_MAX)) + __testlib_fail("random_t::next(unsigned long n): n must be less LONG_MAX"); + return (unsigned long) next((unsigned long long) (n)); + } + + /* Returns random value in range [from,to]. */ + int next(int from, int to) { + return int(next((long long) to - from + 1) + from); + } + + /* Returns random value in range [from,to]. */ + unsigned int next(unsigned int from, unsigned int to) { + return (unsigned int) (next((long long) to - from + 1) + from); + } + + /* Returns random value in range [from,to]. */ + long long next(long long from, long long to) { + return next(to - from + 1) + from; + } + + /* Returns random value in range [from,to]. */ + unsigned long long next(unsigned long long from, unsigned long long to) { + if (from > to) + __testlib_fail("random_t::next(unsigned long long from, unsigned long long to): from can't not exceed to"); + return next(to - from + 1) + from; + } + + /* Returns random value in range [from,to]. */ + long next(long from, long to) { + return next(to - from + 1) + from; + } + + /* Returns random value in range [from,to]. */ + unsigned long next(unsigned long from, unsigned long to) { + if (from > to) + __testlib_fail("random_t::next(unsigned long from, unsigned long to): from can't not exceed to"); + return next(to - from + 1) + from; + } + + /* Random double value in range [0, 1). */ + double next() { + long long left = ((long long) (nextBits(26)) << 27); + long long right = nextBits(27); + return __testlib_crop((double) (left + right) / (double) (1LL << 53), 0.0, 1.0); + } + + /* Random double value in range [0, n). */ + double next(double n) { + if (n <= 0.0) + __testlib_fail("random_t::next(double): n should be positive"); + return __testlib_crop(n * next(), 0.0, n); + } + + /* Random double value in range [from, to). */ + double next(double from, double to) { + if (from >= to) + __testlib_fail("random_t::next(double from, double to): from should be strictly less than to"); + return next(to - from) + from; + } + + /* Returns random element from container. */ + template + typename Container::value_type any(const Container &c) { + int size = int(c.size()); + if (size <= 0) + __testlib_fail("random_t::any(const Container& c): c.size() must be positive"); + typename Container::const_iterator it = c.begin(); + std::advance(it, next(size)); + return *it; + } + + /* Returns random element from iterator range. */ + template + typename Iter::value_type any(const Iter &begin, const Iter &end) { + int size = static_cast(std::distance(begin, end)); + if (size <= 0) + __testlib_fail("random_t::any(const Iter& begin, const Iter& end): range must have positive length"); + Iter it = begin; + std::advance(it, next(size)); + return *it; + } + + /* Random string value by given pattern (see pattern documentation). */ +#ifdef __GNUC__ + __attribute__ ((format (printf, 2, 3))) +#endif + std::string next(const char *format, ...) { + FMT_TO_RESULT(format, format, ptrn); + return next(ptrn); + } + + /* + * Weighted next. If type == 0 than it is usual "next()". + * + * If type = 1, than it returns "max(next(), next())" + * (the number of "max" functions equals to "type"). + * + * If type < 0, than "max" function replaces with "min". + */ + int wnext(int n, int type) { + if (n <= 0) + __testlib_fail("random_t::wnext(int n, int type): n must be positive"); + + if (abs(type) < random_t::lim) { + int result = next(n); + + for (int i = 0; i < +type; i++) + result = __testlib_max(result, next(n)); + + for (int i = 0; i < -type; i++) + result = __testlib_min(result, next(n)); + + return result; + } else { + double p; + + if (type > 0) + p = std::pow(next() + 0.0, 1.0 / (type + 1)); + else + p = 1 - std::pow(next() + 0.0, 1.0 / (-type + 1)); + + return __testlib_crop((int) (double(n) * p), 0, n); + } + } + + /* See wnext(int, int). It uses the same algorithms. */ + long long wnext(long long n, int type) { + if (n <= 0) + __testlib_fail("random_t::wnext(long long n, int type): n must be positive"); + + if (abs(type) < random_t::lim) { + long long result = next(n); + + for (int i = 0; i < +type; i++) + result = __testlib_max(result, next(n)); + + for (int i = 0; i < -type; i++) + result = __testlib_min(result, next(n)); + + return result; + } else { + double p; + + if (type > 0) + p = std::pow(next() + 0.0, 1.0 / (type + 1)); + else + p = 1 - std::pow(next() + 0.0, 1.0 / (-type + 1)); + + return __testlib_crop((long long) (double(n) * p), 0LL, n); + } + } + + /* Returns value in [0, n). See wnext(int, int). It uses the same algorithms. */ + double wnext(double n, int type) { + if (n <= 0) + __testlib_fail("random_t::wnext(double n, int type): n must be positive"); + + if (abs(type) < random_t::lim) { + double result = next(); + + for (int i = 0; i < +type; i++) + result = __testlib_max(result, next()); + + for (int i = 0; i < -type; i++) + result = __testlib_min(result, next()); + + return n * result; + } else { + double p; + + if (type > 0) + p = std::pow(next() + 0.0, 1.0 / (type + 1)); + else + p = 1 - std::pow(next() + 0.0, 1.0 / (-type + 1)); + + return __testlib_crop(n * p, 0.0, n); + } + } + + /* Returns value in [0, 1). See wnext(int, int). It uses the same algorithms. */ + double wnext(int type) { + return wnext(1.0, type); + } + + /* See wnext(int, int). It uses the same algorithms. */ + unsigned int wnext(unsigned int n, int type) { + if (n >= INT_MAX) + __testlib_fail("random_t::wnext(unsigned int n, int type): n must be less INT_MAX"); + return (unsigned int) wnext(int(n), type); + } + + /* See wnext(int, int). It uses the same algorithms. */ + unsigned long long wnext(unsigned long long n, int type) { + if (n >= (unsigned long long) (__TESTLIB_LONGLONG_MAX)) + __testlib_fail("random_t::wnext(unsigned long long n, int type): n must be less LONGLONG_MAX"); + + return (unsigned long long) wnext((long long) (n), type); + } + + /* See wnext(int, int). It uses the same algorithms. */ + long wnext(long n, int type) { + return (long) wnext((long long) (n), type); + } + + /* See wnext(int, int). It uses the same algorithms. */ + unsigned long wnext(unsigned long n, int type) { + if (n >= (unsigned long) (LONG_MAX)) + __testlib_fail("random_t::wnext(unsigned long n, int type): n must be less LONG_MAX"); + + return (unsigned long) wnext((unsigned long long) (n), type); + } + + /* Returns weighted random value in range [from, to]. */ + int wnext(int from, int to, int type) { + if (from > to) + __testlib_fail("random_t::wnext(int from, int to, int type): from can't not exceed to"); + return wnext(to - from + 1, type) + from; + } + + /* Returns weighted random value in range [from, to]. */ + int wnext(unsigned int from, unsigned int to, int type) { + if (from > to) + __testlib_fail("random_t::wnext(unsigned int from, unsigned int to, int type): from can't not exceed to"); + return int(wnext(to - from + 1, type) + from); + } + + /* Returns weighted random value in range [from, to]. */ + long long wnext(long long from, long long to, int type) { + if (from > to) + __testlib_fail("random_t::wnext(long long from, long long to, int type): from can't not exceed to"); + return wnext(to - from + 1, type) + from; + } + + /* Returns weighted random value in range [from, to]. */ + unsigned long long wnext(unsigned long long from, unsigned long long to, int type) { + if (from > to) + __testlib_fail( + "random_t::wnext(unsigned long long from, unsigned long long to, int type): from can't not exceed to"); + return wnext(to - from + 1, type) + from; + } + + /* Returns weighted random value in range [from, to]. */ + long wnext(long from, long to, int type) { + if (from > to) + __testlib_fail("random_t::wnext(long from, long to, int type): from can't not exceed to"); + return wnext(to - from + 1, type) + from; + } + + /* Returns weighted random value in range [from, to]. */ + unsigned long wnext(unsigned long from, unsigned long to, int type) { + if (from > to) + __testlib_fail("random_t::wnext(unsigned long from, unsigned long to, int type): from can't not exceed to"); + return wnext(to - from + 1, type) + from; + } + + /* Returns weighted random double value in range [from, to). */ + double wnext(double from, double to, int type) { + if (from >= to) + __testlib_fail("random_t::wnext(double from, double to, int type): from should be strictly less than to"); + return wnext(to - from, type) + from; + } + + /* Returns weighted random element from container. */ + template + typename Container::value_type wany(const Container &c, int type) { + int size = int(c.size()); + if (size <= 0) + __testlib_fail("random_t::wany(const Container& c, int type): c.size() must be positive"); + typename Container::const_iterator it = c.begin(); + std::advance(it, wnext(size, type)); + return *it; + } + + /* Returns weighted random element from iterator range. */ + template + typename Iter::value_type wany(const Iter &begin, const Iter &end, int type) { + int size = static_cast(std::distance(begin, end)); + if (size <= 0) + __testlib_fail( + "random_t::any(const Iter& begin, const Iter& end, int type): range must have positive length"); + Iter it = begin; + std::advance(it, wnext(size, type)); + return *it; + } + + /* Returns random permutation of the given size (values are between `first` and `first`+size-1)*/ + template + std::vector perm(T size, E first) { + if (size < 0) + __testlib_fail("random_t::perm(T size, E first = 0): size must non-negative"); + else if (size == 0) + return std::vector(); + std::vector p(size); + E current = first; + for (T i = 0; i < size; i++) + p[i] = current++; + if (size > 1) + for (T i = 1; i < size; i++) + std::swap(p[i], p[next(i + 1)]); + return p; + } + + /* Returns random permutation of the given size (values are between 0 and size-1)*/ + template + std::vector perm(T size) { + return perm(size, T(0)); + } + + /* Returns `size` unordered (unsorted) distinct numbers between `from` and `to`. */ + template + std::vector distinct(int size, T from, T to) { + std::vector result; + if (size == 0) + return result; + + if (from > to) + __testlib_fail("random_t::distinct expected from <= to"); + + if (size < 0) + __testlib_fail("random_t::distinct expected size >= 0"); + + uint64_t n = to - from + 1; + if (uint64_t(size) > n) + __testlib_fail("random_t::distinct expected size <= to - from + 1"); + + double expected = 0.0; + for (int i = 1; i <= size; i++) + expected += double(n) / double(n - i + 1); + + if (expected < double(n)) { + std::set vals; + while (int(vals.size()) < size) { + T x = T(next(from, to)); + if (vals.insert(x).second) + result.push_back(x); + } + } else { + if (n > 1000000000) + __testlib_fail("random_t::distinct here expected to - from + 1 <= 1000000000"); + std::vector p(perm(int(n), from)); + result.insert(result.end(), p.begin(), p.begin() + size); + } + + return result; + } + + /* Returns `size` unordered (unsorted) distinct numbers between `0` and `upper`-1. */ + template + std::vector distinct(int size, T upper) { + if (size < 0) + __testlib_fail("random_t::distinct expected size >= 0"); + if (size == 0) + return std::vector(); + + if (upper <= 0) + __testlib_fail("random_t::distinct expected upper > 0"); + if (size > upper) + __testlib_fail("random_t::distinct expected size <= upper"); + + return distinct(size, T(0), upper - 1); + } + + /* Returns random (unsorted) partition which is a representation of sum as a sum of integers not less than min_part. */ + template + std::vector partition(int size, T sum, T min_part) { + if (size < 0) + __testlib_fail("random_t::partition: size < 0"); + if (size == 0 && sum != 0) + __testlib_fail("random_t::partition: size == 0 && sum != 0"); + if (min_part * size > sum) + __testlib_fail("random_t::partition: min_part * size > sum"); + if (size == 0 && sum == 0) + return std::vector(); + + T sum_ = sum; + sum -= min_part * size; + + std::vector septums(size); + std::vector d = distinct(size - 1, T(1), T(sum + size - 1)); + for (int i = 0; i + 1 < size; i++) + septums[i + 1] = d[i]; + sort(septums.begin(), septums.end()); + + std::vector result(size); + for (int i = 0; i + 1 < size; i++) + result[i] = septums[i + 1] - septums[i] - 1; + result[size - 1] = sum + size - 1 - septums.back(); + + for (std::size_t i = 0; i < result.size(); i++) + result[i] += min_part; + + T result_sum = 0; + for (std::size_t i = 0; i < result.size(); i++) + result_sum += result[i]; + if (result_sum != sum_) + __testlib_fail("random_t::partition: partition sum is expected to be the given sum"); + + if (*std::min_element(result.begin(), result.end()) < min_part) + __testlib_fail("random_t::partition: partition min is expected to be no less than the given min_part"); + + if (int(result.size()) != size || result.size() != (size_t) size) + __testlib_fail("random_t::partition: partition size is expected to be equal to the given size"); + + return result; + } + + /* Returns random (unsorted) partition which is a representation of sum as a sum of positive integers. */ + template + std::vector partition(int size, T sum) { + return partition(size, sum, T(1)); + } +}; + +const int random_t::lim = 25; +const unsigned long long random_t::multiplier = 0x5DEECE66DLL; +const unsigned long long random_t::addend = 0xBLL; +const unsigned long long random_t::mask = (1LL << 48) - 1; +int random_t::version = -1; + +/* Pattern implementation */ +bool pattern::matches(const std::string &s) const { + return matches(s, 0); +} + +static bool __pattern_isSlash(const std::string &s, size_t pos) { + return s[pos] == '\\'; +} + +#ifdef __GNUC__ +__attribute__((pure)) +#endif +static bool __pattern_isCommandChar(const std::string &s, size_t pos, char value) { + if (pos >= s.length()) + return false; + + int slashes = 0; + + int before = int(pos) - 1; + while (before >= 0 && s[before] == '\\') + before--, slashes++; + + return slashes % 2 == 0 && s[pos] == value; +} + +static char __pattern_getChar(const std::string &s, size_t &pos) { + if (__pattern_isSlash(s, pos)) + pos += 2; + else + pos++; + + return s[pos - 1]; +} + +#ifdef __GNUC__ +__attribute__((pure)) +#endif +static int __pattern_greedyMatch(const std::string &s, size_t pos, const std::vector chars) { + int result = 0; + + while (pos < s.length()) { + char c = s[pos++]; + if (!std::binary_search(chars.begin(), chars.end(), c)) + break; + else + result++; + } + + return result; +} + +std::string pattern::src() const { + return s; +} + +bool pattern::matches(const std::string &s, size_t pos) const { + std::string result; + + if (to > 0) { + int size = __pattern_greedyMatch(s, pos, chars); + if (size < from) + return false; + if (size > to) + size = to; + pos += size; + } + + if (children.size() > 0) { + for (size_t child = 0; child < children.size(); child++) + if (children[child].matches(s, pos)) + return true; + return false; + } else + return pos == s.length(); +} + +std::string pattern::next(random_t &rnd) const { + std::string result; + result.reserve(20); + + if (to == INT_MAX) + __testlib_fail("pattern::next(random_t& rnd): can't process character '*' for generation"); + + if (to > 0) { + int count = rnd.next(to - from + 1) + from; + for (int i = 0; i < count; i++) + result += chars[rnd.next(int(chars.size()))]; + } + + if (children.size() > 0) { + int child = rnd.next(int(children.size())); + result += children[child].next(rnd); + } + + return result; +} + +static void __pattern_scanCounts(const std::string &s, size_t &pos, int &from, int &to) { + if (pos >= s.length()) { + from = to = 1; + return; + } + + if (__pattern_isCommandChar(s, pos, '{')) { + std::vector parts; + std::string part; + + pos++; + + while (pos < s.length() && !__pattern_isCommandChar(s, pos, '}')) { + if (__pattern_isCommandChar(s, pos, ',')) + parts.push_back(part), part = "", pos++; + else + part += __pattern_getChar(s, pos); + } + + if (part != "") + parts.push_back(part); + + if (!__pattern_isCommandChar(s, pos, '}')) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + pos++; + + if (parts.size() < 1 || parts.size() > 2) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + std::vector numbers; + + for (size_t i = 0; i < parts.size(); i++) { + if (parts[i].length() == 0) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + int number; +#ifdef _MSC_VER + if (sscanf_s(parts[i].c_str(), "%d", &number) != 1) +#else + if (std::sscanf(parts[i].c_str(), "%d", &number) != 1) +#endif + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + numbers.push_back(number); + } + + if (numbers.size() == 1) + from = to = numbers[0]; + else + from = numbers[0], to = numbers[1]; + + if (from > to) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + } else { + if (__pattern_isCommandChar(s, pos, '?')) { + from = 0, to = 1, pos++; + return; + } + + if (__pattern_isCommandChar(s, pos, '*')) { + from = 0, to = INT_MAX, pos++; + return; + } + + if (__pattern_isCommandChar(s, pos, '+')) { + from = 1, to = INT_MAX, pos++; + return; + } + + from = to = 1; + } +} + +static std::vector __pattern_scanCharSet(const std::string &s, size_t &pos) { + if (pos >= s.length()) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + std::vector result; + + if (__pattern_isCommandChar(s, pos, '[')) { + pos++; + bool negative = __pattern_isCommandChar(s, pos, '^'); + if (negative) + pos++; + + char prev = 0; + + while (pos < s.length() && !__pattern_isCommandChar(s, pos, ']')) { + if (__pattern_isCommandChar(s, pos, '-') && prev != 0) { + pos++; + + if (pos + 1 == s.length() || __pattern_isCommandChar(s, pos, ']')) { + result.push_back(prev); + prev = '-'; + continue; + } + + char next = __pattern_getChar(s, pos); + if (prev > next) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + for (char c = prev; c != next; c++) + result.push_back(c); + result.push_back(next); + + prev = 0; + } else { + if (prev != 0) + result.push_back(prev); + prev = __pattern_getChar(s, pos); + } + } + + if (prev != 0) + result.push_back(prev); + + if (!__pattern_isCommandChar(s, pos, ']')) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + pos++; + + if (negative) { + std::sort(result.begin(), result.end()); + std::vector actuals; + for (int code = 0; code < 255; code++) { + char c = char(code); + if (!std::binary_search(result.begin(), result.end(), c)) + actuals.push_back(c); + } + result = actuals; + } + + std::sort(result.begin(), result.end()); + } else + result.push_back(__pattern_getChar(s, pos)); + + return result; +} + +pattern::pattern(std::string s) : s(s), from(0), to(0) { + std::string t; + for (size_t i = 0; i < s.length(); i++) + if (!__pattern_isCommandChar(s, i, ' ')) + t += s[i]; + s = t; + + int opened = 0; + int firstClose = -1; + std::vector seps; + + for (size_t i = 0; i < s.length(); i++) { + if (__pattern_isCommandChar(s, i, '(')) { + opened++; + continue; + } + + if (__pattern_isCommandChar(s, i, ')')) { + opened--; + if (opened == 0 && firstClose == -1) + firstClose = int(i); + continue; + } + + if (opened < 0) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + if (__pattern_isCommandChar(s, i, '|') && opened == 0) + seps.push_back(int(i)); + } + + if (opened != 0) + __testlib_fail("pattern: Illegal pattern (or part) \"" + s + "\""); + + if (seps.size() == 0 && firstClose + 1 == (int) s.length() + && __pattern_isCommandChar(s, 0, '(') && __pattern_isCommandChar(s, s.length() - 1, ')')) { + children.push_back(pattern(s.substr(1, s.length() - 2))); + } else { + if (seps.size() > 0) { + seps.push_back(int(s.length())); + int last = 0; + + for (size_t i = 0; i < seps.size(); i++) { + children.push_back(pattern(s.substr(last, seps[i] - last))); + last = seps[i] + 1; + } + } else { + size_t pos = 0; + chars = __pattern_scanCharSet(s, pos); + __pattern_scanCounts(s, pos, from, to); + if (pos < s.length()) + children.push_back(pattern(s.substr(pos))); + } + } +} + +/* End of pattern implementation */ + +template +inline bool isEof(C c) { + return c == EOFC; +} + +template +inline bool isEoln(C c) { + return (c == LF || c == CR); +} + +template +inline bool isBlanks(C c) { + return (c == LF || c == CR || c == SPACE || c == TAB); +} + +inline std::string trim(const std::string &s) { + if (s.empty()) + return s; + + int left = 0; + while (left < int(s.length()) && isBlanks(s[left])) + left++; + if (left >= int(s.length())) + return ""; + + int right = int(s.length()) - 1; + while (right >= 0 && isBlanks(s[right])) + right--; + if (right < 0) + return ""; + + return s.substr(left, right - left + 1); +} + +enum TMode { + _input, _output, _answer +}; + +/* Outcomes 6-15 are reserved for future use. */ +enum TResult { + _ok = 0, + _wa = 1, + _pe = 2, + _fail = 3, + _dirt = 4, + _points = 5, + _unexpected_eof = 8, + _partially = 16 +}; + +enum TTestlibMode { + _unknown, _checker, _validator, _generator, _interactor, _scorer +}; + +#define _pc(exitCode) (TResult(_partially + (exitCode))) + +/* Outcomes 6-15 are reserved for future use. */ +const std::string outcomes[] = { + "accepted", + "wrong-answer", + "presentation-error", + "fail", + "fail", +#ifndef PCMS2 + "points", +#else + "relative-scoring", +#endif + "reserved", + "reserved", + "unexpected-eof", + "reserved", + "reserved", + "reserved", + "reserved", + "reserved", + "reserved", + "reserved", + "partially-correct" +}; + +class InputStreamReader { +public: + virtual void setTestCase(int testCase) = 0; + + virtual std::vector getReadChars() = 0; + + virtual int curChar() = 0; + + virtual int nextChar() = 0; + + virtual void skipChar() = 0; + + virtual void unreadChar(int c) = 0; + + virtual std::string getName() = 0; + + virtual bool eof() = 0; + + virtual void close() = 0; + + virtual int getLine() = 0; + + virtual ~InputStreamReader() = 0; +}; + +InputStreamReader::~InputStreamReader() { + // No operations. +} + +class StringInputStreamReader : public InputStreamReader { +private: + std::string s; + size_t pos; + +public: + StringInputStreamReader(const std::string &content) : s(content), pos(0) { + // No operations. + } + + void setTestCase(int) { + __testlib_fail("setTestCase not implemented in StringInputStreamReader"); + } + + std::vector getReadChars() { + __testlib_fail("getReadChars not implemented in StringInputStreamReader"); + } + + int curChar() { + if (pos >= s.length()) + return EOFC; + else + return s[pos]; + } + + int nextChar() { + if (pos >= s.length()) { + pos++; + return EOFC; + } else + return s[pos++]; + } + + void skipChar() { + pos++; + } + + void unreadChar(int c) { + if (pos == 0) + __testlib_fail("StringInputStreamReader::unreadChar(int): pos == 0."); + pos--; + if (pos < s.length()) + s[pos] = char(c); + } + + std::string getName() { + return __testlib_part(s); + } + + int getLine() { + return -1; + } + + bool eof() { + return pos >= s.length(); + } + + void close() { + // No operations. + } +}; + +class FileInputStreamReader : public InputStreamReader { +private: + std::FILE *file; + std::string name; + int line; + std::vector undoChars; + std::vector readChars; + std::vector undoReadChars; + + inline int postprocessGetc(int getcResult) { + if (getcResult != EOF) + return getcResult; + else + return EOFC; + } + + int getc(FILE *file) { + int c; + int rc; + + if (undoChars.empty()) { + c = rc = ::getc(file); + } else { + c = undoChars.back(); + undoChars.pop_back(); + rc = undoReadChars.back(); + undoReadChars.pop_back(); + } + + if (c == LF) + line++; + + readChars.push_back(rc); + return c; + } + + int ungetc(int c/*, FILE* file*/) { + if (!readChars.empty()) { + undoReadChars.push_back(readChars.back()); + readChars.pop_back(); + } + if (c == LF) + line--; + undoChars.push_back(c); + return c; + } + +public: + FileInputStreamReader(std::FILE *file, const std::string &name) : file(file), name(name), line(1) { + // No operations. + } + + void setTestCase(int testCase) { + if (testCase < 0 || testCase > __TESTLIB_MAX_TEST_CASE) + __testlib_fail(testlib_format_("testCase expected fit in [1,%d], but %d doesn't", __TESTLIB_MAX_TEST_CASE, testCase)); + readChars.push_back(testCase + 256); + } + + std::vector getReadChars() { + return readChars; + } + + int curChar() { + if (feof(file)) + return EOFC; + else { + int c = getc(file); + ungetc(c/*, file*/); + return postprocessGetc(c); + } + } + + int nextChar() { + if (feof(file)) + return EOFC; + else + return postprocessGetc(getc(file)); + } + + void skipChar() { + getc(file); + } + + void unreadChar(int c) { + ungetc(c/*, file*/); + } + + std::string getName() { + return name; + } + + int getLine() { + return line; + } + + bool eof() { + if (NULL == file || feof(file)) + return true; + else { + int c = nextChar(); + if (c == EOFC || (c == EOF && feof(file))) + return true; + unreadChar(c); + return false; + } + } + + void close() { + if (NULL != file) { + fclose(file); + file = NULL; + } + } +}; + +class BufferedFileInputStreamReader : public InputStreamReader { +private: + static const size_t BUFFER_SIZE; + static const size_t MAX_UNREAD_COUNT; + + std::FILE *file; + std::string name; + int line; + + char *buffer; + bool *isEof; + int bufferPos; + size_t bufferSize; + + bool refill() { + if (NULL == file) + __testlib_fail("BufferedFileInputStreamReader: file == NULL (" + getName() + ")"); + + if (bufferPos >= int(bufferSize)) { + size_t readSize = fread( + buffer + MAX_UNREAD_COUNT, + 1, + BUFFER_SIZE - MAX_UNREAD_COUNT, + file + ); + + if (readSize < BUFFER_SIZE - MAX_UNREAD_COUNT + && ferror(file)) + __testlib_fail("BufferedFileInputStreamReader: unable to read (" + getName() + ")"); + + bufferSize = MAX_UNREAD_COUNT + readSize; + bufferPos = int(MAX_UNREAD_COUNT); + std::memset(isEof + MAX_UNREAD_COUNT, 0, sizeof(isEof[0]) * readSize); + + return readSize > 0; + } else + return true; + } + + char increment() { + char c; + if ((c = buffer[bufferPos++]) == LF) + line++; + return c; + } + +public: + BufferedFileInputStreamReader(std::FILE *file, const std::string &name) : file(file), name(name), line(1) { + buffer = new char[BUFFER_SIZE]; + isEof = new bool[BUFFER_SIZE]; + bufferSize = MAX_UNREAD_COUNT; + bufferPos = int(MAX_UNREAD_COUNT); + } + + ~BufferedFileInputStreamReader() { + if (NULL != buffer) { + delete[] buffer; + buffer = NULL; + } + if (NULL != isEof) { + delete[] isEof; + isEof = NULL; + } + } + + void setTestCase(int) { + __testlib_fail("setTestCase not implemented in BufferedFileInputStreamReader"); + } + + std::vector getReadChars() { + __testlib_fail("getReadChars not implemented in BufferedFileInputStreamReader"); + } + + int curChar() { + if (!refill()) + return EOFC; + + return isEof[bufferPos] ? EOFC : buffer[bufferPos]; + } + + int nextChar() { + if (!refill()) + return EOFC; + + return isEof[bufferPos] ? EOFC : increment(); + } + + void skipChar() { + increment(); + } + + void unreadChar(int c) { + bufferPos--; + if (bufferPos < 0) + __testlib_fail("BufferedFileInputStreamReader::unreadChar(int): bufferPos < 0"); + isEof[bufferPos] = (c == EOFC); + buffer[bufferPos] = char(c); + if (c == LF) + line--; + } + + std::string getName() { + return name; + } + + int getLine() { + return line; + } + + bool eof() { + return !refill() || EOFC == curChar(); + } + + void close() { + if (NULL != file) { + fclose(file); + file = NULL; + } + } +}; + +const size_t BufferedFileInputStreamReader::BUFFER_SIZE = 2000000; +const size_t BufferedFileInputStreamReader::MAX_UNREAD_COUNT = BufferedFileInputStreamReader::BUFFER_SIZE / 2; + +/* + * Streams to be used for reading data in checkers or validators. + * Each read*() method moves pointer to the next character after the + * read value. + */ +struct InStream { + /* Do not use them. */ + InStream(); + + ~InStream(); + + /* Wrap std::string with InStream. */ + InStream(const InStream &baseStream, std::string content); + + InputStreamReader *reader; + int lastLine; + + std::string name; + TMode mode; + bool opened; + bool stdfile; + bool strict; + + int wordReserveSize; + std::string _tmpReadToken; + + int readManyIteration; + size_t maxFileSize; + size_t maxTokenLength; + size_t maxMessageLength; + + void init(std::string fileName, TMode mode); + + void init(std::FILE *f, TMode mode); + + void setTestCase(int testCase); + std::vector getReadChars(); + + /* Moves stream pointer to the first non-white-space character or EOF. */ + void skipBlanks(); + + /* Returns current character in the stream. Doesn't remove it from stream. */ + char curChar(); + + /* Moves stream pointer one character forward. */ + void skipChar(); + + /* Returns current character and moves pointer one character forward. */ + char nextChar(); + + /* Returns current character and moves pointer one character forward. */ + char readChar(); + + /* As "readChar()" but ensures that the result is equal to given parameter. */ + char readChar(char c); + + /* As "readChar()" but ensures that the result is equal to the space (code=32). */ + char readSpace(); + + /* Puts back the character into the stream. */ + void unreadChar(char c); + + /* Reopens stream, you should not use it. */ + void reset(std::FILE *file = NULL); + + /* Checks that current position is EOF. If not it doesn't move stream pointer. */ + bool eof(); + + /* Moves pointer to the first non-white-space character and calls "eof()". */ + bool seekEof(); + + /* + * Checks that current position contains EOLN. + * If not it doesn't move stream pointer. + * In strict mode expects "#13#10" for windows or "#10" for other platforms. + */ + bool eoln(); + + /* Moves pointer to the first non-space and non-tab character and calls "eoln()". */ + bool seekEoln(); + + /* Moves stream pointer to the first character of the next line (if exists). */ + void nextLine(); + + /* + * Reads new token. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + std::string readWord(); + + /* The same as "readWord()", it is preferred to use "readToken()". */ + std::string readToken(); + + /* The same as "readWord()", but ensures that token matches to given pattern. */ + std::string readWord(const std::string &ptrn, const std::string &variableName = ""); + + std::string readWord(const pattern &p, const std::string &variableName = ""); + + std::vector + readWords(int size, const std::string &ptrn, const std::string &variablesName = "", int indexBase = 1); + + std::vector + readWords(int size, const pattern &p, const std::string &variablesName = "", int indexBase = 1); + + std::vector readWords(int size, int indexBase = 1); + + /* The same as "readToken()", but ensures that token matches to given pattern. */ + std::string readToken(const std::string &ptrn, const std::string &variableName = ""); + + std::string readToken(const pattern &p, const std::string &variableName = ""); + + std::vector + readTokens(int size, const std::string &ptrn, const std::string &variablesName = "", int indexBase = 1); + + std::vector + readTokens(int size, const pattern &p, const std::string &variablesName = "", int indexBase = 1); + + std::vector readTokens(int size, int indexBase = 1); + + void readWordTo(std::string &result); + + void readWordTo(std::string &result, const pattern &p, const std::string &variableName = ""); + + void readWordTo(std::string &result, const std::string &ptrn, const std::string &variableName = ""); + + void readTokenTo(std::string &result); + + void readTokenTo(std::string &result, const pattern &p, const std::string &variableName = ""); + + void readTokenTo(std::string &result, const std::string &ptrn, const std::string &variableName = ""); + + /* + * Reads new long long value. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + long long readLong(); + + unsigned long long readUnsignedLong(); + + /* + * Reads new int. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + int readInteger(); + + /* + * Reads new int. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + int readInt(); + + /* As "readLong()" but ensures that value in the range [minv,maxv]. */ + long long readLong(long long minv, long long maxv, const std::string &variableName = ""); + + /* Reads space-separated sequence of long longs. */ + std::vector + readLongs(int size, long long minv, long long maxv, const std::string &variablesName = "", int indexBase = 1); + + /* Reads space-separated sequence of long longs. */ + std::vector readLongs(int size, int indexBase = 1); + + unsigned long long + readUnsignedLong(unsigned long long minv, unsigned long long maxv, const std::string &variableName = ""); + + std::vector + readUnsignedLongs(int size, unsigned long long minv, unsigned long long maxv, const std::string &variablesName = "", + int indexBase = 1); + + std::vector readUnsignedLongs(int size, int indexBase = 1); + + unsigned long long readLong(unsigned long long minv, unsigned long long maxv, const std::string &variableName = ""); + + std::vector + readLongs(int size, unsigned long long minv, unsigned long long maxv, const std::string &variablesName = "", + int indexBase = 1); + + /* As "readInteger()" but ensures that value in the range [minv,maxv]. */ + int readInteger(int minv, int maxv, const std::string &variableName = ""); + + /* As "readInt()" but ensures that value in the range [minv,maxv]. */ + int readInt(int minv, int maxv, const std::string &variableName = ""); + + /* Reads space-separated sequence of integers. */ + std::vector + readIntegers(int size, int minv, int maxv, const std::string &variablesName = "", int indexBase = 1); + + /* Reads space-separated sequence of integers. */ + std::vector readIntegers(int size, int indexBase = 1); + + /* Reads space-separated sequence of integers. */ + std::vector readInts(int size, int minv, int maxv, const std::string &variablesName = "", int indexBase = 1); + + /* Reads space-separated sequence of integers. */ + std::vector readInts(int size, int indexBase = 1); + + /* + * Reads new double. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + double readReal(); + + /* + * Reads new double. Ignores white-spaces into the non-strict mode + * (strict mode is used in validators usually). + */ + double readDouble(); + + /* As "readReal()" but ensures that value in the range [minv,maxv]. */ + double readReal(double minv, double maxv, const std::string &variableName = ""); + + std::vector + readReals(int size, double minv, double maxv, const std::string &variablesName = "", int indexBase = 1); + + std::vector readReals(int size, int indexBase = 1); + + /* As "readDouble()" but ensures that value in the range [minv,maxv]. */ + double readDouble(double minv, double maxv, const std::string &variableName = ""); + + std::vector + readDoubles(int size, double minv, double maxv, const std::string &variablesName = "", int indexBase = 1); + + std::vector readDoubles(int size, int indexBase = 1); + + /* + * As "readReal()" but ensures that value in the range [minv,maxv] and + * number of digit after the decimal point is in range [minAfterPointDigitCount,maxAfterPointDigitCount] + * and number is in the form "[-]digit(s)[.digit(s)]". + */ + double readStrictReal(double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variableName = ""); + + std::vector readStrictReals(int size, double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variablesName = "", int indexBase = 1); + + /* + * As "readDouble()" but ensures that value in the range [minv,maxv] and + * number of digit after the decimal point is in range [minAfterPointDigitCount,maxAfterPointDigitCount] + * and number is in the form "[-]digit(s)[.digit(s)]". + */ + double readStrictDouble(double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variableName = ""); + + std::vector readStrictDoubles(int size, double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variablesName = "", int indexBase = 1); + + /* As readLine(). */ + std::string readString(); + + /* Read many lines. */ + std::vector readStrings(int size, int indexBase = 1); + + /* See readLine(). */ + void readStringTo(std::string &result); + + /* The same as "readLine()/readString()", but ensures that line matches to the given pattern. */ + std::string readString(const pattern &p, const std::string &variableName = ""); + + /* The same as "readLine()/readString()", but ensures that line matches to the given pattern. */ + std::string readString(const std::string &ptrn, const std::string &variableName = ""); + + /* Read many lines. */ + std::vector + readStrings(int size, const pattern &p, const std::string &variableName = "", int indexBase = 1); + + /* Read many lines. */ + std::vector + readStrings(int size, const std::string &ptrn, const std::string &variableName = "", int indexBase = 1); + + /* The same as "readLine()/readString()", but ensures that line matches to the given pattern. */ + void readStringTo(std::string &result, const pattern &p, const std::string &variableName = ""); + + /* The same as "readLine()/readString()", but ensures that line matches to the given pattern. */ + void readStringTo(std::string &result, const std::string &ptrn, const std::string &variableName = ""); + + /* + * Reads line from the current position to EOLN or EOF. Moves stream pointer to + * the first character of the new line (if possible). + */ + std::string readLine(); + + /* Read many lines. */ + std::vector readLines(int size, int indexBase = 1); + + /* See readLine(). */ + void readLineTo(std::string &result); + + /* The same as "readLine()", but ensures that line matches to the given pattern. */ + std::string readLine(const pattern &p, const std::string &variableName = ""); + + /* The same as "readLine()", but ensures that line matches to the given pattern. */ + std::string readLine(const std::string &ptrn, const std::string &variableName = ""); + + /* Read many lines. */ + std::vector + readLines(int size, const pattern &p, const std::string &variableName = "", int indexBase = 1); + + /* Read many lines. */ + std::vector + readLines(int size, const std::string &ptrn, const std::string &variableName = "", int indexBase = 1); + + /* The same as "readLine()", but ensures that line matches to the given pattern. */ + void readLineTo(std::string &result, const pattern &p, const std::string &variableName = ""); + + /* The same as "readLine()", but ensures that line matches to the given pattern. */ + void readLineTo(std::string &result, const std::string &ptrn, const std::string &variableName = ""); + + /* Reads EOLN or fails. Use it in validators. Calls "eoln()" method internally. */ + void readEoln(); + + /* Reads EOF or fails. Use it in validators. Calls "eof()" method internally. */ + void readEof(); + + /* + * Quit-functions aborts program with and : + * input/answer streams replace any result to FAIL. + */ + NORETURN void quit(TResult result, const char *msg); + /* + * Quit-functions aborts program with and : + * input/answer streams replace any result to FAIL. + */ + NORETURN void quitf(TResult result, const char *msg, ...); + + /* + * Quit-functions aborts program with and : + * input/answer streams replace any result to FAIL. + */ + void quitif(bool condition, TResult result, const char *msg, ...); + /* + * Quit-functions aborts program with and : + * input/answer streams replace any result to FAIL. + */ + NORETURN void quits(TResult result, std::string msg); + + /* + * Checks condition and aborts a program if condition is false. + * Returns _wa for ouf and _fail on any other streams. + */ +#ifdef __GNUC__ + __attribute__ ((format (printf, 3, 4))) +#endif + void ensuref(bool cond, const char *format, ...); + + void __testlib_ensure(bool cond, std::string message); + + void close(); + + const static int NO_INDEX = INT_MAX; + const static char OPEN_BRACKET = char(11); + const static char CLOSE_BRACKET = char(17); + + const static WORD LightGray = 0x07; + const static WORD LightRed = 0x0c; + const static WORD LightCyan = 0x0b; + const static WORD LightGreen = 0x0a; + const static WORD LightYellow = 0x0e; + + static void textColor(WORD color); + + static void quitscr(WORD color, const char *msg); + + static void quitscrS(WORD color, std::string msg); + + void xmlSafeWrite(std::FILE *file, const char *msg); + + /* Skips UTF-8 Byte Order Mark. */ + void skipBom(); + +private: + InStream(const InStream &); + + InStream &operator=(const InStream &); +}; + +InStream inf; +InStream ouf; +InStream ans; +bool appesMode; +std::string appesModeEncoding = "windows-1251"; +std::string resultName; +std::string checkerName = "untitled checker"; +random_t rnd; +TTestlibMode testlibMode = _unknown; +double __testlib_points = std::numeric_limits::infinity(); + +const size_t VALIDATOR_MAX_VARIABLE_COUNT = 255; + +struct ValidatorBoundsHit { + static const double EPS; + bool minHit; + bool maxHit; + + ValidatorBoundsHit(bool minHit = false, bool maxHit = false) : minHit(minHit), maxHit(maxHit) { + }; + + ValidatorBoundsHit merge(const ValidatorBoundsHit &validatorBoundsHit, bool ignoreMinBound, bool ignoreMaxBound) { + return ValidatorBoundsHit( + __testlib_max(minHit, validatorBoundsHit.minHit) || ignoreMinBound, + __testlib_max(maxHit, validatorBoundsHit.maxHit) || ignoreMaxBound + ); + } +}; + +struct ConstantBound { + std::string value; + bool broken; + + template + void adjust(T t) { + std::string t_string = std::to_string(t); + if (t_string.length() >= 32) { + broken = true; + value = ""; + } else { + if (!broken && value.empty()) + value = t_string; + if (!broken && value != t_string) { + broken = true; + value = ""; + } + } + } + + bool has_value() { + return !value.empty() && !broken && value.length() < 32; + } +}; + +struct ConstantBounds { + ConstantBound lowerBound; + ConstantBound upperBound; +}; + +const double ValidatorBoundsHit::EPS = 1E-12; + +class Validator { +private: + const static std::string TEST_MARKUP_HEADER; + const static std::string TEST_CASE_OPEN_TAG; + const static std::string TEST_CASE_CLOSE_TAG; + + bool _initialized; + std::string _testset; + std::string _group; + + std::string _testOverviewLogFileName; + std::string _testMarkupFileName; + int _testCase = -1; + std::string _testCaseFileName; + + std::map _boundsHitByVariableName; + std::map _constantBoundsByVariableName; + std::set _features; + std::set _hitFeatures; + std::set _variables; + + bool isVariableNameBoundsAnalyzable(const std::string &variableName) { + for (size_t i = 0; i < variableName.length(); i++) + if ((variableName[i] >= '0' && variableName[i] <= '9') || variableName[i] < ' ') + return false; + return true; + } + + bool isFeatureNameAnalyzable(const std::string &featureName) { + for (size_t i = 0; i < featureName.length(); i++) + if (featureName[i] < ' ') + return false; + return true; + } + +public: + Validator() : _initialized(false), _testset("tests"), _group() { + } + + void initialize() { + _initialized = true; + } + + std::string testset() const { + if (!_initialized) + __testlib_fail("Validator should be initialized with registerValidation(argc, argv) instead of registerValidation() to support validator.testset()"); + return _testset; + } + + std::string group() const { + if (!_initialized) + __testlib_fail("Validator should be initialized with registerValidation(argc, argv) instead of registerValidation() to support validator.group()"); + return _group; + } + + std::string testOverviewLogFileName() const { + return _testOverviewLogFileName; + } + + std::string testMarkupFileName() const { + return _testMarkupFileName; + } + + int testCase() const { + return _testCase; + } + + std::string testCaseFileName() const { + return _testCaseFileName; + } + + void setTestset(const char *const testset) { + _testset = testset; + } + + void setGroup(const char *const group) { + _group = group; + } + + void setTestOverviewLogFileName(const char *const testOverviewLogFileName) { + _testOverviewLogFileName = testOverviewLogFileName; + } + + void setTestMarkupFileName(const char *const testMarkupFileName) { + _testMarkupFileName = testMarkupFileName; + } + + void setTestCase(int testCase) { + _testCase = testCase; + } + + void setTestCaseFileName(const char *const testCaseFileName) { + _testCaseFileName = testCaseFileName; + } + + std::string prepVariableName(const std::string &variableName) { + if (variableName.length() >= 2 && variableName != "~~") { + if (variableName[0] == '~' && variableName.back() != '~') + return variableName.substr(1); + if (variableName[0] != '~' && variableName.back() == '~') + return variableName.substr(0, variableName.length() - 1); + if (variableName[0] == '~' && variableName.back() == '~') + return variableName.substr(1, variableName.length() - 2); + } + return variableName; + } + + bool ignoreMinBound(const std::string &variableName) { + return variableName.length() >= 2 && variableName != "~~" && variableName[0] == '~'; + } + + bool ignoreMaxBound(const std::string &variableName) { + return variableName.length() >= 2 && variableName != "~~" && variableName.back() == '~'; + } + + void addBoundsHit(const std::string &variableName, ValidatorBoundsHit boundsHit) { + if (isVariableNameBoundsAnalyzable(variableName) + && _boundsHitByVariableName.size() < VALIDATOR_MAX_VARIABLE_COUNT) { + std::string preparedVariableName = prepVariableName(variableName); + _boundsHitByVariableName[preparedVariableName] = boundsHit.merge(_boundsHitByVariableName[preparedVariableName], + ignoreMinBound(variableName), ignoreMaxBound(variableName)); + } + } + + void addVariable(const std::string &variableName) { + if (isVariableNameBoundsAnalyzable(variableName) + && _variables.size() < VALIDATOR_MAX_VARIABLE_COUNT) { + std::string preparedVariableName = prepVariableName(variableName); + _variables.insert(preparedVariableName); + } + } + + std::string getVariablesLog() { + std::string result; + for (const std::string &variableName: _variables) + result += "variable \"" + variableName + "\"\n"; + return result; + } + + template + void adjustConstantBounds(const std::string &variableName, T lower, T upper) { + if (isVariableNameBoundsAnalyzable(variableName) + && _constantBoundsByVariableName.size() < VALIDATOR_MAX_VARIABLE_COUNT) { + std::string preparedVariableName = prepVariableName(variableName); + _constantBoundsByVariableName[preparedVariableName].lowerBound.adjust(lower); + _constantBoundsByVariableName[preparedVariableName].upperBound.adjust(upper); + } + } + + std::string getBoundsHitLog() { + std::string result; + for (std::map::iterator i = _boundsHitByVariableName.begin(); + i != _boundsHitByVariableName.end(); + i++) { + result += "\"" + i->first + "\":"; + if (i->second.minHit) + result += " min-value-hit"; + if (i->second.maxHit) + result += " max-value-hit"; + result += "\n"; + } + return result; + } + + std::string getConstantBoundsLog() { + std::string result; + for (std::map::iterator i = _constantBoundsByVariableName.begin(); + i != _constantBoundsByVariableName.end(); + i++) { + if (i->second.lowerBound.has_value() || i->second.upperBound.has_value()) { + result += "constant-bounds \"" + i->first + "\":"; + if (i->second.lowerBound.has_value()) + result += " " + i->second.lowerBound.value; + else + result += " ?"; + if (i->second.upperBound.has_value()) + result += " " + i->second.upperBound.value; + else + result += " ?"; + result += "\n"; + } + } + return result; + } + + std::string getFeaturesLog() { + std::string result; + for (std::set::iterator i = _features.begin(); + i != _features.end(); + i++) { + result += "feature \"" + *i + "\":"; + if (_hitFeatures.count(*i)) + result += " hit"; + result += "\n"; + } + return result; + } + + void writeTestOverviewLog() { + if (!_testOverviewLogFileName.empty()) { + std::string fileName(_testOverviewLogFileName); + _testOverviewLogFileName = ""; + + FILE* f; + bool standard_file = false; + if (fileName == "stdout") + f = stdout, standard_file = true; + else if (fileName == "stderr") + f = stderr, standard_file = true; + else { + f = testlib_fopen_(fileName.c_str(), "wb"); + if (NULL == f) + __testlib_fail("Validator::writeTestOverviewLog: can't write test overview log to (" + fileName + ")"); + } + fprintf(f, "%s%s%s%s", + getBoundsHitLog().c_str(), + getFeaturesLog().c_str(), + getConstantBoundsLog().c_str(), + getVariablesLog().c_str()); + std::fflush(f); + if (!standard_file) + if (std::fclose(f)) + __testlib_fail("Validator::writeTestOverviewLog: can't close test overview log file (" + fileName + ")"); + } + } + + void writeTestMarkup() { + if (!_testMarkupFileName.empty()) { + std::vector readChars = inf.getReadChars(); + if (!readChars.empty()) { + std::string markup(TEST_MARKUP_HEADER); + for (size_t i = 0; i < readChars.size(); i++) { + int c = readChars[i]; + if (i + 1 == readChars.size() && c == -1) + continue; + if (c <= 256) { + char cc = char(c); + if (cc == '\\' || cc == '!') + markup += '\\'; + markup += cc; + } else { + markup += TEST_CASE_OPEN_TAG; + markup += toString(c - 256); + markup += TEST_CASE_CLOSE_TAG; + } + } + FILE* f; + bool standard_file = false; + if (_testMarkupFileName == "stdout") + f = stdout, standard_file = true; + else if (_testMarkupFileName == "stderr") + f = stderr, standard_file = true; + else { + f = testlib_fopen_(_testMarkupFileName.c_str(), "wb"); + if (NULL == f) + __testlib_fail("Validator::writeTestMarkup: can't write test markup to (" + _testMarkupFileName + ")"); + } + std::fprintf(f, "%s", markup.c_str()); + std::fflush(f); + if (!standard_file) + if (std::fclose(f)) + __testlib_fail("Validator::writeTestMarkup: can't close test markup file (" + _testCaseFileName + ")"); + } + } + } + + void writeTestCase() { + if (_testCase > 0) { + std::vector readChars = inf.getReadChars(); + if (!readChars.empty()) { + std::string content, testCaseContent; + bool matchedTestCase = false; + for (size_t i = 0; i < readChars.size(); i++) { + int c = readChars[i]; + if (i + 1 == readChars.size() && c == -1) + continue; + if (c <= 256) + content += char(c); + else { + if (matchedTestCase) { + testCaseContent = content; + matchedTestCase = false; + } + content = ""; + int testCase = c - 256; + if (testCase == _testCase) + matchedTestCase = true; + } + } + if (matchedTestCase) + testCaseContent = content; + + if (!testCaseContent.empty()) { + FILE* f; + bool standard_file = false; + if (_testCaseFileName.empty() || _testCaseFileName == "stdout") + f = stdout, standard_file = true; + else if (_testCaseFileName == "stderr") + f = stderr, standard_file = true; + else { + f = testlib_fopen_(_testCaseFileName.c_str(), "wb"); + if (NULL == f) + __testlib_fail("Validator::writeTestCase: can't write test case to (" + _testCaseFileName + ")"); + } + std::fprintf(f, "%s", testCaseContent.c_str()); + std::fflush(f); + if (!standard_file) + if (std::fclose(f)) + __testlib_fail("Validator::writeTestCase: can't close test case file (" + _testCaseFileName + ")"); + } + } + } + } + + void addFeature(const std::string &feature) { + if (_features.count(feature)) + __testlib_fail("Feature " + feature + " registered twice."); + if (!isFeatureNameAnalyzable(feature)) + __testlib_fail("Feature name '" + feature + "' contains restricted characters."); + + _features.insert(feature); + } + + void feature(const std::string &feature) { + if (!isFeatureNameAnalyzable(feature)) + __testlib_fail("Feature name '" + feature + "' contains restricted characters."); + + if (!_features.count(feature)) + __testlib_fail("Feature " + feature + " didn't registered via addFeature(feature)."); + + _hitFeatures.insert(feature); + } +} validator; + +const std::string Validator::TEST_MARKUP_HEADER = "MU\xF3\x01"; +const std::string Validator::TEST_CASE_OPEN_TAG = "!c"; +const std::string Validator::TEST_CASE_CLOSE_TAG = ";"; + +struct TestlibFinalizeGuard { + static bool alive; + static bool registered; + + int quitCount, readEofCount; + + TestlibFinalizeGuard() : quitCount(0), readEofCount(0) { + // No operations. + } + + ~TestlibFinalizeGuard() { + bool _alive = alive; + alive = false; + + if (_alive) { + if (testlibMode == _checker && quitCount == 0) + __testlib_fail("Checker must end with quit or quitf call."); + + if (testlibMode == _validator && readEofCount == 0 && quitCount == 0) + __testlib_fail("Validator must end with readEof call."); + + /* opts */ + autoEnsureNoUnusedOpts(); + + if (!registered) + __testlib_fail("Call register-function in the first line of the main (registerTestlibCmd or other similar)"); + } + + if (__testlib_exitCode == 0) { + validator.writeTestOverviewLog(); + validator.writeTestMarkup(); + validator.writeTestCase(); + } + } + +private: + /* opts */ + void autoEnsureNoUnusedOpts(); +}; + +bool TestlibFinalizeGuard::alive = true; +bool TestlibFinalizeGuard::registered = false; +extern TestlibFinalizeGuard testlibFinalizeGuard; + +/* + * Call it to disable checks on finalization. + */ +void disableFinalizeGuard() { + TestlibFinalizeGuard::alive = false; +} + +/* Interactor streams. + */ +std::fstream tout; + +/* implementation + */ + +InStream::InStream() { + reader = NULL; + lastLine = -1; + opened = false; + name = ""; + mode = _input; + strict = false; + stdfile = false; + wordReserveSize = 4; + readManyIteration = NO_INDEX; + maxFileSize = 128 * 1024 * 1024; // 128MB. + maxTokenLength = 32 * 1024 * 1024; // 32MB. + maxMessageLength = 32000; +} + +InStream::InStream(const InStream &baseStream, std::string content) { + reader = new StringInputStreamReader(content); + lastLine = -1; + opened = true; + strict = baseStream.strict; + stdfile = false; + mode = baseStream.mode; + name = "based on " + baseStream.name; + readManyIteration = NO_INDEX; + maxFileSize = 128 * 1024 * 1024; // 128MB. + maxTokenLength = 32 * 1024 * 1024; // 32MB. + maxMessageLength = 32000; +} + +InStream::~InStream() { + if (NULL != reader) { + reader->close(); + delete reader; + reader = NULL; + } +} + +void InStream::setTestCase(int testCase) { + if (testlibMode != _validator || mode != _input || !stdfile || this != &inf) + __testlib_fail("InStream::setTestCase can be used only for inf in validator-mode." + " Actually, prefer setTestCase function instead of InStream member"); + reader->setTestCase(testCase); +} + +std::vector InStream::getReadChars() { + if (testlibMode != _validator || mode != _input || !stdfile || this != &inf) + __testlib_fail("InStream::getReadChars can be used only for inf in validator-mode."); + return reader == NULL ? std::vector() : reader->getReadChars(); +} + +void setTestCase(int testCase) { + static bool first_run = true; + static bool zero_based = false; + + if (first_run && testCase == 0) + zero_based = true; + + if (zero_based) + testCase++; + + __testlib_hasTestCase = true; + __testlib_testCase = testCase; + + if (testlibMode == _validator) + inf.setTestCase(testCase); + + first_run = false; +} + +#ifdef __GNUC__ +__attribute__((const)) +#endif +int resultExitCode(TResult r) { + if (r == _ok) + return OK_EXIT_CODE; + if (r == _wa) + return WA_EXIT_CODE; + if (r == _pe) + return PE_EXIT_CODE; + if (r == _fail) + return FAIL_EXIT_CODE; + if (r == _dirt) + return DIRT_EXIT_CODE; + if (r == _points) + return POINTS_EXIT_CODE; + if (r == _unexpected_eof) +#ifdef ENABLE_UNEXPECTED_EOF + return UNEXPECTED_EOF_EXIT_CODE; +#else + return PE_EXIT_CODE; +#endif + if (r >= _partially) + return PC_BASE_EXIT_CODE + (r - _partially); + return FAIL_EXIT_CODE; +} + +void InStream::textColor( +#if !(defined(ON_WINDOWS) && (!defined(_MSC_VER) || _MSC_VER > 1400)) && defined(__GNUC__) + __attribute__((unused)) +#endif + WORD color +) { +#if defined(ON_WINDOWS) && (!defined(_MSC_VER) || _MSC_VER > 1400) + HANDLE handle = GetStdHandle(STD_OUTPUT_HANDLE); + SetConsoleTextAttribute(handle, color); +#endif +#if !defined(ON_WINDOWS) && defined(__GNUC__) + if (isatty(2)) + { + switch (color) + { + case LightRed: + fprintf(stderr, "\033[1;31m"); + break; + case LightCyan: + fprintf(stderr, "\033[1;36m"); + break; + case LightGreen: + fprintf(stderr, "\033[1;32m"); + break; + case LightYellow: + fprintf(stderr, "\033[1;33m"); + break; + case LightGray: + default: + fprintf(stderr, "\033[0m"); + } + } +#endif +} + +#ifdef TESTLIB_THROW_EXIT_EXCEPTION_INSTEAD_OF_EXIT +class exit_exception: public std::exception { +private: + int exitCode; +public: + exit_exception(int exitCode): exitCode(exitCode) {} + int getExitCode() { return exitCode; } +}; +#endif + +NORETURN void halt(int exitCode) { +#ifdef FOOTER + InStream::textColor(InStream::LightGray); + std::fprintf(stderr, "Checker: \"%s\"\n", checkerName.c_str()); + std::fprintf(stderr, "Exit code: %d\n", exitCode); + InStream::textColor(InStream::LightGray); +#endif + __testlib_exitCode = exitCode; +#ifdef TESTLIB_THROW_EXIT_EXCEPTION_INSTEAD_OF_EXIT + throw exit_exception(exitCode); +#endif + std::exit(exitCode); +} + +static bool __testlib_shouldCheckDirt(TResult result) { + return result == _ok || result == _points || result >= _partially; +} + +static std::string __testlib_appendMessage(const std::string &message, const std::string &extra) { + int openPos = -1, closePos = -1; + for (size_t i = 0; i < message.length(); i++) { + if (message[i] == InStream::OPEN_BRACKET) { + if (openPos == -1) + openPos = int(i); + else + openPos = INT_MAX; + } + if (message[i] == InStream::CLOSE_BRACKET) { + if (closePos == -1) + closePos = int(i); + else + closePos = INT_MAX; + } + } + if (openPos != -1 && openPos != INT_MAX + && closePos != -1 && closePos != INT_MAX + && openPos < closePos) { + size_t index = message.find(extra, openPos); + if (index == std::string::npos || int(index) >= closePos) { + std::string result(message); + result.insert(closePos, ", " + extra); + return result; + } + return message; + } + + return message + " " + InStream::OPEN_BRACKET + extra + InStream::CLOSE_BRACKET; +} + +static std::string __testlib_toPrintableMessage(const std::string &message) { + int openPos = -1, closePos = -1; + for (size_t i = 0; i < message.length(); i++) { + if (message[i] == InStream::OPEN_BRACKET) { + if (openPos == -1) + openPos = int(i); + else + openPos = INT_MAX; + } + if (message[i] == InStream::CLOSE_BRACKET) { + if (closePos == -1) + closePos = int(i); + else + closePos = INT_MAX; + } + } + if (openPos != -1 && openPos != INT_MAX + && closePos != -1 && closePos != INT_MAX + && openPos < closePos) { + std::string result(message); + result[openPos] = '('; + result[closePos] = ')'; + return result; + } + + return message; +} + +NORETURN void InStream::quit(TResult result, const char *msg) { + if (TestlibFinalizeGuard::alive) + testlibFinalizeGuard.quitCount++; + + std::string message(msg); + message = trim(message); + + if (__testlib_hasTestCase) { + if (result != _ok && result != _points) + message = __testlib_appendMessage(message, "test case " + vtos(__testlib_testCase)); + else { + if (__testlib_testCase == 1) + message = __testlib_appendMessage(message, vtos(__testlib_testCase) + " test case"); + else + message = __testlib_appendMessage(message, vtos(__testlib_testCase) + " test cases"); + } + } + + // You can change maxMessageLength. + // Example: 'inf.maxMessageLength = 1024 * 1024;'. + if (message.length() > maxMessageLength) { + std::string warn = "message length exceeds " + vtos(maxMessageLength) + + ", the message is truncated: "; + message = warn + message.substr(0, maxMessageLength - warn.length()); + } + +#ifndef ENABLE_UNEXPECTED_EOF + if (result == _unexpected_eof) + result = _pe; +#endif + + if (testlibMode == _scorer && result != _fail) + quits(_fail, "Scorer should return points only. Don't use a quit function."); + + if (mode != _output && result != _fail) { + if (mode == _input && testlibMode == _validator && lastLine != -1) + quits(_fail, __testlib_appendMessage(__testlib_appendMessage(message, name), "line " + vtos(lastLine))); + else + quits(_fail, __testlib_appendMessage(message, name)); + } + + std::FILE *resultFile; + std::string errorName; + + if (__testlib_shouldCheckDirt(result)) { + if (testlibMode != _interactor && !ouf.seekEof()) + quit(_dirt, "Extra information in the output file"); + } + + int pctype = result - _partially; + bool isPartial = false; + + switch (result) { + case _ok: + errorName = "ok "; + quitscrS(LightGreen, errorName); + break; + case _wa: + errorName = "wrong answer "; + quitscrS(LightRed, errorName); + break; + case _pe: + errorName = "wrong output format "; + quitscrS(LightRed, errorName); + break; + case _fail: + errorName = "FAIL "; + quitscrS(LightRed, errorName); + break; + case _dirt: + errorName = "wrong output format "; + quitscrS(LightCyan, errorName); + result = _pe; + break; + case _points: + errorName = "points "; + quitscrS(LightYellow, errorName); + break; + case _unexpected_eof: + errorName = "unexpected eof "; + quitscrS(LightCyan, errorName); + break; + default: + if (result >= _partially) { + errorName = testlib_format_("partially correct (%d) ", pctype); + isPartial = true; + quitscrS(LightYellow, errorName); + } else + quit(_fail, "What is the code ??? "); + } + + if (resultName != "") { + resultFile = testlib_fopen_(resultName.c_str(), "w"); + if (resultFile == NULL) { + resultName = ""; + quit(_fail, "Can not write to the result file"); + } + if (appesMode) { + std::fprintf(resultFile, "", appesModeEncoding.c_str()); + if (isPartial) + std::fprintf(resultFile, "", + outcomes[(int) _partially].c_str(), pctype); + else { + if (result != _points) + std::fprintf(resultFile, "", outcomes[(int) result].c_str()); + else { + if (__testlib_points == std::numeric_limits::infinity()) + quit(_fail, "Expected points, but infinity found"); + std::string stringPoints = removeDoubleTrailingZeroes(testlib_format_("%.10f", __testlib_points)); + std::fprintf(resultFile, "", + outcomes[(int) result].c_str(), stringPoints.c_str()); + } + } + xmlSafeWrite(resultFile, __testlib_toPrintableMessage(message).c_str()); + std::fprintf(resultFile, "\n"); + } else + std::fprintf(resultFile, "%s", __testlib_toPrintableMessage(message).c_str()); + if (NULL == resultFile || fclose(resultFile) != 0) { + resultName = ""; + quit(_fail, "Can not write to the result file"); + } + } + + quitscr(LightGray, __testlib_toPrintableMessage(message).c_str()); + std::fprintf(stderr, "\n"); + + inf.close(); + ouf.close(); + ans.close(); + if (tout.is_open()) + tout.close(); + + textColor(LightGray); + + if (resultName != "") + std::fprintf(stderr, "See file to check exit message\n"); + + halt(resultExitCode(result)); +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 3, 4))) +#endif +NORETURN void InStream::quitf(TResult result, const char *msg, ...) { + FMT_TO_RESULT(msg, msg, message); + InStream::quit(result, message.c_str()); +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +void InStream::quitif(bool condition, TResult result, const char *msg, ...) { + if (condition) { + FMT_TO_RESULT(msg, msg, message); + InStream::quit(result, message.c_str()); + } +} + +NORETURN void InStream::quits(TResult result, std::string msg) { + InStream::quit(result, msg.c_str()); +} + +void InStream::xmlSafeWrite(std::FILE *file, const char *msg) { + size_t lmsg = strlen(msg); + for (size_t i = 0; i < lmsg; i++) { + if (msg[i] == '&') { + std::fprintf(file, "%s", "&"); + continue; + } + if (msg[i] == '<') { + std::fprintf(file, "%s", "<"); + continue; + } + if (msg[i] == '>') { + std::fprintf(file, "%s", ">"); + continue; + } + if (msg[i] == '"') { + std::fprintf(file, "%s", """); + continue; + } + if (0 <= msg[i] && msg[i] <= 31) { + std::fprintf(file, "%c", '.'); + continue; + } + std::fprintf(file, "%c", msg[i]); + } +} + +void InStream::quitscrS(WORD color, std::string msg) { + quitscr(color, msg.c_str()); +} + +void InStream::quitscr(WORD color, const char *msg) { + if (resultName == "") { + textColor(color); + std::fprintf(stderr, "%s", msg); + textColor(LightGray); + } +} + +void InStream::reset(std::FILE *file) { + if (opened && stdfile) + quit(_fail, "Can't reset standard handle"); + + if (opened) + close(); + + if (!stdfile && NULL == file) + if (NULL == (file = testlib_fopen_(name.c_str(), "rb"))) { + if (mode == _output) + quits(_pe, std::string("Output file not found: \"") + name + "\""); + + if (mode == _answer) + quits(_fail, std::string("Answer file not found: \"") + name + "\""); + } + + if (NULL != file) { + opened = true; + __testlib_set_binary(file); + + if (stdfile) + reader = new FileInputStreamReader(file, name); + else + reader = new BufferedFileInputStreamReader(file, name); + } else { + opened = false; + reader = NULL; + } +} + +void InStream::init(std::string fileName, TMode mode) { + opened = false; + name = fileName; + stdfile = false; + this->mode = mode; + + std::ifstream stream; + stream.open(fileName.c_str(), std::ios::in); + if (stream.is_open()) { + std::streampos start = stream.tellg(); + stream.seekg(0, std::ios::end); + std::streampos end = stream.tellg(); + size_t fileSize = size_t(end - start); + stream.close(); + + // You can change maxFileSize. + // Example: 'inf.maxFileSize = 256 * 1024 * 1024;'. + if (fileSize > maxFileSize) + quitf(_pe, "File size exceeds %d bytes, size is %d", int(maxFileSize), int(fileSize)); + } + + reset(); +} + +void InStream::init(std::FILE *f, TMode mode) { + opened = false; + name = "untitled"; + this->mode = mode; + + if (f == stdin) + name = "stdin", stdfile = true; + if (f == stdout) + name = "stdout", stdfile = true; + if (f == stderr) + name = "stderr", stdfile = true; + + reset(f); +} + +void InStream::skipBom() { + const std::string utf8Bom = "\xEF\xBB\xBF"; + size_t index = 0; + while (index < utf8Bom.size() && curChar() == utf8Bom[index]) { + index++; + skipChar(); + } + if (index < utf8Bom.size()) { + while (index != 0) { + unreadChar(utf8Bom[index - 1]); + index--; + } + } +} + +char InStream::curChar() { + return char(reader->curChar()); +} + +char InStream::nextChar() { + return char(reader->nextChar()); +} + +char InStream::readChar() { + return nextChar(); +} + +char InStream::readChar(char c) { + lastLine = reader->getLine(); + char found = readChar(); + if (c != found) { + if (!isEoln(found)) + quit(_pe, ("Unexpected character '" + std::string(1, found) + "', but '" + std::string(1, c) + + "' expected").c_str()); + else + quit(_pe, ("Unexpected character " + ("#" + vtos(int(found))) + ", but '" + std::string(1, c) + + "' expected").c_str()); + } + return found; +} + +char InStream::readSpace() { + return readChar(' '); +} + +void InStream::unreadChar(char c) { + reader->unreadChar(c); +} + +void InStream::skipChar() { + reader->skipChar(); +} + +void InStream::skipBlanks() { + while (isBlanks(reader->curChar())) + reader->skipChar(); +} + +std::string InStream::readWord() { + readWordTo(_tmpReadToken); + return _tmpReadToken; +} + +void InStream::readWordTo(std::string &result) { + if (!strict) + skipBlanks(); + + lastLine = reader->getLine(); + int cur = reader->nextChar(); + + if (cur == EOFC) + quit(_unexpected_eof, "Unexpected end of file - token expected"); + + if (isBlanks(cur)) + quit(_pe, "Unexpected white-space - token expected"); + + result.clear(); + + while (!(isBlanks(cur) || cur == EOFC)) { + result += char(cur); + + // You can change maxTokenLength. + // Example: 'inf.maxTokenLength = 128 * 1024 * 1024;'. + if (result.length() > maxTokenLength) + quitf(_pe, "Length of token exceeds %d, token is '%s...'", int(maxTokenLength), + __testlib_part(result).c_str()); + + cur = reader->nextChar(); + } + + reader->unreadChar(cur); + + if (result.length() == 0) + quit(_unexpected_eof, "Unexpected end of file or white-space - token expected"); +} + +std::string InStream::readToken() { + return readWord(); +} + +void InStream::readTokenTo(std::string &result) { + readWordTo(result); +} + +static std::string __testlib_part(const std::string &s) { + std::string t; + for (size_t i = 0; i < s.length(); i++) + if (s[i] != '\0') + t += s[i]; + else + t += '~'; + if (t.length() <= 64) + return t; + else + return t.substr(0, 30) + "..." + t.substr(s.length() - 31, 31); +} + +#define __testlib_readMany(readMany, readOne, typeName, space) \ + if (size < 0) \ + quit(_fail, #readMany ": size should be non-negative."); \ + if (size > 100000000) \ + quit(_fail, #readMany ": size should be at most 100000000."); \ + \ + std::vector result(size); \ + readManyIteration = indexBase; \ + \ + for (int i = 0; i < size; i++) \ + { \ + result[i] = readOne; \ + readManyIteration++; \ + if (strict && space && i + 1 < size) \ + readSpace(); \ + } \ + \ + readManyIteration = NO_INDEX; \ + return result; \ + + +std::string InStream::readWord(const pattern &p, const std::string &variableName) { + readWordTo(_tmpReadToken); + if (!p.matches(_tmpReadToken)) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, + ("Token \"" + __testlib_part(_tmpReadToken) + "\" doesn't correspond to pattern \"" + p.src() + + "\"").c_str()); + else + quit(_wa, ("Token parameter [name=" + variableName + "] equals to \"" + __testlib_part(_tmpReadToken) + + "\", doesn't correspond to pattern \"" + p.src() + "\"").c_str()); + } else { + if (variableName.empty()) + quit(_wa, ("Token element [index=" + vtos(readManyIteration) + "] equals to \"" + + __testlib_part(_tmpReadToken) + "\" doesn't correspond to pattern \"" + p.src() + + "\"").c_str()); + else + quit(_wa, ("Token element " + variableName + "[" + vtos(readManyIteration) + "] equals to \"" + + __testlib_part(_tmpReadToken) + "\", doesn't correspond to pattern \"" + p.src() + + "\"").c_str()); + } + } + if (strict && !variableName.empty()) + validator.addVariable(variableName); + return _tmpReadToken; +} + +std::vector +InStream::readWords(int size, const pattern &p, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readWords, readWord(p, variablesName), std::string, true); +} + +std::vector InStream::readWords(int size, int indexBase) { + __testlib_readMany(readWords, readWord(), std::string, true); +} + +std::string InStream::readWord(const std::string &ptrn, const std::string &variableName) { + return readWord(pattern(ptrn), variableName); +} + +std::vector +InStream::readWords(int size, const std::string &ptrn, const std::string &variablesName, int indexBase) { + pattern p(ptrn); + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readWords, readWord(p, variablesName), std::string, true); +} + +std::string InStream::readToken(const pattern &p, const std::string &variableName) { + return readWord(p, variableName); +} + +std::vector +InStream::readTokens(int size, const pattern &p, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readTokens, readToken(p, variablesName), std::string, true); +} + +std::vector InStream::readTokens(int size, int indexBase) { + __testlib_readMany(readTokens, readToken(), std::string, true); +} + +std::string InStream::readToken(const std::string &ptrn, const std::string &variableName) { + return readWord(ptrn, variableName); +} + +std::vector +InStream::readTokens(int size, const std::string &ptrn, const std::string &variablesName, int indexBase) { + pattern p(ptrn); + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readTokens, readWord(p, variablesName), std::string, true); +} + +void InStream::readWordTo(std::string &result, const pattern &p, const std::string &variableName) { + readWordTo(result); + if (!p.matches(result)) { + if (variableName.empty()) + quit(_wa, ("Token \"" + __testlib_part(result) + "\" doesn't correspond to pattern \"" + p.src() + + "\"").c_str()); + else + quit(_wa, ("Token parameter [name=" + variableName + "] equals to \"" + __testlib_part(result) + + "\", doesn't correspond to pattern \"" + p.src() + "\"").c_str()); + } + if (strict && !variableName.empty()) + validator.addVariable(variableName); +} + +void InStream::readWordTo(std::string &result, const std::string &ptrn, const std::string &variableName) { + return readWordTo(result, pattern(ptrn), variableName); +} + +void InStream::readTokenTo(std::string &result, const pattern &p, const std::string &variableName) { + return readWordTo(result, p, variableName); +} + +void InStream::readTokenTo(std::string &result, const std::string &ptrn, const std::string &variableName) { + return readWordTo(result, ptrn, variableName); +} + +#ifdef __GNUC__ +__attribute__((pure)) +#endif +static inline bool equals(long long integer, const char *s) { + if (integer == LLONG_MIN) + return strcmp(s, "-9223372036854775808") == 0; + + if (integer == 0LL) + return strcmp(s, "0") == 0; + + size_t length = strlen(s); + + if (length == 0) + return false; + + if (integer < 0 && s[0] != '-') + return false; + + if (integer < 0) + s++, length--, integer = -integer; + + if (length == 0) + return false; + + while (integer > 0) { + int digit = int(integer % 10); + + if (s[length - 1] != '0' + digit) + return false; + + length--; + integer /= 10; + } + + return length == 0; +} + +#ifdef __GNUC__ +__attribute__((pure)) +#endif +static inline bool equals(unsigned long long integer, const char *s) { + if (integer == ULLONG_MAX) + return strcmp(s, "18446744073709551615") == 0; + + if (integer == 0ULL) + return strcmp(s, "0") == 0; + + size_t length = strlen(s); + + if (length == 0) + return false; + + while (integer > 0) { + int digit = int(integer % 10); + + if (s[length - 1] != '0' + digit) + return false; + + length--; + integer /= 10; + } + + return length == 0; +} + +static inline double stringToDouble(InStream &in, const char *buffer) { + double result; + + size_t length = strlen(buffer); + + int minusCount = 0; + int plusCount = 0; + int decimalPointCount = 0; + int digitCount = 0; + int eCount = 0; + + for (size_t i = 0; i < length; i++) { + if (('0' <= buffer[i] && buffer[i] <= '9') || buffer[i] == '.' + || buffer[i] == 'e' || buffer[i] == 'E' + || buffer[i] == '-' || buffer[i] == '+') { + if ('0' <= buffer[i] && buffer[i] <= '9') + digitCount++; + if (buffer[i] == 'e' || buffer[i] == 'E') + eCount++; + if (buffer[i] == '-') + minusCount++; + if (buffer[i] == '+') + plusCount++; + if (buffer[i] == '.') + decimalPointCount++; + } else + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + // If for sure is not a number in standard notation or in e-notation. + if (digitCount == 0 || minusCount > 2 || plusCount > 2 || decimalPointCount > 1 || eCount > 1) + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + char *suffix = new char[length + 1]; + std::memset(suffix, 0, length + 1); + int scanned; +#ifdef _MSC_VER + scanned = sscanf_s(buffer, "%lf%s", &result, suffix, (unsigned int)(length + 1)); +#else + scanned = std::sscanf(buffer, "%lf%s", &result, suffix); +#endif + bool empty = strlen(suffix) == 0; + delete[] suffix; + + if (scanned == 1 || (scanned == 2 && empty)) { + if (__testlib_isNaN(result)) + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + return result; + } else + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); +} + +static inline double stringToDouble(InStream &in, const std::string& buffer) { + for (size_t i = 0; i < buffer.length(); i++) + if (buffer[i] == '\0') + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found (it contains \\0)").c_str()); + return stringToDouble(in, buffer.c_str()); +} + +static inline double stringToStrictDouble(InStream &in, const char *buffer, + int minAfterPointDigitCount, int maxAfterPointDigitCount) { + if (minAfterPointDigitCount < 0) + in.quit(_fail, "stringToStrictDouble: minAfterPointDigitCount should be non-negative."); + + if (minAfterPointDigitCount > maxAfterPointDigitCount) + in.quit(_fail, + "stringToStrictDouble: minAfterPointDigitCount should be less or equal to maxAfterPointDigitCount."); + + double result; + + size_t length = strlen(buffer); + + if (length == 0 || length > 1000) + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + if (buffer[0] != '-' && (buffer[0] < '0' || buffer[0] > '9')) + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + int pointPos = -1; + for (size_t i = 1; i + 1 < length; i++) { + if (buffer[i] == '.') { + if (pointPos > -1) + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + pointPos = int(i); + } + if (buffer[i] != '.' && (buffer[i] < '0' || buffer[i] > '9')) + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + if (buffer[length - 1] < '0' || buffer[length - 1] > '9') + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + int afterDigitsCount = (pointPos == -1 ? 0 : int(length) - pointPos - 1); + if (afterDigitsCount < minAfterPointDigitCount || afterDigitsCount > maxAfterPointDigitCount) + in.quit(_pe, ("Expected strict double with number of digits after point in range [" + + vtos(minAfterPointDigitCount) + + "," + + vtos(maxAfterPointDigitCount) + + "], but \"" + __testlib_part(buffer) + "\" found").c_str() + ); + + int firstDigitPos = -1; + for (size_t i = 0; i < length; i++) + if (buffer[i] >= '0' && buffer[i] <= '9') { + firstDigitPos = int(i); + break; + } + + if (firstDigitPos > 1 || firstDigitPos == -1) + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + if (buffer[firstDigitPos] == '0' && firstDigitPos + 1 < int(length) + && buffer[firstDigitPos + 1] >= '0' && buffer[firstDigitPos + 1] <= '9') + in.quit(_pe, ("Expected strict double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + char *suffix = new char[length + 1]; + std::memset(suffix, 0, length + 1); + int scanned; +#ifdef _MSC_VER + scanned = sscanf_s(buffer, "%lf%s", &result, suffix, (unsigned int)(length + 1)); +#else + scanned = std::sscanf(buffer, "%lf%s", &result, suffix); +#endif + bool empty = strlen(suffix) == 0; + delete[] suffix; + + if (scanned == 1 || (scanned == 2 && empty)) { + if (__testlib_isNaN(result) || __testlib_isInfinite(result)) + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); + if (buffer[0] == '-' && result >= 0) + in.quit(_pe, ("Redundant minus in \"" + __testlib_part(buffer) + "\" found").c_str()); + return result; + } else + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found").c_str()); +} + +static inline double stringToStrictDouble(InStream &in, const std::string& buffer, + int minAfterPointDigitCount, int maxAfterPointDigitCount) { + for (size_t i = 0; i < buffer.length(); i++) + if (buffer[i] == '\0') + in.quit(_pe, ("Expected double, but \"" + __testlib_part(buffer) + "\" found (it contains \\0)").c_str()); + return stringToStrictDouble(in, buffer.c_str(), minAfterPointDigitCount, maxAfterPointDigitCount); +} + +static inline long long stringToLongLong(InStream &in, const char *buffer) { + size_t length = strlen(buffer); + if (length == 0 || length > 20) + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + bool has_minus = (length > 1 && buffer[0] == '-'); + int zeroes = 0; + bool processingZeroes = true; + + for (int i = (has_minus ? 1 : 0); i < int(length); i++) { + if (buffer[i] == '0' && processingZeroes) + zeroes++; + else + processingZeroes = false; + + if (buffer[i] < '0' || buffer[i] > '9') + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + long long int result; + try { + result = std::stoll(buffer); + } catch (const std::exception&) { + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } catch (...) { + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + if ((zeroes > 0 && (result != 0 || has_minus)) || zeroes > 1) + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + return result; +} + +static inline long long stringToLongLong(InStream &in, const std::string& buffer) { + for (size_t i = 0; i < buffer.length(); i++) + if (buffer[i] == '\0') + in.quit(_pe, ("Expected integer, but \"" + __testlib_part(buffer) + "\" found (it contains \\0)").c_str()); + return stringToLongLong(in, buffer.c_str()); +} + +static inline unsigned long long stringToUnsignedLongLong(InStream &in, const char *buffer) { + size_t length = strlen(buffer); + + if (length == 0 || length > 20) + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + if (length > 1 && buffer[0] == '0') + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + + for (int i = 0; i < int(length); i++) { + if (buffer[i] < '0' || buffer[i] > '9') + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + unsigned long long result; + try { + result = std::stoull(buffer); + } catch (const std::exception&) { + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } catch (...) { + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found").c_str()); + } + + return result; +} + +static inline long long stringToUnsignedLongLong(InStream &in, const std::string& buffer) { + for (size_t i = 0; i < buffer.length(); i++) + if (buffer[i] == '\0') + in.quit(_pe, ("Expected unsigned integer, but \"" + __testlib_part(buffer) + "\" found (it contains \\0)").c_str()); + return stringToUnsignedLongLong(in, buffer.c_str()); +} + +int InStream::readInteger() { + if (!strict && seekEof()) + quit(_unexpected_eof, "Unexpected end of file - int32 expected"); + + readWordTo(_tmpReadToken); + + long long value = stringToLongLong(*this, _tmpReadToken); + if (value < INT_MIN || value > INT_MAX) + quit(_pe, ("Expected int32, but \"" + __testlib_part(_tmpReadToken) + "\" found").c_str()); + + return int(value); +} + +long long InStream::readLong() { + if (!strict && seekEof()) + quit(_unexpected_eof, "Unexpected end of file - int64 expected"); + + readWordTo(_tmpReadToken); + + return stringToLongLong(*this, _tmpReadToken); +} + +unsigned long long InStream::readUnsignedLong() { + if (!strict && seekEof()) + quit(_unexpected_eof, "Unexpected end of file - int64 expected"); + + readWordTo(_tmpReadToken); + + return stringToUnsignedLongLong(*this, _tmpReadToken); +} + +long long InStream::readLong(long long minv, long long maxv, const std::string &variableName) { + long long result = readLong(); + + if (result < minv || result > maxv) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, ("Integer " + vtos(result) + " violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + + "]").c_str()); + else + quit(_wa, ("Integer parameter [name=" + std::string(variableName) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + } else { + if (variableName.empty()) + quit(_wa, ("Integer element [index=" + vtos(readManyIteration) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + else + quit(_wa, + ("Integer element " + std::string(variableName) + "[" + vtos(readManyIteration) + "] equals to " + + vtos(result) + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + } + } + + if (strict && !variableName.empty()) { + validator.addBoundsHit(variableName, ValidatorBoundsHit(minv == result, maxv == result)); + validator.adjustConstantBounds(variableName, minv, maxv); + validator.addVariable(variableName); + } + + return result; +} + +std::vector +InStream::readLongs(int size, long long minv, long long maxv, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readLongs, readLong(minv, maxv, variablesName), long long, true) +} + +std::vector InStream::readLongs(int size, int indexBase) { + __testlib_readMany(readLongs, readLong(), long long, true) +} + +unsigned long long +InStream::readUnsignedLong(unsigned long long minv, unsigned long long maxv, const std::string &variableName) { + unsigned long long result = readUnsignedLong(); + + if (result < minv || result > maxv) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, + ("Unsigned integer " + vtos(result) + " violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + + "]").c_str()); + else + quit(_wa, + ("Unsigned integer parameter [name=" + std::string(variableName) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + } else { + if (variableName.empty()) + quit(_wa, + ("Unsigned integer element [index=" + vtos(readManyIteration) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + else + quit(_wa, ("Unsigned integer element " + std::string(variableName) + "[" + vtos(readManyIteration) + + "] equals to " + vtos(result) + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + + "]").c_str()); + } + } + + if (strict && !variableName.empty()) { + validator.addBoundsHit(variableName, ValidatorBoundsHit(minv == result, maxv == result)); + validator.adjustConstantBounds(variableName, minv, maxv); + validator.addVariable(variableName); + } + + return result; +} + +std::vector InStream::readUnsignedLongs(int size, unsigned long long minv, unsigned long long maxv, + const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readUnsignedLongs, readUnsignedLong(minv, maxv, variablesName), unsigned long long, true) +} + +std::vector InStream::readUnsignedLongs(int size, int indexBase) { + __testlib_readMany(readUnsignedLongs, readUnsignedLong(), unsigned long long, true) +} + +unsigned long long +InStream::readLong(unsigned long long minv, unsigned long long maxv, const std::string &variableName) { + return readUnsignedLong(minv, maxv, variableName); +} + +int InStream::readInt() { + return readInteger(); +} + +int InStream::readInt(int minv, int maxv, const std::string &variableName) { + int result = readInt(); + + if (result < minv || result > maxv) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, ("Integer " + vtos(result) + " violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + + "]").c_str()); + else + quit(_wa, ("Integer parameter [name=" + std::string(variableName) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + } else { + if (variableName.empty()) + quit(_wa, ("Integer element [index=" + vtos(readManyIteration) + "] equals to " + vtos(result) + + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + else + quit(_wa, + ("Integer element " + std::string(variableName) + "[" + vtos(readManyIteration) + "] equals to " + + vtos(result) + ", violates the range [" + toHumanReadableString(minv) + ", " + toHumanReadableString(maxv) + "]").c_str()); + } + } + + if (strict && !variableName.empty()) { + validator.addBoundsHit(variableName, ValidatorBoundsHit(minv == result, maxv == result)); + validator.adjustConstantBounds(variableName, minv, maxv); + validator.addVariable(variableName); + } + + return result; +} + +int InStream::readInteger(int minv, int maxv, const std::string &variableName) { + return readInt(minv, maxv, variableName); +} + +std::vector InStream::readInts(int size, int minv, int maxv, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readInts, readInt(minv, maxv, variablesName), int, true) +} + +std::vector InStream::readInts(int size, int indexBase) { + __testlib_readMany(readInts, readInt(), int, true) +} + +std::vector InStream::readIntegers(int size, int minv, int maxv, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readIntegers, readInt(minv, maxv, variablesName), int, true) +} + +std::vector InStream::readIntegers(int size, int indexBase) { + __testlib_readMany(readIntegers, readInt(), int, true) +} + +double InStream::readReal() { + if (!strict && seekEof()) + quit(_unexpected_eof, "Unexpected end of file - double expected"); + + return stringToDouble(*this, readWord()); +} + +double InStream::readDouble() { + return readReal(); +} + +double InStream::readReal(double minv, double maxv, const std::string &variableName) { + double result = readReal(); + + if (result < minv || result > maxv) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, ("Double " + vtos(result) + " violates the range [" + vtos(minv) + ", " + vtos(maxv) + + "]").c_str()); + else + quit(_wa, ("Double parameter [name=" + std::string(variableName) + "] equals to " + vtos(result) + + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + "]").c_str()); + } else { + if (variableName.empty()) + quit(_wa, ("Double element [index=" + vtos(readManyIteration) + "] equals to " + vtos(result) + + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + "]").c_str()); + else + quit(_wa, + ("Double element " + std::string(variableName) + "[" + vtos(readManyIteration) + "] equals to " + + vtos(result) + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + "]").c_str()); + } + } + + if (strict && !variableName.empty()) { + validator.addBoundsHit(variableName, ValidatorBoundsHit( + doubleDelta(minv, result) < ValidatorBoundsHit::EPS, + doubleDelta(maxv, result) < ValidatorBoundsHit::EPS + )); + validator.adjustConstantBounds(variableName, minv, maxv); + validator.addVariable(variableName); + } + + return result; +} + +std::vector +InStream::readReals(int size, double minv, double maxv, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readReals, readReal(minv, maxv, variablesName), double, true) +} + +std::vector InStream::readReals(int size, int indexBase) { + __testlib_readMany(readReals, readReal(), double, true) +} + +double InStream::readDouble(double minv, double maxv, const std::string &variableName) { + return readReal(minv, maxv, variableName); +} + +std::vector +InStream::readDoubles(int size, double minv, double maxv, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readDoubles, readDouble(minv, maxv, variablesName), double, true) +} + +std::vector InStream::readDoubles(int size, int indexBase) { + __testlib_readMany(readDoubles, readDouble(), double, true) +} + +double InStream::readStrictReal(double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variableName) { + if (!strict && seekEof()) + quit(_unexpected_eof, "Unexpected end of file - strict double expected"); + + double result = stringToStrictDouble(*this, readWord(), minAfterPointDigitCount, maxAfterPointDigitCount); + + if (result < minv || result > maxv) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, ("Strict double " + vtos(result) + " violates the range [" + vtos(minv) + ", " + vtos(maxv) + + "]").c_str()); + else + quit(_wa, + ("Strict double parameter [name=" + std::string(variableName) + "] equals to " + vtos(result) + + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + "]").c_str()); + } else { + if (variableName.empty()) + quit(_wa, ("Strict double element [index=" + vtos(readManyIteration) + "] equals to " + vtos(result) + + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + "]").c_str()); + else + quit(_wa, ("Strict double element " + std::string(variableName) + "[" + vtos(readManyIteration) + + "] equals to " + vtos(result) + ", violates the range [" + vtos(minv) + ", " + vtos(maxv) + + "]").c_str()); + } + } + + if (strict && !variableName.empty()) { + validator.addBoundsHit(variableName, ValidatorBoundsHit( + doubleDelta(minv, result) < ValidatorBoundsHit::EPS, + doubleDelta(maxv, result) < ValidatorBoundsHit::EPS + )); + validator.adjustConstantBounds(variableName, minv, maxv); + validator.addVariable(variableName); + } + + return result; +} + +std::vector InStream::readStrictReals(int size, double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readStrictReals, + readStrictReal(minv, maxv, minAfterPointDigitCount, maxAfterPointDigitCount, variablesName), + double, true) +} + +double InStream::readStrictDouble(double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variableName) { + return readStrictReal(minv, maxv, + minAfterPointDigitCount, maxAfterPointDigitCount, + variableName); +} + +std::vector InStream::readStrictDoubles(int size, double minv, double maxv, + int minAfterPointDigitCount, int maxAfterPointDigitCount, + const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readStrictDoubles, + readStrictDouble(minv, maxv, minAfterPointDigitCount, maxAfterPointDigitCount, variablesName), + double, true) +} + +bool InStream::eof() { + if (!strict && NULL == reader) + return true; + + return reader->eof(); +} + +bool InStream::seekEof() { + if (!strict && NULL == reader) + return true; + skipBlanks(); + return eof(); +} + +bool InStream::eoln() { + if (!strict && NULL == reader) + return true; + + int c = reader->nextChar(); + + if (!strict) { + if (c == EOFC) + return true; + + if (c == CR) { + c = reader->nextChar(); + + if (c != LF) { + reader->unreadChar(c); + reader->unreadChar(CR); + return false; + } else + return true; + } + + if (c == LF) + return true; + + reader->unreadChar(c); + return false; + } else { + bool returnCr = false; + +#if (defined(ON_WINDOWS) && !defined(FOR_LINUX)) || defined(FOR_WINDOWS) + if (c != CR) { + reader->unreadChar(c); + return false; + } else { + if (!returnCr) + returnCr = true; + c = reader->nextChar(); + } +#endif + if (c != LF) { + reader->unreadChar(c); + if (returnCr) + reader->unreadChar(CR); + return false; + } + + return true; + } +} + +void InStream::readEoln() { + lastLine = reader->getLine(); + if (!eoln()) + quit(_pe, "Expected EOLN"); +} + +void InStream::readEof() { + lastLine = reader->getLine(); + if (!eof()) + quit(_pe, "Expected EOF"); + + if (TestlibFinalizeGuard::alive && this == &inf) + testlibFinalizeGuard.readEofCount++; +} + +bool InStream::seekEoln() { + if (!strict && NULL == reader) + return true; + + int cur; + do { + cur = reader->nextChar(); + } while (cur == SPACE || cur == TAB); + + reader->unreadChar(cur); + return eoln(); +} + +void InStream::nextLine() { + readLine(); +} + +void InStream::readStringTo(std::string &result) { + if (NULL == reader) + quit(_pe, "Expected line"); + + result.clear(); + + for (;;) { + int cur = reader->curChar(); + + if (cur == LF || cur == EOFC) + break; + + if (cur == CR) { + cur = reader->nextChar(); + if (reader->curChar() == LF) { + reader->unreadChar(cur); + break; + } + } + + lastLine = reader->getLine(); + result += char(reader->nextChar()); + } + + if (strict) + readEoln(); + else + eoln(); +} + +std::string InStream::readString() { + readStringTo(_tmpReadToken); + return _tmpReadToken; +} + +std::vector InStream::readStrings(int size, int indexBase) { + __testlib_readMany(readStrings, readString(), std::string, false) +} + +void InStream::readStringTo(std::string &result, const pattern &p, const std::string &variableName) { + readStringTo(result); + if (!p.matches(result)) { + if (readManyIteration == NO_INDEX) { + if (variableName.empty()) + quit(_wa, ("Line \"" + __testlib_part(result) + "\" doesn't correspond to pattern \"" + p.src() + + "\"").c_str()); + else + quit(_wa, ("Line [name=" + variableName + "] equals to \"" + __testlib_part(result) + + "\", doesn't correspond to pattern \"" + p.src() + "\"").c_str()); + } else { + if (variableName.empty()) + quit(_wa, + ("Line element [index=" + vtos(readManyIteration) + "] equals to \"" + __testlib_part(result) + + "\" doesn't correspond to pattern \"" + p.src() + "\"").c_str()); + else + quit(_wa, + ("Line element " + std::string(variableName) + "[" + vtos(readManyIteration) + "] equals to \"" + + __testlib_part(result) + "\", doesn't correspond to pattern \"" + p.src() + "\"").c_str()); + } + } + if (strict && !variableName.empty()) + validator.addVariable(variableName); +} + +void InStream::readStringTo(std::string &result, const std::string &ptrn, const std::string &variableName) { + readStringTo(result, pattern(ptrn), variableName); +} + +std::string InStream::readString(const pattern &p, const std::string &variableName) { + readStringTo(_tmpReadToken, p, variableName); + return _tmpReadToken; +} + +std::vector +InStream::readStrings(int size, const pattern &p, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readStrings, readString(p, variablesName), std::string, false) +} + +std::string InStream::readString(const std::string &ptrn, const std::string &variableName) { + readStringTo(_tmpReadToken, ptrn, variableName); + return _tmpReadToken; +} + +std::vector +InStream::readStrings(int size, const std::string &ptrn, const std::string &variablesName, int indexBase) { + pattern p(ptrn); + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readStrings, readString(p, variablesName), std::string, false) +} + +void InStream::readLineTo(std::string &result) { + readStringTo(result); +} + +std::string InStream::readLine() { + return readString(); +} + +std::vector InStream::readLines(int size, int indexBase) { + __testlib_readMany(readLines, readString(), std::string, false) +} + +void InStream::readLineTo(std::string &result, const pattern &p, const std::string &variableName) { + readStringTo(result, p, variableName); +} + +void InStream::readLineTo(std::string &result, const std::string &ptrn, const std::string &variableName) { + readStringTo(result, ptrn, variableName); +} + +std::string InStream::readLine(const pattern &p, const std::string &variableName) { + return readString(p, variableName); +} + +std::vector +InStream::readLines(int size, const pattern &p, const std::string &variablesName, int indexBase) { + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readLines, readString(p, variablesName), std::string, false) +} + +std::string InStream::readLine(const std::string &ptrn, const std::string &variableName) { + return readString(ptrn, variableName); +} + +std::vector +InStream::readLines(int size, const std::string &ptrn, const std::string &variablesName, int indexBase) { + pattern p(ptrn); + if (strict && !variablesName.empty()) + validator.addVariable(variablesName); + __testlib_readMany(readLines, readString(p, variablesName), std::string, false) +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 3, 4))) +#endif +void InStream::ensuref(bool cond, const char *format, ...) { + if (!cond) { + FMT_TO_RESULT(format, format, message); + this->__testlib_ensure(cond, message); + } +} + +void InStream::__testlib_ensure(bool cond, std::string message) { + if (!cond) + this->quit(_wa, message.c_str()); +} + +void InStream::close() { + if (NULL != reader) { + reader->close(); + delete reader; + reader = NULL; + } + + opened = false; +} + +NORETURN void quit(TResult result, const std::string &msg) { + ouf.quit(result, msg.c_str()); +} + +NORETURN void quit(TResult result, const char *msg) { + ouf.quit(result, msg); +} + +double __testlib_preparePoints(double points_) { + volatile double points = points_; + if (__testlib_isNaN(points)) + quit(_fail, "Parameter 'points' can't be nan"); + if (__testlib_isInfinite(points)) + quit(_fail, "Parameter 'points' can't be infinite"); + if (points < -1E-8) + quit(_fail, "Parameter 'points' can't be negative"); + if (points <= 0.0) + points = +0.0; + if (points > 1E6 + 1E-8) + quit(_fail, "Parameter 'points' can't be greater than 1E6"); + if (points >= 1E6) + points = 1E6; + return points; +} + +NORETURN void __testlib_quitp(double points, const char *message) { + __testlib_points = __testlib_preparePoints(points); + std::string stringPoints = removeDoubleTrailingZeroes(testlib_format_("%.10f", __testlib_points)); + + std::string quitMessage; + if (NULL == message || 0 == strlen(message)) + quitMessage = stringPoints; + else + quitMessage = stringPoints + " " + message; + + quit(_points, quitMessage.c_str()); +} + +NORETURN void __testlib_quitp(int points, const char *message) { + __testlib_points = __testlib_preparePoints(points); + std::string stringPoints = testlib_format_("%d", points); + + std::string quitMessage; + if (NULL == message || 0 == strlen(message)) + quitMessage = stringPoints; + else + quitMessage = stringPoints + " " + message; + + quit(_points, quitMessage.c_str()); +} + +NORETURN void quitp(float points, const std::string &message = "") { + __testlib_quitp(double(points), message.c_str()); +} + +NORETURN void quitp(double points, const std::string &message = "") { + __testlib_quitp(points, message.c_str()); +} + +NORETURN void quitp(long double points, const std::string &message = "") { + __testlib_quitp(double(points), message.c_str()); +} + +NORETURN void quitp(int points, const std::string &message = "") { + __testlib_quitp(points, message.c_str()); +} + +NORETURN void quitpi(const std::string &points_info, const std::string &message = "") { + if (points_info.find(' ') != std::string::npos) + quit(_fail, "Parameter 'points_info' can't contain spaces"); + if (message.empty()) + quit(_points, ("points_info=" + points_info).c_str()); + else + quit(_points, ("points_info=" + points_info + " " + message).c_str()); +} + +template +#ifdef __GNUC__ +__attribute__ ((format (printf, 2, 3))) +#endif +NORETURN void quitp(F points, const char *format, ...) { + FMT_TO_RESULT(format, format, message); + quitp(points, message); +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 2, 3))) +#endif +NORETURN void quitf(TResult result, const char *format, ...) { + FMT_TO_RESULT(format, format, message); + quit(result, message); +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 3, 4))) +#endif +void quitif(bool condition, TResult result, const char *format, ...) { + if (condition) { + FMT_TO_RESULT(format, format, message); + quit(result, message); + } +} + +NORETURN void __testlib_help() { + InStream::textColor(InStream::LightCyan); + std::fprintf(stderr, "TESTLIB %s, https://github.com/MikeMirzayanov/testlib/ ", VERSION); + std::fprintf(stderr, "by Mike Mirzayanov, copyright(c) 2005-2020\n"); + std::fprintf(stderr, "Checker name: \"%s\"\n", checkerName.c_str()); + InStream::textColor(InStream::LightGray); + + std::fprintf(stderr, "\n"); + std::fprintf(stderr, "Latest features: \n"); + for (size_t i = 0; i < sizeof(latestFeatures) / sizeof(char *); i++) { + std::fprintf(stderr, "*) %s\n", latestFeatures[i]); + } + std::fprintf(stderr, "\n"); + + std::fprintf(stderr, "Program must be run with the following arguments: \n"); + std::fprintf(stderr, " [--testset testset] [--group group] [ [<-appes>]]\n\n"); + + __testlib_exitCode = FAIL_EXIT_CODE; + std::exit(FAIL_EXIT_CODE); +} + +static void __testlib_ensuresPreconditions() { + // testlib assumes: sizeof(int) = 4. + __TESTLIB_STATIC_ASSERT(sizeof(int) == 4); + + // testlib assumes: INT_MAX == 2147483647. + __TESTLIB_STATIC_ASSERT(INT_MAX == 2147483647); + + // testlib assumes: sizeof(long long) = 8. + __TESTLIB_STATIC_ASSERT(sizeof(long long) == 8); + + // testlib assumes: sizeof(double) = 8. + __TESTLIB_STATIC_ASSERT(sizeof(double) == 8); + + // testlib assumes: no -ffast-math. + if (!__testlib_isNaN(+__testlib_nan())) + quit(_fail, "Function __testlib_isNaN is not working correctly: possible reason is '-ffast-math'"); + if (!__testlib_isNaN(-__testlib_nan())) + quit(_fail, "Function __testlib_isNaN is not working correctly: possible reason is '-ffast-math'"); +} + +std::string __testlib_testset; + +std::string getTestset() { + return __testlib_testset; +} + +std::string __testlib_group; + +std::string getGroup() { + return __testlib_group; +} + +static void __testlib_set_testset_and_group(int argc, char* argv[]) { + for (int i = 1; i < argc; i++) { + if (!strcmp("--testset", argv[i])) { + if (i + 1 < argc && strlen(argv[i + 1]) > 0) + __testlib_testset = argv[++i]; + else + quit(_fail, std::string("Expected non-empty testset after --testset command line parameter")); + } else if (!strcmp("--group", argv[i])) { + if (i + 1 < argc) + __testlib_group = argv[++i]; + else + quit(_fail, std::string("Expected group after --group command line parameter")); + } + } +} + +void registerGen(int argc, char *argv[], int randomGeneratorVersion) { + if (randomGeneratorVersion < 0 || randomGeneratorVersion > 1) + quitf(_fail, "Random generator version is expected to be 0 or 1."); + random_t::version = randomGeneratorVersion; + + __testlib_ensuresPreconditions(); + TestlibFinalizeGuard::registered = true; + + testlibMode = _generator; + __testlib_set_binary(stdin); + rnd.setSeed(argc, argv); + +#if __cplusplus > 199711L || defined(_MSC_VER) + prepareOpts(argc, argv); +#endif +} + +#ifdef USE_RND_AS_BEFORE_087 +void registerGen(int argc, char* argv[]) +{ + registerGen(argc, argv, 0); +} +#else +#ifdef __GNUC__ +#if (__GNUC__ > 4) || ((__GNUC__ == 4) && (__GNUC_MINOR__ > 4)) +__attribute__ ((deprecated("Use registerGen(argc, argv, 0) or registerGen(argc, argv, 1)." +" The third parameter stands for the random generator version." +" If you are trying to compile old generator use macro -DUSE_RND_AS_BEFORE_087 or registerGen(argc, argv, 0)." +" Version 1 has been released on Spring, 2013. Use it to write new generators."))) +#else +__attribute__ ((deprecated)) +#endif +#endif +#ifdef _MSC_VER +__declspec(deprecated("Use registerGen(argc, argv, 0) or registerGen(argc, argv, 1)." + " The third parameter stands for the random generator version." + " If you are trying to compile old generator use macro -DUSE_RND_AS_BEFORE_087 or registerGen(argc, argv, 0)." + " Version 1 has been released on Spring, 2013. Use it to write new generators.")) +#endif +void registerGen(int argc, char *argv[]) { + std::fprintf(stderr, "Use registerGen(argc, argv, 0) or registerGen(argc, argv, 1)." + " The third parameter stands for the random generator version." + " If you are trying to compile old generator use macro -DUSE_RND_AS_BEFORE_087 or registerGen(argc, argv, 0)." + " Version 1 has been released on Spring, 2013. Use it to write new generators.\n\n"); + registerGen(argc, argv, 0); +} +#endif + +void setAppesModeEncoding(std::string appesModeEncoding) { + static const char* const ENCODINGS[] = {"ascii", "utf-7", "utf-8", "utf-16", "utf-16le", "utf-16be", "utf-32", "utf-32le", "utf-32be", "iso-8859-1", +"iso-8859-2", "iso-8859-3", "iso-8859-4", "iso-8859-5", "iso-8859-6", "iso-8859-7", "iso-8859-8", "iso-8859-9", "iso-8859-10", "iso-8859-11", +"iso-8859-13", "iso-8859-14", "iso-8859-15", "iso-8859-16", "windows-1250", "windows-1251", "windows-1252", "windows-1253", "windows-1254", "windows-1255", +"windows-1256", "windows-1257", "windows-1258", "gb2312", "gbk", "gb18030", "big5", "shift-jis", "euc-jp", "euc-kr", +"euc-cn", "euc-tw", "koi8-r", "koi8-u", "tis-620", "ibm437", "ibm850", "ibm852", "ibm855", "ibm857", +"ibm860", "ibm861", "ibm862", "ibm863", "ibm865", "ibm866", "ibm869", "macroman", "maccentraleurope", "maciceland", +"maccroatian", "macromania", "maccyrillic", "macukraine", "macgreek", "macturkish", "machebrew", "macarabic", "macthai", "hz-gb-2312", +"iso-2022-jp", "iso-2022-kr", "iso-2022-cn", "armscii-8", "tscii", "iscii", "viscii", "geostd8", "cp949", "cp874", +"cp1006", "cp775", "cp858", "cp737", "cp853", "cp856", "cp922", "cp1046", "cp1125", "cp1131", +"ptcp154", "koi8-t", "koi8-ru", "mulelao-1", "cp1133", "iso-ir-166", "tcvn", "iso-ir-14", "iso-ir-87", "iso-ir-159"}; + + appesModeEncoding = lowerCase(appesModeEncoding); + bool valid = false; + for (size_t i = 0; i < sizeof(ENCODINGS) / sizeof(ENCODINGS[0]); i++) + if (appesModeEncoding == ENCODINGS[i]) { + valid = true; + break; + } + if (!valid) + quit(_fail, "Unexpected encoding for setAppesModeEncoding(encoding)"); + ::appesModeEncoding = appesModeEncoding; +} + +void registerInteraction(int argc, char *argv[]) { + __testlib_ensuresPreconditions(); + __testlib_set_testset_and_group(argc, argv); + TestlibFinalizeGuard::registered = true; + + testlibMode = _interactor; + __testlib_set_binary(stdin); + + if (argc > 1 && !strcmp("--help", argv[1])) + __testlib_help(); + + if (argc < 3 || argc > 6) { + quit(_fail, std::string("Program must be run with the following arguments: ") + + std::string(" [ [ [<-appes>]]]") + + "\nUse \"--help\" to get help information"); + } + + if (argc <= 4) { + resultName = ""; + appesMode = false; + } + +#ifndef EJUDGE + if (argc == 5) { + resultName = argv[4]; + appesMode = false; + } + + if (argc == 6) { + if (strcmp("-APPES", argv[5]) && strcmp("-appes", argv[5])) { + quit(_fail, std::string("Program must be run with the following arguments: ") + + " [ [<-appes>]]"); + } else { + resultName = argv[4]; + appesMode = true; + } + } +#endif + + inf.init(argv[1], _input); + + tout.open(argv[2], std::ios_base::out); + if (tout.fail() || !tout.is_open()) + quit(_fail, std::string("Can not write to the test-output-file '") + argv[2] + std::string("'")); + + ouf.init(stdin, _output); + + if (argc >= 4) + ans.init(argv[3], _answer); + else + ans.name = "unopened answer stream"; +} + +void registerValidation() { + __testlib_ensuresPreconditions(); + TestlibFinalizeGuard::registered = true; + + testlibMode = _validator; + + __testlib_set_binary(stdin); + __testlib_set_binary(stdout); + __testlib_set_binary(stderr); + + inf.init(stdin, _input); + inf.strict = true; +} + +void registerValidation(int argc, char *argv[]) { + registerValidation(); + __testlib_set_testset_and_group(argc, argv); + + validator.initialize(); + TestlibFinalizeGuard::registered = true; + + std::string comment = "Validator must be run with the following arguments:" + " [--testset testset]" + " [--group group]" + " [--testOverviewLogFileName fileName]" + " [--testMarkupFileName fileName]" + " [--testCase testCase]" + " [--testCaseFileName fileName]" + ; + + for (int i = 1; i < argc; i++) { + if (!strcmp("--testset", argv[i])) { + if (i + 1 < argc && strlen(argv[i + 1]) > 0) + validator.setTestset(argv[++i]); + else + quit(_fail, comment); + } + if (!strcmp("--group", argv[i])) { + if (i + 1 < argc) + validator.setGroup(argv[++i]); + else + quit(_fail, comment); + } + if (!strcmp("--testOverviewLogFileName", argv[i])) { + if (i + 1 < argc) + validator.setTestOverviewLogFileName(argv[++i]); + else + quit(_fail, comment); + } + if (!strcmp("--testMarkupFileName", argv[i])) { + if (i + 1 < argc) + validator.setTestMarkupFileName(argv[++i]); + else + quit(_fail, comment); + } + if (!strcmp("--testCase", argv[i])) { + if (i + 1 < argc) { + long long testCase = stringToLongLong(inf, argv[++i]); + if (testCase < 1 || testCase >= __TESTLIB_MAX_TEST_CASE) + quit(_fail, testlib_format_("Argument testCase should be between 1 and %d, but ", __TESTLIB_MAX_TEST_CASE) + + toString(testCase) + " found"); + validator.setTestCase(int(testCase)); + } else + quit(_fail, comment); + } + if (!strcmp("--testCaseFileName", argv[i])) { + if (i + 1 < argc) { + validator.setTestCaseFileName(argv[++i]); + } else + quit(_fail, comment); + } + } +} + +void addFeature(const std::string &feature) { + if (testlibMode != _validator) + quit(_fail, "Features are supported in validators only."); + validator.addFeature(feature); +} + +void feature(const std::string &feature) { + if (testlibMode != _validator) + quit(_fail, "Features are supported in validators only."); + validator.feature(feature); +} + +class Checker { +private: + bool _initialized; + std::string _testset; + std::string _group; + +public: + Checker() : _initialized(false), _testset("tests"), _group() { + } + + void initialize() { + _initialized = true; + } + + std::string testset() const { + if (!_initialized) + __testlib_fail("Checker should be initialized with registerTestlibCmd(argc, argv) instead of registerTestlibCmd() to support checker.testset()"); + return _testset; + } + + std::string group() const { + if (!_initialized) + __testlib_fail("Checker should be initialized with registerTestlibCmd(argc, argv) instead of registerTestlibCmd() to support checker.group()"); + return _group; + } + + void setTestset(const char *const testset) { + _testset = testset; + } + + void setGroup(const char *const group) { + _group = group; + } +} checker; + +void registerTestlibCmd(int argc, char *argv[]) { + __testlib_ensuresPreconditions(); + __testlib_set_testset_and_group(argc, argv); + TestlibFinalizeGuard::registered = true; + + testlibMode = _checker; + __testlib_set_binary(stdin); + + std::vector args(1, argv[0]); + checker.initialize(); + + for (int i = 1; i < argc; i++) { + if (!strcmp("--testset", argv[i])) { + if (i + 1 < argc && strlen(argv[i + 1]) > 0) + checker.setTestset(argv[++i]); + else + quit(_fail, std::string("Expected testset after --testset command line parameter")); + } else if (!strcmp("--group", argv[i])) { + if (i + 1 < argc) + checker.setGroup(argv[++i]); + else + quit(_fail, std::string("Expected group after --group command line parameter")); + } else + args.push_back(argv[i]); + } + + argc = int(args.size()); + if (argc > 1 && "--help" == args[1]) + __testlib_help(); + + if (argc < 4 || argc > 6) { + quit(_fail, std::string("Program must be run with the following arguments: ") + + std::string("[--testset testset] [--group group] [ [<-appes>]]") + + "\nUse \"--help\" to get help information"); + } + + if (argc == 4) { + resultName = ""; + appesMode = false; + } + +#ifndef EJUDGE + if (argc == 5) { + resultName = args[4]; + appesMode = false; + } + + if (argc == 6) { + if ("-APPES" != args[5] && "-appes" != args[5]) { + quit(_fail, std::string("Program must be run with the following arguments: ") + + " [ [<-appes>]]"); + } else { + resultName = args[4]; + appesMode = true; + } + } +#endif + + inf.init(args[1], _input); + ouf.init(args[2], _output); + ouf.skipBom(); + ans.init(args[3], _answer); +} + +void registerTestlib(int argc, ...) { + if (argc < 3 || argc > 5) + quit(_fail, std::string("Program must be run with the following arguments: ") + + " [ [<-appes>]]"); + + char **argv = new char *[argc + 1]; + + va_list ap; + va_start(ap, argc); + argv[0] = NULL; + for (int i = 0; i < argc; i++) { + argv[i + 1] = va_arg(ap, char*); + } + va_end(ap); + + registerTestlibCmd(argc + 1, argv); + delete[] argv; +} + +static inline void __testlib_ensure(bool cond, const std::string &msg) { + if (!cond) + quit(_fail, msg.c_str()); +} + +#ifdef __GNUC__ +__attribute__((unused)) +#endif +static inline void __testlib_ensure(bool cond, const char *msg) { + if (!cond) + quit(_fail, msg); +} + +#define ensure(cond) __testlib_ensure((cond), "Condition failed: \"" #cond "\"") +#define STRINGIZE_DETAIL(x) (#x) +#define STRINGIZE(x) STRINGIZE_DETAIL((x)) +#define ensure_ext(cond) __testlib_ensure((cond), "Line " STRINGIZE(__LINE__) ": Condition failed: \"" #cond "\"") + +#ifdef __GNUC__ +__attribute__ ((format (printf, 2, 3))) +#endif +inline void ensuref(bool cond, const char *format, ...) { + if (!cond) { + FMT_TO_RESULT(format, format, message); + __testlib_ensure(cond, message); + } +} + +NORETURN static void __testlib_fail(const std::string &message) { + quitf(_fail, "%s", message.c_str()); +} + +#ifdef __GNUC__ +__attribute__ ((format (printf, 1, 2))) +#endif +void setName(const char *format, ...) { + FMT_TO_RESULT(format, format, name); + checkerName = name; +} + +/* + * Do not use random_shuffle, because it will produce different result + * for different C++ compilers. + * + * This implementation uses testlib random_t to produce random numbers, so + * it is stable. + */ +template +void shuffle(_RandomAccessIter __first, _RandomAccessIter __last) { + if (__first == __last) return; + for (_RandomAccessIter __i = __first + 1; __i != __last; ++__i) + std::iter_swap(__i, __first + rnd.next(int(__i - __first) + 1)); +} + + +template +#if defined(__GNUC__) && !defined(__clang__) +__attribute__ ((error("Don't use random_shuffle(), use shuffle() instead"))) +#endif +void random_shuffle(_RandomAccessIter, _RandomAccessIter) { + quitf(_fail, "Don't use random_shuffle(), use shuffle() instead"); +} + +#ifdef __GLIBC__ +# define RAND_THROW_STATEMENT throw() +#else +# define RAND_THROW_STATEMENT +#endif + +#if defined(__GNUC__) && !defined(__clang__) + +__attribute__ ((error("Don't use rand(), use rnd.next() instead"))) +#endif +#ifdef _MSC_VER +# pragma warning( disable : 4273 ) +#endif +int rand() RAND_THROW_STATEMENT +{ + quitf(_fail, "Don't use rand(), use rnd.next() instead"); + + /* This line never runs. */ + //throw "Don't use rand(), use rnd.next() instead"; +} + +#if defined(__GNUC__) && !defined(__clang__) + +__attribute__ ((error("Don't use srand(), you should use " +"'registerGen(argc, argv, 1);' to initialize generator seed " +"by hash code of the command line params. The third parameter " +"is randomGeneratorVersion (currently the latest is 1)."))) +#endif +#ifdef _MSC_VER +# pragma warning( disable : 4273 ) +#endif +void srand(unsigned int seed) RAND_THROW_STATEMENT +{ + quitf(_fail, "Don't use srand(), you should use " + "'registerGen(argc, argv, 1);' to initialize generator seed " + "by hash code of the command line params. The third parameter " + "is randomGeneratorVersion (currently the latest is 1) [ignored seed=%u].", seed); +} + +void startTest(int test) { + const std::string testFileName = vtos(test); + if (NULL == testlib_freopen_(testFileName.c_str(), "wt", stdout)) + __testlib_fail("Unable to write file '" + testFileName + "'"); +} + +inline std::string compress(const std::string &s) { + return __testlib_part(s); +} + +inline std::string englishEnding(int x) { + x %= 100; + if (x / 10 == 1) + return "th"; + if (x % 10 == 1) + return "st"; + if (x % 10 == 2) + return "nd"; + if (x % 10 == 3) + return "rd"; + return "th"; +} + +template +std::string join(_ForwardIterator first, _ForwardIterator last, _Separator separator) { + std::stringstream ss; + bool repeated = false; + for (_ForwardIterator i = first; i != last; i++) { + if (repeated) + ss << separator; + else + repeated = true; + ss << *i; + } + return ss.str(); +} + +template +std::string join(_ForwardIterator first, _ForwardIterator last) { + return join(first, last, ' '); +} + +template +std::string join(const _Collection &collection, _Separator separator) { + return join(collection.begin(), collection.end(), separator); +} + +template +std::string join(const _Collection &collection) { + return join(collection, ' '); +} + +/** + * Splits string s by character separator returning exactly k+1 items, + * where k is the number of separator occurrences. + */ +std::vector split(const std::string &s, char separator) { + std::vector result; + std::string item; + for (size_t i = 0; i < s.length(); i++) + if (s[i] == separator) { + result.push_back(item); + item = ""; + } else + item += s[i]; + result.push_back(item); + return result; +} + +/** + * Splits string s by character separators returning exactly k+1 items, + * where k is the number of separator occurrences. + */ +std::vector split(const std::string &s, const std::string &separators) { + if (separators.empty()) + return std::vector(1, s); + + std::vector isSeparator(256); + for (size_t i = 0; i < separators.size(); i++) + isSeparator[(unsigned char) (separators[i])] = true; + + std::vector result; + std::string item; + for (size_t i = 0; i < s.length(); i++) + if (isSeparator[(unsigned char) (s[i])]) { + result.push_back(item); + item = ""; + } else + item += s[i]; + result.push_back(item); + return result; +} + +/** + * Splits string s by character separator returning non-empty items. + */ +std::vector tokenize(const std::string &s, char separator) { + std::vector result; + std::string item; + for (size_t i = 0; i < s.length(); i++) + if (s[i] == separator) { + if (!item.empty()) + result.push_back(item); + item = ""; + } else + item += s[i]; + if (!item.empty()) + result.push_back(item); + return result; +} + +/** + * Splits string s by character separators returning non-empty items. + */ +std::vector tokenize(const std::string &s, const std::string &separators) { + if (separators.empty()) + return std::vector(1, s); + + std::vector isSeparator(256); + for (size_t i = 0; i < separators.size(); i++) + isSeparator[(unsigned char) (separators[i])] = true; + + std::vector result; + std::string item; + for (size_t i = 0; i < s.length(); i++) + if (isSeparator[(unsigned char) (s[i])]) { + if (!item.empty()) + result.push_back(item); + item = ""; + } else + item += s[i]; + + if (!item.empty()) + result.push_back(item); + + return result; +} + +NORETURN void __testlib_expectedButFound(TResult result, std::string expected, std::string found, const char *prepend) { + std::string message; + if (strlen(prepend) != 0) + message = testlib_format_("%s: expected '%s', but found '%s'", + compress(prepend).c_str(), compress(expected).c_str(), compress(found).c_str()); + else + message = testlib_format_("expected '%s', but found '%s'", + compress(expected).c_str(), compress(found).c_str()); + quit(result, message); +} + +NORETURN void __testlib_expectedButFound(TResult result, double expected, double found, const char *prepend) { + std::string expectedString = removeDoubleTrailingZeroes(testlib_format_("%.12f", expected)); + std::string foundString = removeDoubleTrailingZeroes(testlib_format_("%.12f", found)); + __testlib_expectedButFound(result, expectedString, foundString, prepend); +} + +template +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void expectedButFound(TResult result, T expected, T found, const char *prependFormat = "", ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + std::string expectedString = vtos(expected); + std::string foundString = vtos(found); + __testlib_expectedButFound(result, expectedString, foundString, prepend.c_str()); +} + +template<> +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void +expectedButFound(TResult result, std::string expected, std::string found, const char *prependFormat, ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + __testlib_expectedButFound(result, expected, found, prepend.c_str()); +} + +template<> +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void expectedButFound(TResult result, double expected, double found, const char *prependFormat, ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + std::string expectedString = removeDoubleTrailingZeroes(testlib_format_("%.12f", expected)); + std::string foundString = removeDoubleTrailingZeroes(testlib_format_("%.12f", found)); + __testlib_expectedButFound(result, expectedString, foundString, prepend.c_str()); +} + +template<> +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void +expectedButFound(TResult result, const char *expected, const char *found, const char *prependFormat, + ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + __testlib_expectedButFound(result, std::string(expected), std::string(found), prepend.c_str()); +} + +template<> +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void expectedButFound(TResult result, float expected, float found, const char *prependFormat, ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + __testlib_expectedButFound(result, double(expected), double(found), prepend.c_str()); +} + +template<> +#ifdef __GNUC__ +__attribute__ ((format (printf, 4, 5))) +#endif +NORETURN void +expectedButFound(TResult result, long double expected, long double found, const char *prependFormat, ...) { + FMT_TO_RESULT(prependFormat, prependFormat, prepend); + __testlib_expectedButFound(result, double(expected), double(found), prepend.c_str()); +} + +#if __cplusplus > 199711L || defined(_MSC_VER) +template +struct is_iterable { + template + static char test(typename U::iterator *x); + + template + static long test(U *x); + + static const bool value = sizeof(test(0)) == 1; +}; + +template +struct __testlib_enable_if { +}; + +template +struct __testlib_enable_if { + typedef T type; +}; + +template +typename __testlib_enable_if::value, void>::type __testlib_print_one(const T &t) { + std::cout << t; +} + +template +typename __testlib_enable_if::value, void>::type __testlib_print_one(const T &t) { + bool first = true; + for (typename T::const_iterator i = t.begin(); i != t.end(); i++) { + if (first) + first = false; + else + std::cout << " "; + std::cout << *i; + } +} + +template<> +typename __testlib_enable_if::value, void>::type +__testlib_print_one(const std::string &t) { + std::cout << t; +} + +template +void __println_range(A begin, B end) { + bool first = true; + for (B i = B(begin); i != end; i++) { + if (first) + first = false; + else + std::cout << " "; + __testlib_print_one(*i); + } + std::cout << std::endl; +} + +template +struct is_iterator { + static T makeT(); + + typedef void *twoptrs[2]; + + static twoptrs &test(...); + + template + static typename R::iterator_category *test(R); + + template + static void *test(R *); + + static const bool value = sizeof(test(makeT())) == sizeof(void *); +}; + +template +struct is_iterator::value>::type> { + static const bool value = false; +}; + +template +typename __testlib_enable_if::value, void>::type println(const A &a, const B &b) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << std::endl; +} + +template +typename __testlib_enable_if::value, void>::type println(const A &a, const B &b) { + __println_range(a, b); +} + +template +void println(const A *a, const A *b) { + __println_range(a, b); +} + +template<> +void println(const char *a, const char *b) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << std::endl; +} + +template +void println(const T &x) { + __testlib_print_one(x); + std::cout << std::endl; +} + +template +void println(const A &a, const B &b, const C &c) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << " "; + __testlib_print_one(c); + std::cout << std::endl; +} + +template +void println(const A &a, const B &b, const C &c, const D &d) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << " "; + __testlib_print_one(c); + std::cout << " "; + __testlib_print_one(d); + std::cout << std::endl; +} + +template +void println(const A &a, const B &b, const C &c, const D &d, const E &e) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << " "; + __testlib_print_one(c); + std::cout << " "; + __testlib_print_one(d); + std::cout << " "; + __testlib_print_one(e); + std::cout << std::endl; +} + +template +void println(const A &a, const B &b, const C &c, const D &d, const E &e, const F &f) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << " "; + __testlib_print_one(c); + std::cout << " "; + __testlib_print_one(d); + std::cout << " "; + __testlib_print_one(e); + std::cout << " "; + __testlib_print_one(f); + std::cout << std::endl; +} + +template +void println(const A &a, const B &b, const C &c, const D &d, const E &e, const F &f, const G &g) { + __testlib_print_one(a); + std::cout << " "; + __testlib_print_one(b); + std::cout << " "; + __testlib_print_one(c); + std::cout << " "; + __testlib_print_one(d); + std::cout << " "; + __testlib_print_one(e); + std::cout << " "; + __testlib_print_one(f); + std::cout << " "; + __testlib_print_one(g); + std::cout << std::endl; +} + +/* opts */ + +/** + * A struct for a singular testlib opt, containing the raw string value, + * and a boolean value for marking whether the opt is used. + */ +struct TestlibOpt { + std::string value; + bool used; + + TestlibOpt() : value(), used(false) {} +}; + +/** + * Get the type of opt based on the number of `-` at the beginning and the + * _validity_ of the key name. + * + * A valid key name must start with an alphabetical character. + * + * Returns: 1 if s has one `-` at the beginning, that is, "-keyName". + * 2 if s has two `-` at the beginning, that is, "--keyName". + * 0 otherwise. That is, if s has no `-` at the beginning, or has more + * than 2 at the beginning ("---keyName", "----keyName", ...), or the + * keyName is invalid (the first character is not an alphabetical + * character). + */ +size_t getOptType(char *s) { + if (!s || strlen(s) <= 1) + return 0; + + if (s[0] == '-') { + if (isalpha(s[1])) + return 1; + else if (s[1] == '-') + return isalpha(s[2]) ? 2 : 0; + } + + return 0; +} + +/** + * Parse the opt at a given index, and put it into the opts maps. + * + * An opt can has the following form: + * 1) -keyName=value or --keyName=value (ex. -n=10 --test-count=20) + * 2) -keyName value or --keyName value (ex. -n 10 --test-count 20) + * 3) -kNumval or --kNumval (ex. -n10 --t20) + * 4) -boolProperty or --boolProperty (ex. -sorted --tree-only) + * + * Only the second form consumes 2 arguments. The other consumes only 1 + * argument. + * + * In the third form, the key is a single character, and after the key is the + * value. The value _should_ be a number. + * + * In the forth form, the value is true. + * + * Params: + * - argc and argv: the number of command line arguments and the command line + * arguments themselves. + * - index: the starting index of the opts. + * - opts: the map containing the resulting opt. + * + * Returns: the number of consumed arguments to parse the opt. + * 0 if there is no arguments to parse. + * + * Algorithm details: + * TODO. Please refer to the implementation to see how the code handles the 3rd and 4th forms separately. + */ +size_t parseOpt(size_t argc, char *argv[], size_t index, std::map &opts) { + if (index >= argc) + return 0; + + size_t type = getOptType(argv[index]), inc = 1; + if (type > 0) { + std::string key(argv[index] + type), val; + size_t sep = key.find('='); + if (sep != std::string::npos) { + val = key.substr(sep + 1); + key = key.substr(0, sep); + } else { + if (index + 1 < argc && getOptType(argv[index + 1]) == 0) { + val = argv[index + 1]; + inc = 2; + } else { + if (key.length() > 1 && isdigit(key[1])) { + val = key.substr(1); + key = key.substr(0, 1); + } else { + val = "true"; + } + } + } + opts[key].value = val; + } else { + return inc; + } + + return inc; +} + +/** + * Global list containing all the arguments in the order given in the command line. + */ +std::vector __testlib_argv; + +/** + * Global dictionary containing all the parsed opts. + */ +std::map __testlib_opts; + +/** + * Whether automatic no unused opts ensurement should be done. This flag will + * be turned on when `has_opt` or `opt(key, default_value)` is called. + * + * The automatic ensurement can be suppressed when + * __testlib_ensureNoUnusedOptsSuppressed is true. + */ +bool __testlib_ensureNoUnusedOptsFlag = false; + +/** + * Suppress no unused opts automatic ensurement. Can be set to true with + * `suppressEnsureNoUnusedOpts()`. + */ +bool __testlib_ensureNoUnusedOptsSuppressed = false; + +/** + * Parse command line arguments into opts. + * The results are stored into __testlib_argv and __testlib_opts. + */ +void prepareOpts(int argc, char *argv[]) { + if (argc <= 0) + __testlib_fail("Opts: expected argc>=0 but found " + toString(argc)); + size_t n = static_cast(argc); // NOLINT(hicpp-use-auto,modernize-use-auto) + __testlib_opts = std::map(); + for (size_t index = 1; index < n; index += parseOpt(n, argv, index, __testlib_opts)); + __testlib_argv = std::vector(n); + for (size_t index = 0; index < n; index++) + __testlib_argv[index] = argv[index]; +} + +/** + * An utility function to get the argument with a given index. This function + * also print a readable message when no arguments are found. + */ +std::string __testlib_indexToArgv(int index) { + if (index < 0 || index >= int(__testlib_argv.size())) + __testlib_fail("Opts: index '" + toString(index) + "' is out of range [0," + + toString(__testlib_argv.size()) + ")"); + return __testlib_argv[size_t(index)]; +} + +/** + * An utility function to get the opt with a given key . This function + * also print a readable message when no opts are found. + */ +std::string __testlib_keyToOpts(const std::string &key) { + auto it = __testlib_opts.find(key); + if (it == __testlib_opts.end()) + __testlib_fail("Opts: unknown key '" + compress(key) + "'"); + it->second.used = true; + return it->second.value; +} + +template +T optValueToIntegral(const std::string &s, bool nonnegative); + +long double optValueToLongDouble(const std::string &s); + +std::string parseExponentialOptValue(const std::string &s) { + size_t pos = std::string::npos; + for (size_t i = 0; i < s.length(); i++) + if (s[i] == 'e' || s[i] == 'E') { + if (pos != std::string::npos) + __testlib_fail("Opts: expected typical exponential notation but '" + compress(s) + "' found"); + pos = i; + } + if (pos == std::string::npos) + return s; + std::string e = s.substr(pos + 1); + if (!e.empty() && e[0] == '+') + e = e.substr(1); + if (e.empty()) + __testlib_fail("Opts: expected typical exponential notation but '" + compress(s) + "' found"); + if (e.length() > 20) + __testlib_fail("Opts: expected typical exponential notation but '" + compress(s) + "' found"); + int ne = optValueToIntegral(e, false); + std::string num = s.substr(0, pos); + if (num.length() > 20) + __testlib_fail("Opts: expected typical exponential notation but '" + compress(s) + "' found"); + if (!num.empty() && num[0] == '+') + num = num.substr(1); + optValueToLongDouble(num); + bool minus = false; + if (num[0] == '-') { + minus = true; + num = num.substr(1); + } + for (int i = 0; i < +ne; i++) { + size_t sep = num.find('.'); + if (sep == std::string::npos) + num += '0'; + else { + if (sep + 1 == num.length()) + num[sep] = '0'; + else + std::swap(num[sep], num[sep + 1]); + } + } + for (int i = 0; i < -ne; i++) { + size_t sep = num.find('.'); + if (sep == std::string::npos) + num.insert(num.begin() + int(num.length()) - 1, '.'); + else { + if (sep == 0) + num.insert(num.begin() + 1, '0'); + else + std::swap(num[sep - 1], num[sep]); + } + } + while (!num.empty() && num[0] == '0') + num = num.substr(1); + while (num.find('.') != std::string::npos && num.back() == '0') + num = num.substr(0, num.length() - 1); + if (!num.empty() && num.back() == '.') + num = num.substr(0, num.length() - 1); + if ((!num.empty() && num[0] == '.') || num.empty()) + num.insert(num.begin(), '0'); + return (minus ? "-" : "") + num; +} + +template +T optValueToIntegral(const std::string &s_, bool nonnegative) { + std::string s(parseExponentialOptValue(s_)); + if (s.empty()) + __testlib_fail("Opts: expected integer but '" + compress(s_) + "' found"); + T value = 0; + long double about = 0.0; + signed char sign = +1; + size_t pos = 0; + if (s[pos] == '-') { + if (nonnegative) + __testlib_fail("Opts: expected non-negative integer but '" + compress(s_) + "' found"); + sign = -1; + pos++; + } + for (size_t i = pos; i < s.length(); i++) { + if (s[i] < '0' || s[i] > '9') + __testlib_fail("Opts: expected integer but '" + compress(s_) + "' found"); + value = T(value * 10 + s[i] - '0'); + about = about * 10 + s[i] - '0'; + } + value *= sign; + about *= sign; + if (fabsl(value - about) > 0.1) + __testlib_fail("Opts: integer overflow: expected integer but '" + compress(s_) + "' found"); + return value; +} + +long double optValueToLongDouble(const std::string &s_) { + std::string s(parseExponentialOptValue(s_)); + if (s.empty()) + __testlib_fail("Opts: expected float number but '" + compress(s_) + "' found"); + long double value = 0.0; + signed char sign = +1; + size_t pos = 0; + if (s[pos] == '-') { + sign = -1; + pos++; + } + bool period = false; + long double mul = 1.0; + for (size_t i = pos; i < s.length(); i++) { + if (s[i] == '.') { + if (period) + __testlib_fail("Opts: expected float number but '" + compress(s_) + "' found"); + else { + period = true; + continue; + } + } + if (period) + mul *= 10.0; + if (s[i] < '0' || s[i] > '9') + __testlib_fail("Opts: expected float number but '" + compress(s_) + "' found"); + if (period) + value += (s[i] - '0') / mul; + else + value = value * 10 + s[i] - '0'; + } + value *= sign; + return value; +} + +/** + * Return true if there is an opt with a given key. + * + * By calling this function, automatic ensurement for no unused opts will be + * done when the program is finalized. Call suppressEnsureNoUnusedOpts() to + * turn it off. + */ +bool has_opt(const std::string &key) { + __testlib_ensureNoUnusedOptsFlag = true; + return __testlib_opts.count(key) != 0; +} + +/* About the following part for opt with 2 and 3 arguments. + * + * To parse the argv/opts correctly for a give type (integer, floating point or + * string), some meta programming must be done to determine the type of + * the type, and use the correct parsing function accordingly. + * + * The pseudo algorithm for determining the type of T and parse it accordingly + * is as follows: + * + * if (T is integral type) { + * if (T is unsigned) { + * parse the argv/opt as an **unsigned integer** of type T. + * } else { + * parse the argv/opt as an **signed integer** of type T. + * } else { + * if (T is floating point type) { + * parse the argv/opt as an **floating point** of type T. + * } else { + * // T should be std::string + * just the raw content of the argv/opts. + * } + * } + * + * To help with meta programming, some `opt` function with 2 or 3 arguments are + * defined. + * + * Opt with 3 arguments: T opt(true/false is_integral, true/false is_unsigned, index/key) + * + * + The first argument is for determining whether the type T is an integral + * type. That is, the result of std::is_integral() should be passed to + * this argument. When false, the type _should_ be either floating point or a + * std::string. + * + * + The second argument is for determining whether the signedness of the type + * T (if it is unsigned or signed). That is, the result of + * std::is_unsigned() should be passed to this argument. This argument can + * be ignored if the first one is false, because it only applies to integer. + * + * Opt with 2 arguments: T opt(true/false is_floating_point, index/key) + * + The first argument is for determining whether the type T is a floating + * point type. That is, the result of std::is_floating_point() should be + * passed to this argument. When false, the type _should_ be a std::string. + */ + +template +T opt(std::false_type is_floating_point, int index); + +template<> +std::string opt(std::false_type /*is_floating_point*/, int index) { + return __testlib_indexToArgv(index); +} + +template +T opt(std::true_type /*is_floating_point*/, int index) { + return T(optValueToLongDouble(__testlib_indexToArgv(index))); +} + +template +T opt(std::false_type /*is_integral*/, U /*is_unsigned*/, int index) { + return opt(std::is_floating_point(), index); +} + +template +T opt(std::true_type /*is_integral*/, std::false_type /*is_unsigned*/, int index) { + return optValueToIntegral(__testlib_indexToArgv(index), false); +} + +template +T opt(std::true_type /*is_integral*/, std::true_type /*is_unsigned*/, int index) { + return optValueToIntegral(__testlib_indexToArgv(index), true); +} + +template<> +bool opt(std::true_type /*is_integral*/, std::true_type /*is_unsigned*/, int index) { + std::string value = __testlib_indexToArgv(index); + if (value == "true" || value == "1") + return true; + if (value == "false" || value == "0") + return false; + __testlib_fail("Opts: opt by index '" + toString(index) + "': expected bool true/false or 0/1 but '" + + compress(value) + "' found"); +} + +/** + * Return the parsed argv by a given index. + */ +template +T opt(int index) { + return opt(std::is_integral(), std::is_unsigned(), index); +} + +/** + * Return the raw string value of an argv by a given index. + */ +std::string opt(int index) { + return opt(index); +} + +/** + * Return the parsed argv by a given index. If the index is bigger than + * the number of argv, return the given default_value. + */ +template +T opt(int index, const T &default_value) { + if (index >= int(__testlib_argv.size())) { + return default_value; + } + return opt(index); +} + +/** + * Return the raw string value of an argv by a given index. If the index is + * bigger than the number of argv, return the given default_value. + */ +std::string opt(int index, const std::string &default_value) { + return opt(index, default_value); +} + +template +T opt(std::false_type is_floating_point, const std::string &key); + +template<> +std::string opt(std::false_type /*is_floating_point*/, const std::string &key) { + return __testlib_keyToOpts(key); +} + +template +T opt(std::true_type /*is_integral*/, const std::string &key) { + return T(optValueToLongDouble(__testlib_keyToOpts(key))); +} + +template +T opt(std::false_type /*is_integral*/, U, const std::string &key) { + return opt(std::is_floating_point(), key); +} + +template +T opt(std::true_type /*is_integral*/, std::false_type /*is_unsigned*/, const std::string &key) { + return optValueToIntegral(__testlib_keyToOpts(key), false); +} + +template +T opt(std::true_type /*is_integral*/, std::true_type /*is_unsigned*/, const std::string &key) { + return optValueToIntegral(__testlib_keyToOpts(key), true); +} + +template<> +bool opt(std::true_type /*is_integral*/, std::true_type /*is_unsigned*/, const std::string &key) { + if (!has_opt(key)) + return false; + std::string value = __testlib_keyToOpts(key); + if (value == "true" || value == "1") + return true; + if (value == "false" || value == "0") + return false; + __testlib_fail("Opts: key '" + compress(key) + "': expected bool true/false or 0/1 but '" + + compress(value) + "' found"); +} + +/** + * Return the parsed opt by a given key. + */ +template +T opt(const std::string &key) { + return opt(std::is_integral(), std::is_unsigned(), key); +} + +/** + * Return the raw string value of an opt by a given key + */ +std::string opt(const std::string &key) { + return opt(key); +} + +/* Scorer started. */ + +enum TestResultVerdict { + SKIPPED, + OK, + WRONG_ANSWER, + RUNTIME_ERROR, + TIME_LIMIT_EXCEEDED, + IDLENESS_LIMIT_EXCEEDED, + MEMORY_LIMIT_EXCEEDED, + COMPILATION_ERROR, + CRASHED, + FAILED +}; + +std::string serializeVerdict(TestResultVerdict verdict) { + switch (verdict) { + case SKIPPED: return "SKIPPED"; + case OK: return "OK"; + case WRONG_ANSWER: return "WRONG_ANSWER"; + case RUNTIME_ERROR: return "RUNTIME_ERROR"; + case TIME_LIMIT_EXCEEDED: return "TIME_LIMIT_EXCEEDED"; + case IDLENESS_LIMIT_EXCEEDED: return "IDLENESS_LIMIT_EXCEEDED"; + case MEMORY_LIMIT_EXCEEDED: return "MEMORY_LIMIT_EXCEEDED"; + case COMPILATION_ERROR: return "COMPILATION_ERROR"; + case CRASHED: return "CRASHED"; + case FAILED: return "FAILED"; + } + throw "Unexpected verdict"; +} + +TestResultVerdict deserializeTestResultVerdict(std::string s) { + if (s == "SKIPPED") + return SKIPPED; + else if (s == "OK") + return OK; + else if (s == "WRONG_ANSWER") + return WRONG_ANSWER; + else if (s == "RUNTIME_ERROR") + return RUNTIME_ERROR; + else if (s == "TIME_LIMIT_EXCEEDED") + return TIME_LIMIT_EXCEEDED; + else if (s == "IDLENESS_LIMIT_EXCEEDED") + return IDLENESS_LIMIT_EXCEEDED; + else if (s == "MEMORY_LIMIT_EXCEEDED") + return MEMORY_LIMIT_EXCEEDED; + else if (s == "COMPILATION_ERROR") + return COMPILATION_ERROR; + else if (s == "CRASHED") + return CRASHED; + else if (s == "FAILED") + return FAILED; + ensuref(false, "Unexpected serialized TestResultVerdict"); + // No return actually. + return FAILED; +} + +struct TestResult { + int testIndex; + std::string testset; + std::string group; + TestResultVerdict verdict; + double points; + long long timeConsumed; + long long memoryConsumed; + std::string input; + std::string output; + std::string answer; + int exitCode; + std::string checkerComment; +}; + +std::string serializePoints(double points) { + if (std::isnan(points)) + return ""; + else { + char c[64]; + snprintf(c, 64, "%.03lf", points); + return c; + } +} + +double deserializePoints(std::string s) { + if (s.empty()) + return std::numeric_limits::quiet_NaN(); + else { + double result; +#ifdef _MSC_VER + ensuref(sscanf_s(s.c_str(), "%lf", &result) == 1, "Invalid serialized points"); +#else + ensuref(std::sscanf(s.c_str(), "%lf", &result) == 1, "Invalid serialized points"); +#endif + return result; + } +} + +std::string escapeTestResultString(std::string s) { + std::string result; + for (size_t i = 0; i < s.length(); i++) { + if (s[i] == '\r') + continue; + if (s[i] == '\n') { + result += "\\n"; + continue; + } + if (s[i] == '\\' || s[i] == ';') + result += '\\'; + result += s[i]; + } + return result; +} + +std::string unescapeTestResultString(std::string s) { + std::string result; + for (size_t i = 0; i < s.length(); i++) { + if (s[i] == '\\' && i + 1 < s.length()) { + if (s[i + 1] == 'n') { + result += '\n'; + i++; + continue; + } else if (s[i + 1] == ';' || s[i + 1] == '\\') { + result += s[i + 1]; + i++; + continue; + } + } + result += s[i]; + } + return result; +} + +std::string serializeTestResult(TestResult tr) { + std::string result; + result += std::to_string(tr.testIndex); + result += ";"; + result += escapeTestResultString(tr.testset); + result += ";"; + result += escapeTestResultString(tr.group); + result += ";"; + result += serializeVerdict(tr.verdict); + result += ";"; + result += serializePoints(tr.points); + result += ";"; + result += std::to_string(tr.timeConsumed); + result += ";"; + result += std::to_string(tr.memoryConsumed); + result += ";"; + result += escapeTestResultString(tr.input); + result += ";"; + result += escapeTestResultString(tr.output); + result += ";"; + result += escapeTestResultString(tr.answer); + result += ";"; + result += std::to_string(tr.exitCode); + result += ";"; + result += escapeTestResultString(tr.checkerComment); + return result; +} + +TestResult deserializeTestResult(std::string s) { + std::vector items; + std::string t; + for (size_t i = 0; i < s.length(); i++) { + if (s[i] == '\\') { + t += s[i]; + if (i + 1 < s.length()) + t += s[i + 1]; + i++; + continue; + } else { + if (s[i] == ';') { + items.push_back(t); + t = ""; + } else + t += s[i]; + } + } + items.push_back(t); + + ensuref(items.size() == 12, "Invalid TestResult serialization: expected exactly 12 items"); + + TestResult tr; + size_t pos = 0; + tr.testIndex = stoi(items[pos++]); + tr.testset = unescapeTestResultString(items[pos++]); + tr.group = unescapeTestResultString(items[pos++]); + tr.verdict = deserializeTestResultVerdict(items[pos++]); + tr.points = deserializePoints(items[pos++]); + tr.timeConsumed = stoll(items[pos++]); + tr.memoryConsumed = stoll(items[pos++]); + tr.input = unescapeTestResultString(items[pos++]); + tr.output = unescapeTestResultString(items[pos++]); + tr.answer = unescapeTestResultString(items[pos++]); + tr.exitCode = stoi(items[pos++]); + tr.checkerComment = unescapeTestResultString(items[pos++]); + + return tr; +} + +std::vector readTestResults(std::string fileName) { + std::ifstream stream; + stream.open(fileName.c_str(), std::ios::in); + ensuref(stream.is_open(), "Can't read test results file '%s'", fileName.c_str()); + std::vector result; + std::string line; + while (getline(stream, line)) + if (!line.empty()) + result.push_back(deserializeTestResult(line)); + stream.close(); + return result; +} + +std::function)> __testlib_scorer; + +struct TestlibScorerGuard { + ~TestlibScorerGuard() { + if (testlibMode == _scorer) { + std::vector testResults; + while (!inf.eof()) { + std::string line = inf.readLine(); + if (!line.empty()) + testResults.push_back(deserializeTestResult(line)); + } + inf.readEof(); + printf("%.3f\n", __testlib_scorer(testResults)); + } + } +} __testlib_scorer_guard; + +void registerScorer(int argc, char *argv[], std::function)> scorer) { + /* Suppress unused. */ + (void)(argc), (void)(argv); + + __testlib_ensuresPreconditions(); + + testlibMode = _scorer; + __testlib_set_binary(stdin); + + inf.init(stdin, _input); + inf.strict = false; + + __testlib_scorer = scorer; +} + +/* Scorer ended. */ + +/** + * Return the parsed opt by a given key. If no opts with the given key are + * found, return the given default_value. + * + * By calling this function, automatic ensurement for no unused opts will be + * done when the program is finalized. Call suppressEnsureNoUnusedOpts() to + * turn it off. + */ +template +T opt(const std::string &key, const T &default_value) { + if (!has_opt(key)) { + return default_value; + } + return opt(key); +} + +/** + * Return the raw string value of an opt by a given key. If no opts with the + * given key are found, return the given default_value. + * + * By calling this function, automatic ensurement for no unused opts will be + * done when the program is finalized. Call suppressEnsureNoUnusedOpts() to + * turn it off. + */ +std::string opt(const std::string &key, const std::string &default_value) { + return opt(key, default_value); +} + +/** + * Check if all opts are used. If not, __testlib_fail is called. + * Should be used after calling all opt() function calls. + * + * This function is useful when opt() with default_value for checking typos + * in the opt's key. + */ +void ensureNoUnusedOpts() { + for (const auto &opt: __testlib_opts) { + if (!opt.second.used) { + __testlib_fail(testlib_format_("Opts: unused key '%s'", compress(opt.first).c_str())); + } + } +} + +void suppressEnsureNoUnusedOpts() { + __testlib_ensureNoUnusedOptsSuppressed = true; +} + +void TestlibFinalizeGuard::autoEnsureNoUnusedOpts() { + if (__testlib_ensureNoUnusedOptsFlag && !__testlib_ensureNoUnusedOptsSuppressed) { + ensureNoUnusedOpts(); + } +} + +TestlibFinalizeGuard testlibFinalizeGuard; +#endif + +#ifdef __GNUC__ +__attribute__ ((format (printf, 1, 2))) +#endif +std::string testlib_format_(const char *fmt, ...) { + FMT_TO_RESULT(fmt, fmt, result); + return result; +} + +std::string testlib_format_(const std::string fmt, ...) { + FMT_TO_RESULT(fmt, fmt.c_str(), result); + return result; +} + +#if (__cplusplus >= 202002L && __has_include()) || __cpp_lib_format +template +std::string format(const char* fmt, Args&&... args) { + size_t size = size_t(std::snprintf(nullptr, 0, fmt, args...) + 1); + std::vector buffer(size); + std::snprintf(buffer.data(), size, fmt, args...); + return std::string(buffer.data()); +} + +template +std::string format(const std::string fmt, Args&&... args) { + size_t size = size_t(std::snprintf(nullptr, 0, fmt.c_str(), args...) + 1); + std::vector buffer(size); + std::snprintf(buffer.data(), size, fmt.c_str(), args...); + return std::string(buffer.data()); +} +#else +#ifdef __GNUC__ +__attribute__ ((format (printf, 1, 2))) +#endif +std::string format(const char *fmt, ...) { + FMT_TO_RESULT(fmt, fmt, result); + return result; +} + +std::string format(const std::string fmt, ...) { + FMT_TO_RESULT(fmt, fmt.c_str(), result); + return result; +} +#endif + +#endif diff --git a/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/validation-and-experiments/SKILL.md b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/validation-and-experiments/SKILL.md new file mode 100644 index 000000000..75a42e696 --- /dev/null +++ b/skills/task-oriented/FrontierCS/algorithmic-problem-solving/sub-skills/validation-and-experiments/SKILL.md @@ -0,0 +1,171 @@ +--- +name: validation-and-experiments +description: >- + Design the smallest falsifying test loop for an algorithmic solver, including + brute oracles, differential and metamorphic tests, champion/challenger + comparisons, paired seeds, holdouts, resource probes, and release gates. Use + as an internal algorithmic-problem-solving recovery route when the smallest + counterexample is unclear, comparisons are noisy, or a correction needs + independent evidence before promotion. Do not use to implement a doubtful + checker or to diagnose a known compiler or runtime-engineering failure. +--- + +# Validation and Experiments + +Use the smallest test loop that can falsify the current algorithmic hypothesis. Do not begin with a generic benchmark platform or a large campaign. + +## Ownership and handoffs + +This skill owns oracle choice, counterexample search, controlled candidate comparison, stochastic evidence, and promotion/release gates. Use: + +- [checker and local evaluation](../checker-and-local-evaluation/SKILL.md) to implement or repair an independent checker, scorer, interactor, simulator, or process harness; +- [model and route algorithms](../model-and-route-algorithms/SKILL.md) when evidence contradicts the contract, state, reduction, proof, or route class; +- [contest solver engineering](../contest-solver-engineering/SKILL.md) for compiler, overflow, memory-layout, I/O, or measured deadline failures; +- the shared [heuristic search](../../references/heuristic-search.md) for representation, moves, deltas, reachability, and optimizer mechanics. + +Do not start a local generated-data campaign merely because a candidate is non-full or because a local harness could be built. If accepted online submissions provide stable legality/protocol evidence and coherent scores, defer local construction while the unresolved question is algorithmic quality. Start local testing when a named hypothesis needs a counterexample, official feedback cannot isolate the failure, paired evidence is needed for promotion, a rare/resource boundary must be probed, or Checker and Local Evaluation has been evidence-triggered. + +When generated inputs are justified, this skill chooses the coverage families, oracle relationships, development/holdout split, and comparison design. Execute them through [Checker and Local Evaluation's end-to-end data workflow](../checker-and-local-evaluation/SKILL.md#build-problem-finding-data-end-to-end): parameterized generator code, independent input validation, solver/evaluator execution, and a reproducible seed/failure manifest. Pair with [testlib C++ judging](../testlib-cpp-judging/SKILL.md) when using Testlib generators or validators. Randomized, batch, and maximum-scale inputs must be generated by code rather than maintained as hand-edited data. + +## 1. Validation ladder + +Build evidence in this order: + +1. target-language compile with the official flags; +2. samples and hand-computed examples; +3. empty/minimal, maximum, equality, duplicate, and overflow boundaries; +4. one case for each hard constraint and failure mode; +5. brute or independent oracle on tiny instances; +6. randomized or metamorphic tests; +7. representative runtime and memory probes; +8. official evaluator or legal remote feedback. + +Stop and fix the first failing layer before running a larger campaign. + +## 2. Exact and constructive problems + +For an exact solver, prefer a tiny obviously correct oracle over a second optimized implementation. Generate cases small enough for full enumeration and compare the complete answer or a canonical witness. + +Check: + +- empty/nonempty distinctions and impossible cases; +- equality, duplicates, disconnected components, and symmetric solutions; +- reconstruction independently from the optimum value; +- overflow before narrowing and exact division/modulo semantics; +- dense and sparse extremes; +- the first size where an optimization changes behavior. + +For constructive output, validate syntax, hard constraints, and final serialization independently. A promised-feasible input does not prove the construction reaches a valid witness. + +When a mismatch appears, minimize it and identify the first violated model or proof premise before changing downstream implementation details. + +## 3. Scored solvers + +Always keep: + +``` +fallback: simplest guaranteed-valid emergency output +challenger/current: mutable experimental candidate +champion: best independently validated legal artifact +``` + +Before using a routed local campaign to tune quality or promote a challenger: + +- validate every output; +- recompute the true raw objective; +- compare incremental deltas with full recomputation; +- test apply/undo round trips; +- confirm score direction and any clamp, threshold, or normalization; +- verify the internal deadline includes a safety margin. + +Use Checker and Local Evaluation when the official checker/scorer is absent or doubtful and that uncertainty blocks the current decision. Do not compare solver quality under an evaluator whose contract is still contradicted by official or hand-computed evidence. + +## 4. Minimal champion/challenger loop + +Record one line per material attempt: + +``` +champion | hypothesis | one change | fixed cases/seeds +validity | per-case score/runtime delta | keep/revert | next action +``` + +Use a direct shell command or a short task-specific script. Default to: + +- a few representative development cases; +- the same small set of solver seeds for champion and challenger; +- one untouched boundary or holdout set; +- saved failing inputs and outputs; +- the full generator parameter tuple, instance seed, validator result, and input hash for every generated case; +- per-case results, not only an aggregate. + +Promote only when the challenger stays legal, fits the budget with margin, and improves the relevant fixed evidence. Revert the challenger otherwise. + +## 5. Randomness and holdouts + +Keep randomness sources distinct: + +``` +instance seed | solver seed | judge/noise seed | host/runtime noise +``` + +Pair champion and challenger on the same inputs and allowed noise seeds. Tune on development cases, then check an untouched set. Do not repeatedly select against one public set or one lucky solver seed. + +Three to five fixed seeds are usually enough to reject a clearly weak idea. Expand only when results are close, variance is material, or remote scoring is noisy. + +## 6. Metamorphic tests + +When an exact oracle is hard, transform an instance in a way with a known effect: + +- permute labels or input order when semantics are invariant; +- add an isolated or dominated object with predictable behavior; +- scale or translate geometry when the contract preserves it; +- duplicate independent components and combine their objectives; +- reverse, rotate, or complement only when the contract has that symmetry. + +Verify the expected relationship, not necessarily a fixed output string. + +## 7. Interactive and reactive evaluator validation + +Only when Checker and Local Evaluation has been evidence-triggered or a Plateau Escape phase-local gate is active, use it to build and self-test the required process-level simulator, interactor, reactive episode replayer, scorer, real-pipe runner, or transcript contract. That sub-skill owns evaluator implementation, the C++ implementation policy, adversarial fixtures, and the route to concrete `testlib.h` APIs. If accepted official interactions are stable and coherent, defer these local artifacts rather than entering Checker through this section alone. + +This skill owns how those evaluator artifacts enter the broader experiment loop. Pin their build/run commands and evaluator version, compare policies or strategies on the same allowed hidden states and noise seeds, retain failing transcripts, and report protocol failure separately from legality, reward, runtime, and strategy quality. Do not promote a solver merely because it performs well under one uncorroborated local simulator. + +## 8. When heavier evidence is justified + +Use larger seed sets, quantiles, confidence intervals, or formal benchmark automation only when: + +- stochastic variance can reverse the decision; +- candidates differ by a small margin; +- failures have expensive lower-tail risk; +- the work is a long-running solver project rather than one problem; +- a formal report requires reproducible aggregate statistics. + +Even then, begin with paired per-case deltas and explicit invalid/timeout counts. Do not let statistical machinery hide contract or checker errors. + +## 9. Failure-directed next action + +| Symptom | Next route or check | +|---|---| +| Invalid output | Parser, bounds, reconstruction, checker assumptions | +| Exact mismatch | Model-and-route: smallest brute counterexample and proof premise | +| TLE/MLE | Contest-solver-engineering: measured hot path, state count, copying, allocation | +| Legal but very low score | Objective direction, external reference/bound gap, representation, construction | +| Local search plateau | Delta correctness, reachability, compound moves | +| Seed-sensitive gain | Per-instance deltas, outliers, overfit | +| Local/remote disagreement | Checker contract and score transform | + +If the current route appears structurally weak but the admission criteria in [plateau escape](../plateau-escape/SKILL.md) are not yet met, test one falsifiable structural challenger. Once they are met, use that sub-skill before choosing another model, representation, or algorithm family. Do not answer every failure with another parameter or bypass the plateau gate by expanding the test campaign. + +## 10. Release gate + +Before delivery: + +- rebuild the champion cleanly; +- rerun samples and saved regressions; +- rerun the smallest independent oracle/checker suite already justified or available; do not build a new local campaign solely for release when coherent accepted official evidence leaves no evaluator-facing uncertainty; +- validate the final output or transcript; +- probe worst or representative runtime and memory; +- confirm deterministic seeds and release flags; +- ensure the delivered artifact is the champion, not the latest edit. + +Return the falsified or supported hypothesis, exact commands and fixtures used, per-case evidence, promotion/rejection decision, and remaining material uncertainty.