[ZeeshanAdilButt]

What this adds

The capstone AI checker and completion certificates for the Dev Weekends guided-lab tracks (Level 1, 2 and 3). Design: docs/16-capstone-checker-and-certificates.mdx, approved before the build.

Capstone checker

• New CAPSTONE_CHECK block, authored as a ```capstone fence that names a rubric pack (capstone/<name>.yaml) in the course package. The importer validates the pack and stores it in the block's server-only config. Calibration files never ship to learners.
• The learner submits a GitHub repo (and a live URL when the pack asks). The server runs automated checks first: repo public, GitHub Actions green on the latest commit, tags, required files, live site responds, live page contains a snippet. We never run learner code.
• Integrity checks the level sessions asked for:
  • files_match: course-owned files (workflow, supplied tests) must be byte-identical to the template at a tag, so edited tests cannot fake a green build.
  • file_contains: for example a pinned reusable workflow.
  • check_run: one named CI check's result.
  • Any check can be set to stop the run or not.
• A red build, a changed course file or a missing repo stops the run with fix-it steps and no model call. Otherwise one structured-output call to claude-sonnet-5-5 grades the repo snapshot against the rubric. The server recomputes the percent, the pass rules and the automated gates, so text in a repo cannot raise its own grade.
• The learner sees: complete or not, the percent against the pass mark, what is left with numbered steps, the checks, the rubric scores, quality notes and their check history.
• The grader can raise integrity flags, for example text aimed at the grader or copied work. Only mentors see them, and any flag sends the check to the mentor.
• A complete check, a flagged one, or one the learner sends reaches the existing mentor queue. There is one waiting item per learner per capstone. The review page shows the full report with evidence, commit, model, tokens and cost. The form is prefilled from the AI result, so Approve confirms it and any edit overrides it. The AI never passes anyone on its own.
• Cost controls: same commit and pack version reuse the earlier grade for free. There is an input budget per check (the snapshot is trimmed to fit), an output cap, a limit of checks per learner per block per day, and a platform-wide daily cap on AI grades. Tokens and cost are stored on every check.
• Live URL fetches are guarded: HTTPS only, public addresses only (checked inside the socket's DNS lookup, so there is no rebinding window), GET only, 5 s, 512 KB, 3 redirects.

Certificates

• A course opts in with a certificate block in course.yaml (title, code, skills). The course's trainer signs it: the instructor name the catalog shows, overridable per course.
• Eligible when every chapter is complete and a mentor approved every capstone. The learner confirms their name once and gets a public id like DW-L2-7K3F9QXM.
• Public verify page /verify/<id>: valid or withdrawn status, the certificate, skills and capstone links. It also has Add to LinkedIn (add-to-profile URL with the certification fields; LinkedIn lists Edorax as the issuer), Share a post, and Download image. The share image is a 1200 x 627 opengraph-image. Mentors and admins can revoke a certificate, giving a reason.
• Design: Edorax type with the Dev Weekends wordmark and week bars in black and white. White document, hairline border, square corners, no gradients, seals or ribbons. Checked at desktop and phone width.

Rubric packs and calibration

• Loader and schema with readable errors that name the file and field.
• pnpm --dir server capstone:calibrate --pack <rubric.yaml> replays recorded grader outputs for free. With --live --record it runs the real model, prints the cost and saves the outputs. It fails when an example lands outside its tolerance.
• A sample pack with three examples (pass, near miss, red CI) runs in CI. Courses with several packs keep each pack's examples in calibration/<pack name>/.
• The three level sessions have the pack format and are writing their packs in Edorax-core.

Level 3 oral exam

oralExam.enabled exists in the pack schema and is refused until the stage is built; the next phase builds it on the existing AI interview block.

Tests

• Server, 60 new tests:
  • 51 unit tests: schema, gates, integrity checks, recompute, URL guard, snapshot, grader call, calibration, LinkedIn URL, certificate ids.
  • 9 API tests: the block state hides the rubric, red CI stops the run, the AI grade reaches the mentor queue, reuse is free, the daily budget holds, flagged checks reach the mentor with the flags hidden from the learner, plus mentor review data, approval, certificate claim, verify and revoke.
• Full server suite: all files pass. The dev-login tests need a seeded DB, as on main.
• Frontend: 4768 tests pass. Typecheck and lint are clean on both apps. sync-shared --check is clean.
• Checked by hand in a local preview: learner report, mentor review with the prefilled confirm, approval, certificate claim, verify page, share image.

Needs keys or approval before it is live

Item	Who
Nothing for the AI key: the checker uses the ANTHROPIC_API_KEY already set for chapter Q&A and is on whenever it is set (CAPSTONE_CHECK_ENABLED=false turns it off)	None
GITHUB_TOKEN is optional: files are read from GitHub's raw host, so a check uses about five API calls and 60 an hour without a token covers about 12 checks	Optional
CERTIFICATE_LINKEDIN_ORG_ID: Edorax's own LinkedIn page id, once it exists (until then LinkedIn gets "Edorax" as text)	Later
Optional title under the trainer's name (certificate.issuer.title in course.yaml)	Optional
Daily caps: defaults are 3 checks per learner per block and 150 AI grades per day (about $15 worst case)	User
One live calibration run per level pack, about $0.20 per example	Level sessions, with the key

The migration adds a block type, two tables and one nullable column. Nothing changes for existing courses until a course uses the fence.

Known limits

• The desktop app reader does not draw the capstone block yet; the web app does.
• Certificate eligibility does not filter capstones by course variant or release yet.
• Claude's server-side refusal fallback is not enabled. A refused grade asks the learner to send the work to their mentor.
• Requested by the level sessions, not built: live-site screenshots for the grader (needs a headless browser outside the API) and reading CI for a tag's commit.