Repository navigation
Integrate Judgement feedback tools and strict PR checks - #3310
Merged
Merged
Conversation
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Roomote uses Judgement's shared CLI for local checks, example capture, and calibration report comparison. Findings include a bounded changed-line preview and the rule cutoff; longer checks report progress. The inference adapter continues to use Roomote model settings and keeps caching disabled while backend configuration can change.
pnpm judgement capture --rule <id> --path <file> --name <name> --expected pass|violationpreviews a fixture from the index. Saving requires--output <new-file>and refuses to overwrite existing files.pnpm judgement compare --before <report.json> --after <report.json>identifies per-example regressions on matching fixtures without inference.JudgementPR status: violations, incomplete evidence, timeouts, and inference failures fail the check. The workflow collects snapshot objects in a read-only job without inference credentials. A separate job installs the pinned checker and imports the objects into an empty repository without checking out PR files. Git configuration, hooks, and repository history are excluded. Status writes run in separate jobs; the API credential is available only to the inference step.Rollout
Pins published
@roo-code/judgement@0.4.0in the root, worker, cloud-agents, and CI workflow. The lockfile uses the registry integrity hash, and tester presets are regenerated against the installed release.The
TYPESAFE_API_KEYrepository secret is configured; its presence has been verified, but inference with it has not yet run in this workflow.JUDGEMENT_MODELdefaults tojev-1.13.0and should match the calibrated model. After the workflow lands and a successful run is verified, addJudgementas a required status check on protected branches. The new workflow has not run live yet.Rules, cutoffs, and model questions remain unchanged. No generated calibration reports are committed.
Validation
pnpm judgement:presets --checkpassed; 47 synthetic examples prepared without inference.pnpm lint:fast,pnpm check-types:fast, andpnpm knippassed.