Skip to content

Add evidence-backed architecture manifests - #11

Open
Alajmah wants to merge 11 commits into
atlas-generated-views-v0.1from
atlas-architecture-manifests-v0.1
Open

Alajmah wants to merge 11 commits into
atlas-generated-views-v0.1from
atlas-architecture-manifests-v0.1

Conversation

@Alajmah

@Alajmah Alajmah commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Summary

This stacked PR adds the first structured architecture manifests to ModelMap, with field-level provenance and repository-level validation.

Architecture schema

  • binds every architecture record to one model_release or model_variant through target
  • adds review status, Gemma-compatible pre/post normalization flags, structured attention fields, and structured feed-forward/MoE fields
  • adds field-level evidence entries (source, narrow locator, exact fields)

Validator hardening

  • resolves architecture targets and evidence sources against canonical IDs
  • requires every populated architecture fact to be covered by valid source evidence
  • rejects evidence that names an absent or null fact
  • validates zero-based inclusive block plans for ordering, bounds, and overlap

Initial manifests

Adds reviewed manifests for:

  • Mistral 7B
  • Gemma 2 9B
  • DeepSeek-V3
  • Llama 3.1 405B

The manifests use primary technical reports and a pinned first-party Meta SKU registry where exact checkpoint configuration matters.

Llama 3.1 405B ambiguity

attention.kv_heads is deliberately absent from the model-variant manifest. Meta's first-party SKU registry exposes n_kv_heads=8 for BF16 MP8/FP8 packaging and n_kv_heads=16 for BF16 MP16 for the same 405B core model. Encoding either value as intrinsic to variant:llama-3.1-405b would overstate the evidence. The manifest records the ambiguity in notes instead of inferring a value.

Tests and documentation

Adds architecture-specific regression tests for schema requirements, reference integrity, complete evidence coverage, invalid evidence paths, block-plan invariants, and the Llama KV-head omission policy. The data and ontology documentation now define architecture targeting, evidence semantics, block ranges, and omission of packaging-dependent values.

Review

The branch was reviewed at HEAD before opening this PR, including schema semantics, validator failure modes, data/source alignment, test coverage, CI integration, and documentation consistency. The existing validation workflow automatically runs the full unittest suite, corpus validator, and generated-view builds on the PR.

Stack

Base: atlas-generated-views-v0.1 (PR #10)
Head: atlas-architecture-manifests-v0.1

@gitwire-hq gitwire-hq Bot added the size/XL label Sep 29, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3dc8b0db6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/validate.py

fact_paths = architecture_fact_paths(record)
covered_fields: set[str] = set()
for evidence in record.get("evidence", []):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard non-list evidence before iterating

When an architecture document contains a schema-invalid value such as evidence: null, schema validation records the expected error but execution then reaches this loop and raises TypeError: 'NoneType' object is not iterable. This prevents the corpus validator from reporting the collected validation errors and makes a malformed architecture record crash validation instead of failing cleanly; check that evidence is a list before iterating it.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant