Skip to content

Gemma 2 annotation field rename: regenerate the SDK (migration, not a hotfix) - #44

Open
ppavlidis wants to merge 2 commits into
masterfrom
gemma2-annotation-field-rename
Open

Gemma 2 annotation field rename: regenerate the SDK (migration, not a hotfix)#44
ppavlidis wants to merge 2 commits into
masterfrom
gemma2-annotation-field-rename

Conversation

@ppavlidis

Copy link
Copy Markdown

Nothing released is broken — this is migration work, not a hotfix

Released gemmapy targets Gemma 1.0. The default host is
https://gemma.msl.ubc.ca/rest/v2/, the 1.32.x deployment, and that host still
serves the old field names. I fetched /rest/v2/openapi.yaml from both hosts
while preparing this: gemma.msl.ubc.ca still declares className / classUri
/ termName / termUri. So there is no urgency here and nothing on PyPI needs
a rescue release.

What changed on the server

Gemma 2 (gemma2.msl.ubc.ca, build b5c6747f68) renamed four fields on the
annotation value object:

Gemma 1.0 Gemma 2
className category
classUri categoryUri
termName value
termUri valueUri

That affects GET/PUT /datasets/{id}/annotations, GET /datasets/annotations,
the sample-characteristics routes, and the experiment-tag add/remove responses.
GET /datasets/categories renames its own className / classUri to
category / categoryUri the same way.

Second change, shipped with it: on a factor-value row the term field used to
hold a composed sentence; it is now the term's own label. Plain tags, subset tags
and sample characteristics are unchanged. Same dataset, same row, both hosts:

// gemma.msl.ubc.ca  /rest/v2/datasets/1/annotations
{"className": "disease", "termName": "acute  disease related to solid organ transplantation",
 "objectClass": "FactorValue"}

// gemma2.msl.ubc.ca /rest/v2/datasets/1/annotations
{"category": "disease", "value": "disease related to solid organ transplantation",
 "objectClass": "FactorValue",
 "predicate": "has phenotype", "predicateUri": ".../RO_0002200",
 "object": "acute",            "objectUri": ".../PATO_0000389"}

The "acute" qualifier did not disappear — it moved into new structured
predicate / object fields. See "What this PR does not do" below.

⚠️ This is a breaking change for gemmapy users — Python attributes are renamed

Regenerating the SDK renames the Python attribute names, not just the wire
keys:

before after
AnnotationValueObject.class_name .category
AnnotationValueObject.class_uri .category_uri
AnnotationValueObject.term_name .value
AnnotationValueObject.term_uri .value_uri

Same four on AnnotationWithUsageStatisticsValueObject;
CategoryWithUsageStatisticsValueObject gets .category / .category_uri.
Any user code reading those attributes off the raw SDK breaks.

The high-level DataFrame API is unaffected: get_dataset_annotations still
returns the same six columns under the same names (class_name, class_URI,
term_name, term_URI, object_class, evidence_code). The resulting
attribute-vs-column mismatch inside _processors.process_annotations is
deliberate and is commented as such.

Whether gemmapy absorbs the attribute rename now, ships aliases for a release, or
waits is your call — I have not invented a compatibility mechanism.

The question this PR is really putting to you

Does gemmapy want to target Gemma 2 yet, and if so, one server or both?

As it stands this branch is a hard switch. The costs, as far as I can see them:

  • Hard switch (this branch as-is). Simplest, and the SDK stays 100% generated
    output. But anyone pointed at Gemma 1.0 — which is the default host, so that is
    everyone by default — silently gets four empty columns. It does not raise; the
    deserializer just finds no matching keys. Demonstrated below.
  • Support both spellings. Nobody breaks, but it means hand-written code
    living in gemmapy/sdk/, which sdk-gen/generate-sdk.sh overwrites wholesale
    on the next regeneration, or a translation layer above the SDK.
  • Wait. Do nothing until gemmapy's default host moves to Gemma 2, and land
    this then. Costs nothing today; the branch is here when you want it.

I have deliberately not picked one.

Regenerated, not hand-edited — and how I know

gemmapy/sdk/** is genuine codegen output, not three hand-patched attribute maps.
I did not take that on trust; I re-ran the codegen
(swagger-codegen-cli 3.0.73, -t sdk-gen/templ, per generate-sdk.sh) against
the openapi.yaml on this branch and diffed the result against the working tree.
All 593 files matched byte for byte, with one exception: api_client.py's
user-agent line, which differs only because generate-sdk.sh extracts the version
with a sed that leaves the quotes in (gemmapy/"2.0.7"). That line is unchanged
from master, so it is pre-existing and not something this branch introduces.

What was hand-edited is sdk-gen/openapi.yaml, and only the ten lines that
spell those field names. It is still the Gemma 1.0 spec, not Gemma 2's spec
re-downloaded — I diffed master's copy against the live
gemma.msl.ubc.ca spec and they agree except for 36 lines of property/enum
ordering noise.

🛑 A trap worth knowing about: running sdk-gen/generate-sdk.sh unmodified
will curl the spec back from gemma.msl.ubc.ca and silently undo these renames,
because that host is Gemma 1.0. This is noted in the first commit message.

What I verified

  • pytest: 11 passed on this branch, 11 passed on master. Python 3.10.6,
    pytest 9.0.3, against the live default host (Gemma 1.0). No change in outcome.

  • tox: not run — not installed on this machine, and only 3.10 of the
    3.10, 3.11, 3.12, 3.13 matrix is available locally. tox.ini only runs
    pytest, and CI covers the matrix.

  • The suite passing does not mean the change is exercised. The tests hit a
    live Gemma, and test_get_dataset_annotations asserts only the column count
    (res.shape[1] == 6), so it stays green while the contents go empty. Actual
    get_dataset_annotations(1) output on this branch:

    host class_name class_URI term_name term_URI
    gemma.msl.ubc.ca (default) 0/3 non-null 0/3 0/3 0/3
    gemma2.msl.ubc.ca 3/3 3/3 3/3 3/3

    CI will be green on this PR for the same reason — the workflow runs bare
    pytest against the default host. Please do not read that green tick as
    evidence the rename works.

    I have left the test alone: making it assert non-empty data would turn it red
    against Gemma 1.0, and which server it should be asserting against is exactly
    the question above.

  • Confirmed no stragglers: zero occurrences of class_name / class_uri /
    term_name / term_uri remain anywhere in gemmapy/sdk/, and none of the old
    spellings remain in sdk-gen/openapi.yaml. The two surviving term_name
    references in _processors.py and gemmapy_api.py belong to process_GO and
    are a different value object.

  • process_search_annotations needed no change — AnnotationSearchResultValueObject
    already used category / value on Gemma 1.0.

What this PR does not do

  • It does not model the nine fields Gemma 2 adds to the annotation value object
    (predicate, predicateUri, object, objectUri, secondPredicate,
    secondPredicateUri, secondObject, secondObjectUri, supportingEvidence).
    The spec on this branch has none of them, so the SDK drops them. Concretely,
    the acute qualifier shown in the payloads above is not reachable from
    gemmapy on this branch — against Gemma 2 the term_name column loses it with
    no warning. That is the part of the term-field change that is a real data
    loss, and it wants a decision rather than a quiet fix from me.
  • It does not switch the default host, add aliases, or add a compatibility shim.
  • It does not touch gemma2_compat on gemma.R, which is being handled separately.

Files changed

sdk-gen/openapi.yaml                                                 (hand-edited, 10 renames)
gemmapy/sdk/models/annotation_value_object.py                        (regenerated)
gemmapy/sdk/models/annotation_with_usage_statistics_value_object.py  (regenerated)
gemmapy/sdk/models/category_with_usage_statistics_value_object.py    (regenerated)
gemmapy/_processors.py                                               (hand-edited)
gemmapy/gemmapy_api.py                                               (docstring only)

🤖 Generated with Claude Code

https://claude.ai/code/session_01P2yKU9FAQmF6T99v23vWQt

Gemma 2's REST API renames four fields on the annotation value objects:
className -> category, classUri -> categoryUri, termName -> value,
termUri -> valueUri. CategoryWithUsageStatisticsValueObject renames its
className/classUri the same way.

sdk-gen/openapi.yaml here is the Gemma 1.0 spec with those ten lines
renamed in place, not Gemma 2's spec re-downloaded; gemmapy/sdk was then
regenerated from it with the sdk-gen codegen, so the three model files
are generated output rather than hand edits. Note that running
sdk-gen/generate-sdk.sh unmodified will curl the spec back from
gemma.msl.ubc.ca and undo the renames, because that host is Gemma 1.0.

Renaming the wire fields renames the Python attributes with them:
AnnotationValueObject.class_name is now .category, .term_name is now
.value, and likewise for the URI fields.
process_annotations now reads category / category_uri / value /
value_uri off the regenerated AnnotationValueObject. The DataFrame
column names (class_name, class_URI, term_name, term_URI) are left
alone, so get_dataset_annotations still returns a six-column frame.

The get_dataset_annotations docstring records that on a factor-value row
the term field is now the term's label rather than a composed sentence
such as "wild type genotype has background APP/PS1".
@ppavlidis
ppavlidis requested a review from oganm August 31, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant