Skip to content

Give every environment variable one _51DEGREES_ name - #105

Merged
jwrosewell merged 8 commits into
rcfrom
remaining-names-java
Nov 11, 2026
Merged

jwrosewell merged 8 commits into
rcfrom
remaining-names-java

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

The problem

A POSIX shell cannot set an environment variable whose name starts with a
digit, and three of this repository's names did. Measured in bash:

$ export 51DEGREES_IPI_PATH=/data/file.ipi
bash: export: `51DEGREES_IPI_PATH=/data/file.ipi': not a valid identifier
$ 51DEGREES_IPI_PATH=/data/file.ipi mvn test
bash: 51DEGREES_IPI_PATH=/data/file.ipi: command not found

The readmes told a developer to set them.
examples/ip-intelligence/README.md line 31
says to "set the 51DEGREES_IPI_PATH environment variable or system
property", and
examples/device-detection/README.md line 125
says the same for 51DEGREES_DD_PATH. The only ways to set either were
PowerShell, env, or a CI definition, so the instructions could not be
followed in the shell most developers use.

Four more names were read behind a _51DEGREES_ one, so a variable had two
names and the one that answered depended on which was set. RESOURCE_KEY and
LICENSE_KEY were read as the second argument to an env(...) helper in
CreatorContextDemoServer.java lines 122 and 124
and
DidClientLiveTests.java lines 62 and 66,
and as a second read in
cloud-engine/Main.java line 80.

The cloud endpoint was read as FOD_CLOUD_API_URL, in
pipeline.cloudrequestengine/.../Constants.java line 30
and
DidClient.java line 117.

What the specification says

POSIX.1-2024 (IEEE Std 1003.1-2024),
Base Definitions section 8.1, Environment Variable Definition:

Environment variable names used by the utilities in the Shell and Utilities
volume of POSIX.1-2024 consist solely of uppercase letters, digits, and the
<underscore> ('_') from the characters defined in Portable Character Set
and do not begin with a digit. Other characters, and byte sequences that do
not form valid characters, may be permitted by an implementation;
applications shall tolerate the presence of such names.

Base Definitions 3.216, Name,
which is what export and a variable assignment take:

In the shell command language, a word consisting solely of underscores,
digits, and alphabetics from the portable character set. The first character
of a name is not a digit.

So a process may receive a digit-initial name, but a shell can never set one.

The rest is James Rosewell's decision of 10 November 2026, quoted in
51degrees-php#67: "Just
use the version that works everywhere all the time with no fallback." Every
51Degrees environment variable name becomes _51DEGREES_<NAME>, and nothing
is read in its place.

The cloud endpoint takes the name the Rust cloud request engine already reads,
in
cloud-request-engine/src/constants.rs line 33,
so one setting carries one name in every language.

What changed

Eight commits.

f5e2b65
adds the check, before the fix.
ci/check-environment-variable-names.ps1
reads every file git ls-files reports, which leaves out build output and the
submodules, and fails on two kinds of name.

  1. A name that starts with the digit, in any letter case, because Windows
    reads an environment variable name in any case, so 51degrees_ipi_path is
    the same variable there and a POSIX shell can set neither. A name is
    skipped when a name character comes before it, so _51DEGREES_IPI_PATH
    passes.
  2. An old name, being RESOURCE_KEY, LICENSE_KEY or
    FOD_CLOUD_API_URL where a variable is read, set or named.

The second keeps its case, unlike the first. The cloud request engine's tests
name the query string parameter resource_key, which differs from
RESOURCE_KEY only by case, and a query string parameter is on the wire
rather than in the environment, so matching in any case would fail on a name
that is not a variable at all. The comment above the pattern says so, and
assertEquals("resource_key=" + key, query); is one of the lines the tests
require to pass.

The tests in
ci/check-environment-variable-names.Tests.ps1
give the matcher one line at a time, and one case runs the whole check over a
small repository holding a file named Donn + é + es.java, because git
quotes a non-ASCII path unless told not to and a quoted path is not found on
disk. The check lists paths with -c core.quotePath=false for that reason and
prints how many it read and which it skipped, which are the two submodules and
the JavaScript templates directory.

.github/workflows/environment-variable-names.yml
runs the tests and then the check, on every pull request and on pushes to
main and rc, following the pattern of the utm link lint workflow already
here.

e70a841
is the fix, across 24 files.

Old name New name Where
51DEGREES_IPI_PATH _51DEGREES_IPI_PATH ip-intelligence.shared/.../testhelpers/FileUtils.java:46, examples/ip-intelligence/shared/.../DataFileHelper.java:136, CountryTranslationIntegrationTest.java:53,69, OfflineProcessing.java:187, AGENTS.md:128, ip-intelligence/README.md:39, examples/ip-intelligence/README.md:31,64
51DEGREES_DD_PATH _51DEGREES_DD_PATH 51Degrees-OnPrem.xml:31,35, which is a config placeholder rather than a System.getenv call, OnPremDataFileOptionTest.java:38,43, GettingStartedWebOnPrem.java:51, ci/run-integration-tests.ps1:168, examples/device-detection/README.md:125
51DEGREES_RESOURCE_KEY _51DEGREES_RESOURCE_KEY examples/ip-intelligence/shared/.../KeyHelper.java:38, ip-intelligence.shared/.../testhelpers/KeyUtils.java:36, KeyHelperTest.java:91, AGENTS.md:186, two readmes
FOD_CLOUD_API_URL _51DEGREES_CLOUD_ENDPOINT pipeline.cloudrequestengine/.../Constants.java:30, whose constant becomes CLOUD_ENDPOINT_VARIABLE, CloudRequestEngineBuilder.java:90,95, DidClient.java:117, DidClientTests.java:133, cloud-engine/Main.java, CreatorContextDemoServer.java, pipeline/README.md and three other readmes
RESOURCE_KEY, read only when _51DEGREES_RESOURCE_KEY was unset removed CreatorContextDemoServer.java:122,135, DidClientLiveTests.java:62, cloud-engine/Main.java:80, the fodid readme
LICENSE_KEY, read only when _51DEGREES_LICENSE_KEY was unset removed CreatorContextDemoServer.java:124, DidClientLiveTests.java:66, the fodid readme

DidClient.ENDPOINT_VARIABLE says what it is rather than repeating the name,
so only its value moves. The cloud request engine's constant was named after
the variable, so it becomes CLOUD_ENDPOINT_VARIABLE. Anyone referring to
Constants.FOD_CLOUD_API_URL has to use the new name, which is a source level
change to a public constant and is the one thing in this draft a consumer
could notice at compile time.

IP_INTELLIGENCE_KEY in the translation module is an element data key with
the value "ip", not a variable, and resource_key in the cloud request
engine's tests is the query string parameter, so neither moves.

c885993
widens the check after review. The old-name half needed a quote, a dollar
sign, a brace or a getenv call in front of the name, so it could not see one
declared as a key in a workflow env: block or listed at the start of a line
in a readme. A name at the start of a line, past any spaces, tabs and a list
dash, now counts. The cases added require a workflow line whose key is an old
name, a readme list entry and a shell assignment to fail, and a workflow line
whose key is not an old name to pass, so
DeviceDetection: ${{ secrets.DEVICE_DETECTION_KEY }} is left alone because a
secret is read through secrets. and not env..

The braced forms, after review

d82ff33
adds six cases that fail, and
7ebe282
makes them pass.

The prefix set wanted the dollar sign and the env token next to each other,
and its brace branch wanted the name straight after the brace, so a name
written with braces fell between the two and went unseen. Five lines went
past it, with the plain form that already worked at the bottom for contrast:

Line Before After
${env:LICENSE_KEY} = $k missed caught
${Env:RESOURCE_KEY} missed caught
$ENV:RESOURCE_KEY = "key" missed caught
Remove-Item Env:\LICENSE_KEY missed caught
${{ env.FOD_CLOUD_API_URL }} missed caught
$env:LICENSE_KEY = $k caught caught

This matters here more than the shape of the hole suggests, because two of
this repository's own file types write the braced form. The CI scripts are
PowerShell, where a name is braced whenever it sits next to other text, and
the pipeline's own config lookup reads a variable as ${env:NAME}, which
FiftyOneLookupTest
exercises. A check that cannot see that form would let an old name back in
through the files it exists to watch.

The fix matches the env token on its own, which is what the
51degrees-go check already does,
and in any letter case and with a following backslash allowed on top of that,
so $ENV: and the Env: drive are seen as well. Three cases pin that it
stays narrow, being
${env:_51DEGREES_LICENSE_KEY}, the ${env:user.home} line in
FiftyOneLookupTest, and ${{ env.PAID_RESOURCE_KEY }}, which is the
browser suite's own name and not one of ours.

Taking rc

db2f8aa
merges rc, which had moved far enough to conflict. Until it was merged
GitHub could not build a merge commit, so no check ran on this draft at all.

rc has since taken a helper that reads a resource key and a licence key
from several names, in
pipeline.cloudrequestengine/.../LiveCloudKeys.java,
pipeline.did/.../LiveCloudKeys.java
and
cloud-engine/Main.java.
That design is kept whole, including the endpoint following the key it was
issued for. Two entries leave each list, being RESOURCE_KEY and
LICENSE_KEY, because every variable has one name and nothing is read in its
place, which is the decision quoted above and what this draft's check
enforces.

Nothing in CI sets either of the dropped names, so no run loses a key by
it. The 51Did tests are run by
.github/workflows/cloud-51did.yml,
which sets _51DEGREES_RESOURCE_KEY to each key the organisation holds in
turn with _51DEGREES_LICENSE_KEY beside it, both still first in their
lists. The cloud request engine's live tests are not named by any workflow,
so they run with the rest in ci/run-unit-tests.ps1, which passes
-DTestResourceKey and -DSuperResourceKey as system properties, and both
of those names stay in the list. ci/run-integration-tests.ps1 sets
neither old name anywhere.

The merge also moves the endpoint to _51DEGREES_CLOUD_ENDPOINT in the three
files rc had written with the old name, two of them in test documentation
and one an example read.

1db46a9
rewraps one comment in cloud-engine/Main.java that the longer name had
left broken across a phrase. No code in it.

2405c68
is the head, and takes rc again for
51degrees-java#104
and
51degrees-java#108.
The second of those matters to this draft's build rather than to its
change. ci/run-integration-tests.ps1 line 36 ran the developer example
tests with no -DTestResourceKey, so SimpleCloudEngine_Test failed for
want of a key on any branch that did not have the one line #108 adds. The
merge is clean, and both of the check's patterns were run over the merged
tree before pushing, with nothing found.

The build that failed for want of that key also shows the shortened name
list behaving, because SimpleCloudEngine_Test builds its message from
Main.RESOURCE_KEY_NAMES rather than spelling the names out, and the
message it printed was "None of _51DEGREES_RESOURCE_KEY,
_51DEGREES_RESOURCE_KEY_BESPOKE, TestResourceKey, SUPER_RESOURCE_KEY
is set". Four names rather than five, nothing asserts a count, and the four
are the ones a developer can actually set.

The old names are no longer read. Anyone who set 51DEGREES_IPI_PATH,
51DEGREES_DD_PATH, 51DEGREES_RESOURCE_KEY, FOD_CLOUD_API_URL,
RESOURCE_KEY or LICENSE_KEY must set the new name instead. The cloud
request engine and the 51Did client are published packages, so a deployment
that pointed either at a privately hosted copy of the cloud with
FOD_CLOUD_API_URL has to move to _51DEGREES_CLOUD_ENDPOINT.

How it was tested

The braced forms:

  • Red, run 43068667009,
    on db2f8aa, the six cases before the fix. 29 tests pass and 6 fail, each
    reading "Expected 'fallback' to be found in collection $null, but it was not
    found", for ${env:LICENSE_KEY} = $Keys.TestResourceKey,
    ${Env:RESOURCE_KEY}, $ENV:RESOURCE_KEY = "key",
    Remove-Item Env:\LICENSE_KEY, <Endpoint>${env:FOD_CLOUD_API_URL}</Endpoint>
    and url: ${{ env.FOD_CLOUD_API_URL }}.
  • Green, run 43068967766,
    on 7ebe282, which is the fix. 35 tests pass and none fails, and the check
    then reads 1,021 paths, skips the two submodules and the JavaScript
    templates directory, and passes.
  • Green on 1db46a9,
    run 43071096938.
    The same 35 tests pass, and the check reads 1,022 paths and passes. The
    check runs on the merge of this branch with rc, so the one extra path is
    pipeline/pipeline.jsonbuilder/src/test/java/fiftyone/pipeline/jsonbuilder/JsonBuilderSequenceTests.java,
    added to rc by
    51degrees-java#104
    between the two runs, and not a file of this draft.
  • Green on the head 2405c68,
    run 43073692200,
    after rc was taken again. 35 tests pass and the check reads the same
    1,022 paths and passes.

The rename itself, on the earlier heads:

  • Red, run 43009710661,
    on f5e2b65, the check alone before the rename. Its tests pass, 21 of 21, and
    the check then reads 1,004 paths and fails with "22 names start with a
    digit, which a POSIX shell cannot set, so each must start with an
    underscore, and 9 old names are read, set or kept".

  • Green, run 43019544305,
    on c885993. Its tests pass, 26 of 26, and the check reads 1,009 paths and
    passes.

  • Full build on the head 2405c68,
    run 43073755228.
    Passed. All 11 jobs that ran succeeded and 2 were skipped, every
    Setting 'PullRequestSha' line, 10 of them, is
    2405c68e6ff4f3c9db4e5ad5985e127ace76486c, and Configure printed "Left out
    6 macOS entries".

Two earlier full builds failed, and both are named here because each says
something worth knowing.

  • Run 43069200489,
    on 1db46a9, stopped in pipeline.engines on
    DataUpdateServiceTests.DataUpdateService_Register_AutoUpdateConfiguredInterval[0]
    with a Mockito TooManyActualInvocations on
    futureFactory.schedule(<any Runnable>, leq(600000)). That is a timing
    check on a scheduled task, and nothing in this draft reaches it.
  • Run 43071230247,
    on the same commit, ran that test green in every job, so the failure above
    was flaky rather than a regression. It then stopped at "Pipeline developer
    examples" for want of a resource key, which is the fault Give the pipeline developer examples the test resource key #108 fixes and
    which rc did not have when that build started.

Of the tests in the passing build, the one worth naming is
OnPremDataFileOptionTest,
because the data file name in 51Degrees-OnPrem.xml is resolved by the
pipeline's config substitution rather than by System.getenv, so a leading
underscore could in principle have broken the placeholder. That test sets
_51DEGREES_DD_PATH as a system property, which substitution reads the same
way, and asserts the example picks from-dd-path.hash ahead of
TestDataFile and ahead of the Lite file. It passes, so the placeholder
resolves the new name.

ci/run-integration-tests.ps1 line 168 sets the same variable for the
Selenium run of that example, and the live 51Did tests read the resource key
and the endpoint by their one name.

What remains

  • Any deployment or secret that sets only an old name has to move to the
    _51DEGREES_ name before this merges, or it goes unread. CI sets the new
    names itself.
  • The two LiveCloudKeys lists and the one in cloud-engine/Main.java still
    carry TestResourceKey, SUPER_RESOURCE_KEY and TestLicenseKey after the
    two old names were dropped. Those three are mixed case or otherwise outside
    the convention, they came into rc with the live cloud tests rather than
    with this rename, and moving them needs the organisation secrets to move
    with them. The same decision points at them, so they are worth a change that
    can do both.
  • TestResourceKey and SuperResourceKey, read through
    Constants.RESOURCE_KEY_ENV_VAR in
    device-detection.shared/.../testhelpers/Constants.java:54 and
    location/geo-location/src/test/java/fiftyone/geolocation/Constants.java:28,
    are the same two names in the test helpers.
  • The IP Intelligence examples' KeyHelper.getOrSetResourceKey, at
    lines 159 to 173,
    still reads _51DEGREES_RESOURCE_KEY first and then the name it is passed,
    and its error message tells the reader that "the legacy name is also still
    accepted". That is true today rather than stale, because the second name is
    TestResourceKey, which this draft leaves alone for the reason above. It is
    named here because it is the one place left where a second name is both read
    and advertised, and it goes when TestResourceKey goes. The Device
    Detection examples' KeyHelper reads one name already.
  • CLOUD_ROOT_URL, PAID_RESOURCE_KEY, EXAMPLE_URL, EXAMPLE_LANG and
    PORT, set by ci/run-integration-tests.ps1 lines 98 to 147, are left
    alone. They are read by the shared browser test suite in
    51Degrees/selenium-api-tests,
    so renaming them here alone would stop that suite finding them. They move
    when the suite moves, in one change across every language that runs it.
  • The cxx submodule's own C and C++ examples read the IP Intelligence path
    under a digit-initial name.
    51degrees-cxx#147
    renames it there, and this repository takes it with the next cxx update. The
    check skips the submodule, so it does not fail on that.
  • The other 51Degrees languages are being renamed separately.

51degrees-java#100,
which renamed the three digit-initial names and added a JUnit guard instead of
this PowerShell one, is closed, so Java ends with one guard and this draft
carries the whole rename.

macOS was left out of this draft's full build under the 9 November decision.
The rc head is built with macOS before it merges into main.

Reads every tracked file and fails on a name that starts with the digit,
which a POSIX shell cannot set, and on an old name that is still read
where each variable has one name. Pester tests give the check one line at
a time and a workflow runs both on every pull request.

This commit is the check alone, so it fails over the repository as it
stands and names every place the next commit changes.
A POSIX shell cannot set a name that starts with a digit, so the three
names that did move to the leading underscore, and the names read behind
a _51DEGREES_ one are no longer read at all.

51DEGREES_IPI_PATH, 51DEGREES_DD_PATH and 51DEGREES_RESOURCE_KEY become
_51DEGREES_IPI_PATH, _51DEGREES_DD_PATH and _51DEGREES_RESOURCE_KEY.
FOD_CLOUD_API_URL becomes _51DEGREES_CLOUD_ENDPOINT, which is the name
the Rust library already reads, and the constant holding it is renamed
with it. The RESOURCE_KEY and LICENSE_KEY reads behind the _51DEGREES_
names in the 51Did and cloud engine examples and the live 51Did tests
are removed.
The old-name half needed a quote, a dollar sign, a brace or a getenv call
in front of the name, so it could not see one declared as a key in a
workflow env: block or listed at the start of a line in a readme. A name
at the start of a line, past any spaces, tabs and a list dash, now counts.

The tests state it, with a workflow line whose key is an old name, a
readme list entry and a shell assignment on the failing side, and a
workflow line whose key is not an old name, and a longer name that ends
in one of these words, on the passing side.
The check misses an old name written as ${env:NAME}, as $ENV:NAME, as
Env:\NAME or as ${{ env.NAME }}, because its prefix set wants the dollar
and the env: adjacent, and its brace branch wants the name straight after
the brace. Both the CI scripts and the pipeline config lookup write the
braced form, so an old name can come back through the file types the check
watches.

These cases fail until the prefix set takes the env token on its own.
rc has taken a helper that reads a resource key and a licence key from
several names, in the cloud request engine tests, the 51Did tests and the
developer cloud engine example. The design is kept. The old names
RESOURCE_KEY and LICENSE_KEY are dropped from each list, because every
variable has one name and nothing is read in its place, and the workflow
that runs these tests sets _51DEGREES_RESOURCE_KEY and
_51DEGREES_LICENSE_KEY, so nothing in CI reads the dropped names.

The cloud endpoint reads _51DEGREES_CLOUD_ENDPOINT in the example and is
named that way in the test documentation.
The prefix set wanted the dollar and the env token adjacent, and its brace
branch wanted the name straight after the brace, so ${env:NAME}, $ENV:NAME,
Env:\NAME and a workflow ${{ env.NAME }} all went past. Matching the token
on its own, in any case and with a following backslash allowed, closes all
four, which is what the Go check already does.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant