[jwrosewell]

Problem

For the rule on licence keys in logs and errors, logging.md, License keys (lines 20 to 39 on specifications rc). It says "A license key MUST NOT appear in full in a log entry or in an error message, at any log level. Where a message names a license key it MUST show only the key's last 4 characters, with each character before them replaced by one *" and "A key of 4 characters or fewer MUST be replaced entirely, by one * for each character. Every package masks the same way, so ABCDEFGH is written ****EFGH in every language." It also says a message that names a URL carrying a key, or the message of an error raised by an HTTP library, MUST hold only the masked key, and that there MUST be a test of a failed request.

The update URL carries licenseKeys=..., and datafile_update_service.py logged it in full for 429, 304, 403, other failures and a failed MD5 check (lines 124 to 153 on rc). A failed request raises an error whose message names the URL as well, and the Cloud Request Engine's no-data message names the response URL.

What changed

• New fiftyone_pipeline_core/keymask.py with mask_licence_key and mask_licence_keys_in_text. The second finds licence parameters in text (licenseKeys, licenceKey, license_keys and the like, in any case) and masks each key of a list separated by | or %7C.

• The update service logs a masked URL, and re-raises a failed request with a masked message.

• The Cloud Request Engine masks the URL in its no-data message.

• The Cloud Request Engine also masks the licence key by value in every exception validate_response raises, through the new CloudRequestEngine.masked. The cloud's accessible properties answer to a licence key that grants no cloud product names the key in its errors text ("License key(s): '...'"), and validate_response put that text into the exception in full (cloudrequestengine.py lines 339 to 343 and 367 to 376 on rc at f55f0d9). A non-JSON answer's content is masked the same way.

A review on 10 November found that cloud errors case. Nothing else in the src folders logs or raises a key or a URL that carries one (searched log, raise and Exception).

Overlap

The data update draft #69 rewrites the same log lines, and the two conflict in datafile_update_service.py. #69 holds this draft's commits as they were at b919b6b, merged in, and masks its own new messages, so this draft merges first and #69 after it. In #69 a failed request is logged rather than raised, so #69 also changes this draft's test_an_error_raised_by_the_request_does_not_hold_the_key to check the logged message.

How it was verified

New tests in pipeline/fiftyone_pipeline_engines/tests/test_licence_key_masking.py.

• Red: run 42704777740, the tests cannot import keymask, as the module is new. A red that shows these tests catching a key logged in full is Align the data update service to the specification: HTTP date, expected time, update from memory and logging #69's run 42796630384, on Align the data update service to the specification: HTTP date, expected time, update from memory and logging #69 with this draft merged in and before Align the data update service to the specification: HTTP date, expected time, update from memory and logging #69 masked its own messages, where test_no_message_holds_the_key fails with 'ABCDEFGHIJKL' unexpectedly found in "Checking for update from ...".
• Green, head b919b6b9: run 42704807459, 9 passed, core 354 passed.
• For the cloud errors text, new tests in pipeline/fiftyone_pipeline_cloudrequestengine/tests/test_licence_key_errors.py give validate_response a stub answer whose errors text, and a non-JSON content, name the key. Red, the tests on 7aad19d (this draft with rc 451c17e merged in, before the change): run 42869089360, both fail on the key in full. Green, head 930b677: run 42869122613, 2 passed, fiftyone_pipeline_core 360 passed.
• 6ef9917 merges rc in, with no conflict.
• Full build of head 6ef991724891aa75f8359fb3db25e4551c1d287d: run 42895898002. 12 jobs passed and 2 were skipped (the performance comparisons), and all 11 Setting 'PullRequestSha' lines equal the head. The Configure log reads "Left out 5 macOS entries".
  macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

The last release into main merged on 2026-11-04.

#69 now holds this draft's later commits too, merged in at 2a90195.