[jwrosewell]

Follows the merged specification pages Usage sharing element and Web integration, from 51Degrees/specifications#99 and #103, both merged into rc on 9 November 2026.

The problem

The usage sharing element's filter, EvidenceKeyFilterShareUsage, accepted every header except Cookie, and the Pipeline's filter is the union of its elements' filters, so it accepted every header too. The client-side endpoints' ETag is a hash of the Evidence in that filter, so a revalidation carrying If-None-Match hashed differently and never got a 304, and the Vary header named every header.

What changed

1. The usage sharing element accepts no Evidence. ShareUsageBase.getEvidenceKeyFilter returns an empty list. EvidenceKeyFilterShareUsage holds the rules for which values are shared. The keys never shared are id.email, id.salt, header.cookie, request-query and request-path, each matched as the whole key or after a .. Then come the block list, 51D_ cookies and session values, and the query string allow list.
2. It shares what it finds. The new ShareUsageEvidence.find gives every Evidence value and, when the Evidence holds server.http-request, every value of the web request, keyed without regard to case. A key in both keeps the Evidence value, and server.http-request itself is never one of the values. The ignore-data filter and the values shared both read these.
3. The session tracking hash is of what would be shared. ShareUsageTracker keys its cache on the found values its filter includes, rather than on the Evidence alone, which no longer holds headers no other element accepts. It keeps its own cache for that, because the base class's cache is keyed by FlowData.generateKey, and changing the base class would release pipeline.engines.
4. The web integration adds server.http-request on every request. WebRequestEvidenceServiceCore records every value it offers, accepted or not, in a new WebRequestEvidence, and adds it directly rather than through the accepted-Evidence check. That covers headers, cookies, query string and form parameters, session values, server.client-ip, header.protocol, the method and path of the request line as sent (server.request-method and server.request-path, the path not decoded), and the Spring service's server.contextroot. The path is never shared and the method is.
   • The form chosen is a read-only map keyed the way Evidence is keyed (header.<name>, cookie.<name>, query.<name>, session.<name>), not the HttpServletRequest. pipeline.engines.fiftyone, where the usage sharing element lives, does not depend on the servlet API. The values are copied so that they stay the same once the container recycles the request. Its toString gives a count rather than the values, so a debug log of Evidence, or an example page that lists all Evidence, does not print every header and cookie.
5. The cloud request engine leaves the web request out. CloudRequestEngineDefault.getFormData sends every Evidence value whose prefix is not header, cookie or query, whether or not the engine accepts it, so it would have sent server.http-request to the cloud service as http-request. It now leaves that key out. The key is repeated as a private constant there, because pipeline.cloudrequestengine does not depend on pipeline.engines.fiftyone.
6. The sequence element accepts the session id and sequence. SequenceElement accepted no Evidence and relied on usage sharing accepting query.session-id and query.sequence. With usage sharing accepting nothing, every client-side callback would start a new session. It now accepts both, as its specification lists. It also reads a sequence that arrives as text, which the cast to int failed on, as the red run below shows happens on rc today.

How to verify

Both runs are on ubuntu-latest with JDK 17, with the JavaScript templates submodule checked out, running ShareUsage*Tests, EvidenceKeyFilterShareUsageTests, CloudRequestEngineTests, UsageSharingEvidenceTests, ClientsidePropertyServiceTests, PipelineFilterTests, UACHServiceCoreTests and FiftyOneInterceptorTests through mvn -f pipeline/pom.xml -pl pipeline.engines.fiftyone,pipeline.cloudrequestengine,pipeline.web.packages/pipeline.web,pipeline.web.packages/pipeline.web.mvc -am test.

• Red, rc at 2299507 plus the three test commits (8977586, a645d1f, 2b1e52a): run 42424560368, TOTALS {'passed': 76, 'failure': 7, 'skipped': 1, 'error': 2}. The skip is ShareUsageBuilder_IgnoreData_InvalidFilter, which is @Disabled. The run that printed the red errors' causes in full, without the third test commit, is run 42419662468.
• Green, this branch at 2b1e52a: run 42424572596, TOTALS {'passed': 85, 'skipped': 1}.
• The two web request tracker tests then got a tracker of their own with an hour's interval (60b5550), because the class's 50 millisecond interval lets a busy runner count their second request as new, which is how Share a repeated evidence set at most once in each interval #61's tests were seen to fail on macOS. Green at 60b5550: run 42425700016, TOTALS {'passed': 85, 'skipped': 1}. It merges with Share a repeated evidence set at most once in each interval #61 without conflict.

The brief's four checks, each red on rc unless noted.

1. The Pipeline does not accept header.if-none-match or an arbitrary header. ShareUsageAcceptedEvidenceTests builds a Pipeline of the test engine EmptyEngine and the usage sharing element.

FAILURE  ...ShareUsageAcceptedEvidenceTests.Pipeline_WithShareUsage_AcceptsOnlyTheOtherElementsEvidence
         If-None-Match is accepted, so a revalidation's ETag differs from the one it sends. ==> expected: <false> but was: <true>

2. With the web request supplied, a header no other element accepts is still shared, and the cookie header and a blocked header are not. ShareUsageElement_WebRequest_SharesWhatNoOtherElementAccepts also checks that User-Agent, in both places, is shared once with the Evidence value, that a 51D_ cookie and the client IP from the request are shared, and that the web request itself is not.

FAILURE  ...ShareUsageElementTests.ShareUsageElement_WebRequest_SharesWhatNoOtherElementAccepts
         A header only the web request holds was not shared. ==> expected: <shared> but was: <>

The web integration side is addEvidenceFromRequest_AddsTheWebRequest.

FAILURE  ...UsageSharingEvidenceTests.addEvidenceFromRequest_AddsTheWebRequest
         The web request was not added.

3. A revalidation's ETag is the one it sends, so it gets a 304. Revalidation_SameETag_Gets304 serves the JSON endpoint from a real Pipeline of the sequence, JSON builder and usage sharing elements, with a request carrying the session id and sequence as a callback does, then sends If-None-Match with the ETag it got. On rc the Pipeline cannot process that request at all.

ERROR    ...UsageSharingEvidenceTests.Revalidation_SameETag_Gets304
           fiftyone.pipeline.exceptions.AggregateException: Exception(s) occurred processing evidence.
           Suppressed: java.lang.ClassCastException: class java.lang.String cannot be cast to class java.lang.Integer ...
           Suppressed: fiftyone.pipeline.core.exceptions.PipelineConfigurationException: Make sure there is a SequenceElement placed before this JsonBuilderElement in the pipeline

An earlier red run with the first test commit only, before the requests carried a session id and sequence (run 42418221825), failed on the ETag itself: The revalidation's ETag differs from the one it sends. expected:<113796253> but was:<1812379018>. Callback_SessionAndSequence_CarriedOn fails on rc with the same ClassCastException, and passes here with the session id kept and the sequence at 3.

4. Without a web request, Evidence added by hand is shared as before. The existing ShareUsageElementTests, which add Evidence by hand, pass unchanged on both sides, apart from the one below. Two new tracker tests show the session tracking hash. ShareUsageTracker_WebRequest_SessionCookieTracked passes on both sides, and ShareUsageTracker_WebRequest_UnsharedValueIgnored, two requests differing only in a cookie that is not shared, fails on rc with expected: <1> but was: <2>.

ShareUsageElement_TrackSession_SessionCookie asserted that the element accepted the session cookie. It now asserts that it accepts nothing, and still checks that the session cookie keeps two sessions apart and is never sent, so it is red on rc by design. CloudRequestEngineTests.getFormData_LeavesOutTheWebRequest fails on rc with The web request was sent to the cloud service.

Every element's accepted Evidence

Under #99's rule that an element accepts only the Evidence that can change its results, I read the filter of every flow element in the repository's main code. Usage sharing was the only one that accepted every header, and the sequence element relied on it. Every other filter is an EvidenceKeyFilterWhitelist, which matches only the keys listed, so none accepts every key or every header.

Element	Accepts
ShareUsageElement	nothing (this pull request)
SequenceElement	query.session-id, query.sequence (this pull request)
SetHeadersElement, JsonBuilderElement	nothing
JavaScriptBuilderElement	header.Host, header.protocol, query.fod-js-object-name, query.fod-js-enable-cookies, server.contextroot
CloudRequestEngineDefault	the keys the cloud service lists at https://cloud.51degrees.com/api/v4/evidencekeys. On 7 November that list was 138 named keys, with no If-None-Match and no wildcard
DeviceDetectionCloudEngine, IPIntelligenceCloudEngine, GeoLocationCloudEngine, the hardware profile engines (PropertyKeyedCloudEngineBase)	nothing
DeviceDetectionHashEngine, IPIntelligenceOnPremiseEngine	the keys the native engine lists for its data file (engine.getKeys())
TranslationEngine, CountriesTranslationEngine, CountryCodeTranslationEngine	query.translation, query.accept-language, header.accept-language
ParallelElements, the Pipeline	the union of their elements'
The developer examples' SimpleFlowElement, SimpleOnPremiseEngine, SimpleClientSideElement, SimpleCloudEngine	date-of-birth, date-of-birth, cookie.date-of-birth, nothing

WebPipeline_AcceptsOnlyWhatItsElementsUse builds a Pipeline of the sequence, set headers, JSON builder, JavaScript builder and usage sharing elements, and checks that it accepts neither header.if-none-match nor header.x-test nor server.http-request, while it still accepts header.host and query.session-id. On rc it fails with If-None-Match is accepted., and here it passes.

Other drafts touching the same files

• Share a repeated evidence set at most once in each interval #61 changes ShareUsageTracker.java and ShareUsageTrackerTests.java.
• Pause usage sharing sends after repeated failures #62 changes ShareUsageBase.java.
• git merge-tree of this branch with each of them merges without conflict. Cache the client-side endpoints' answers as the specification and RFC 9110 ask #67 changes ClientsidePropertyServiceCore.java, which this does not, and the revalidation test here sends a GET, so it holds under Cache the client-side endpoints' answers as the specification and RFC 9110 ask #67's 412 rule for other methods.

The JavaScript file and the session id

A request for 51Degrees.core.js carries no session id, so the sequence element creates one and writes it into Evidence, where, as an accepted key, it is part of the ETag, and the file gets a new ETag every time. James Rosewell decided on 7 November 2026 that this is correct, because the session id is rightly part of the ETag and the result changes as the sequence increases, so nothing changes there. A callback carries its session id and sequence, so its revalidation gets a 304.

For James to decide

1. Two more packages change. The cloud request engine fix changes pipeline.cloudrequestengine, which with pipeline.developerexamples.cloudengine is not in the 29 packages Monday's plan already releases. Without it, the cloud service would receive every header and cookie of each request as one form field.
2. Other languages' cloud request engines may send the web request too, if they also send Evidence they do not accept.
3. Query string parameters starting 51D_ that only usage sharing accepted no longer reach Evidence, so the JavaScript builder no longer echoes them into its callback URL. Usage sharing still shares them from the web request.

The earlier head's full pull request build, common-ci's nightly-pull-request.yml run as a dry run from a temporary branch, built 60b5550, the head before the two commits above, merged with rc (run 42455459082). All 15 build and test jobs passed on the second attempt, on Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, Windows with Java 8, 11, 17 and 21, and macOS arm64 and x64 with Java 11, 17 and 21. The first attempt failed only in MacOS_X64_Java_11, in DataUpdateServiceTests.DataUpdateService_UpdateFromFile_FileUpdated, a race in the data update service that this change does not touch (#70), and MacOS_X64_Java_21 was cancelled with it. An earlier full build of 2b1e52a, before the tracker interval commit, passed all 15 (run 42424866504).

What remains

• Nothing in the specification pages is left unmatched in this repository. WebRequestEvidence is a map keyed the way Evidence is keyed rather than an object with a method for each part, which suits a module that cannot see the servlet API. The query string is not added whole as server.request-query, as the pages ask only of an integration that supplies the request line as Evidence, which this one does not yet.

Checked against the merged pages, and against Go

I read both pages in full and compared this branch with 51degrees-go#62, which was aligned to the same text. Two differences remained, and each now has a test that failed first.

1. The web request did not hold the method and path of the request line, which the web integration page lists as parts of server.http-request. WebRequestEvidenceServiceCore now adds server.request-method and server.request-path, as Go's view does. addEvidenceFromRequest_WebRequestHoldsMethodAndPath checks them, and ShareUsageElement_WebRequest_SharesWhatNoOtherElementAccepts now checks that the method is shared and the path is not.
2. The usage sharing page says the Cookie header is never shared and that never-shared keys match as the whole key or after a .. Here the Cookie header was only blocked by the block list, so a key such as x.header.cookie was shared. header.cookie is now in the never-shared list, and cookieHeaderNeverShared checks it.

The rest matched. The element accepts no Evidence. Only 51D_ cookies are shared, as their own cookie. Evidence, and the Cookie header is not. session. values are shared only when they start 51D_. id.email, id.salt, request-query and request-path are never shared. server.http-request is a read-only copy of the request and is never shared itself. This repository has no option to share all Evidence, so the "even when all Evidence is shared" clause has no setting to test here.

• Red, the two commits of tests alone on top of 60b5550: run 42636980581, TOTALS {'passed': 85, 'failure': 2, 'skipped': 1}. cookieHeaderNeverShared failed with x.header.cookie ==> expected: <false> but was: <true>, and addEvidenceFromRequest_WebRequestHoldsMethodAndPath with The method is missing. expected:<GET> but was:<null>.
• Green, with the change at ec35e71: run 42637231961, TOTALS {'passed': 87, 'skipped': 1}.
• GitHub reports no conflict with rc (63d0bb8), so rc was not merged in.

The full pull request build of this head, ec35e71, merged with rc (run 42669364643). "Setting 'PullRequestSha'" in the logs of all 10 completed jobs is ec35e71, which is the head. Every non-macOS job passed, being Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21, with the configure job. I cancelled the run while the six macOS jobs were still queued.

macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

Release note

Usage sharing no longer makes a Pipeline accept every HTTP header. The client-side endpoints' ETag and Vary header now cover only the Evidence the other elements use, so a browser's revalidation gets a 304 and the Vary header no longer names every header.

What usage sharing sends changes as follows.

• The Cookie header is never sent. Cookies starting 51D_ are sent as their own entries, and no other cookie is.
• The method of each request is now sent as server.request-method. The path and query string are never sent.
• A query string parameter starting 51D_ that only usage sharing used is still sent, but no longer reaches Evidence, so the JavaScript builder no longer repeats it in its callback address.

The web integration adds the request to Evidence as server.http-request, a read-only copy. It prints as a count rather than listing headers and cookies. The Cloud Request Engine does not send it to the cloud service. The sequence element now accepts the session id and sequence of a callback and reads a sequence that arrives as text, which used to fail with a class cast error.