[jwrosewell]

For #11. This brings the client-side JavaScript and JSON endpoints of the web integration in line with the Cache header detail section of the web integration specification on rc, which 51Degrees/specifications#53 wrote, and with three drafts that changed the same section, being 51Degrees/specifications#81 (the 304 and 412 answers), 51Degrees/specifications#115 (the weak entity tag) and 51Degrees/specifications#121 (the Evidence the request supplied). #115 and #121 are on rc of the specifications repository. #81 is still an open draft. The 304 and 412 split by method is already on rc, in the Cache header detail section quoted below, and the headers a 304 carries follow RFC 9110 section 15.4.5 until #81 adds them to the specification.

serveContent in ClientsidePropertyServiceCore.java serves both endpoints, for the servlet filter and for the Spring interceptor.

The problem

On rc at 7299688 the endpoints had these faults.

1. The ETag was not an entity tag. The ETag was the exclusive or of the Evidence values' Java hash codes, sent as a bare number such as 661586230. RFC 9110 writes an entity tag in double quotes, so the number was neither a weak nor a strong entity tag.
2. The ETag changed on every request. The ETag was worked out from the Evidence after processing. The Sequence Element adds a new query.session-id to every request that has none, and the Pipeline's Evidence key filter holds that key, so the JavaScript endpoint sent a different ETag every time and no revalidation ever got a 304.
3. If-None-Match was compared as one whole string. A list of tags, a weak tag, * or a second If-None-Match line never matched.
4. A failed response was cached like a good one. Every 200 got Cache-Control: private,max-age=1800 and the ETag, even when processing had recorded errors, so a browser kept a response built without the failed element's data for 30 minutes and then revalidated it into a 304.
5. A matching If-None-Match got a bare 304 whatever the method. The 304 carried no headers, and a POST to the JSON endpoint got a 304 too.
6. A missing builder result threw. When the builder element is the one that failed, the content is null, and response.getWriter().write(content) threw a NullPointerException after the headers were set.
7. Vary left out the headers of a parallel group. Only a filter whose class was exactly EvidenceKeyFilterWhitelist was read, and a parallel group's filter is an EvidenceKeyFilterAggregator.
8. Content-Length counted characters rather than bytes. When an application writes its responses in UTF-8, content holding a character outside ASCII, such as a city name with an accent, is longer than the header says, and the servlet container stops at the length the header gives.
9. The max-age could not be changed. It was fixed at 1800 seconds, where the specification says "Max-age SHOULD be configurable to allow this to be addressed if it is found to be an issue for a particular client", the issue being JavaScript that gathers values which change, such as a location.

What the specifications say

The Cache header detail section on rc gives a table of the headers both endpoints MUST set. Its rows are Cache-Control max-age=1800 private, Vary "all headers in Pipeline Evidence key filter", ETag "Calculated hash of ALL the Evidence values in Evidence filter key for this Pipeline", Content-Type application/x-javascript or application/JSON, and Content-Length "Content length in bytes". The same section says the following.

• "The same Evidence MUST give the same ETag in every process running the same build of the Pipeline API."
• "A response to a request whose processing recorded errors ... MUST carry Cache-Control: private, no-cache and MUST NOT carry an ETag."
• A request whose If-None-Match matches gets "a 304 status code (Not Modified) to a GET or HEAD request and a 412 status code (Precondition Failed) to any other method", whether or not processing recorded errors.

#115 and #121 added two rules.

• #115 says "The ETag MUST be sent as a weak entity tag, W/"...", because it is not a strong validator." The ETag depends on the Evidence alone, so a data file update that changes the content for the same Evidence leaves the ETag as it was.
• #121 says "The ETag MUST be calculated from the Evidence values in the Pipeline's Evidence key filter exactly as the request supplied them, before any Flow Element has added or replaced a value."

RFC 9110 and RFC 9111 are both Internet Standards, and their rfc-editor.org records list nothing that obsoletes or updates them, checked on 9 November 2026.

• RFC 9110 section 8.8.1 says a strong validator "changes value whenever a change occurs to the representation data that would be observable in the content of a 200 (OK) response to GET".
• Section 8.8.3 gives the grammar entity-tag = [ weak ] opaque-tag, weak = %s"W/", opaque-tag = DQUOTE *etagc DQUOTE and etagc = %x21 / %x23-7E / obs-text. An entity tag that does not meet the strong validator definition MUST be marked weak with W/, which is case sensitive.
• Section 8.8.3.2 says that with the weak comparison "two entity tags are equivalent if their opaque-tags match character-by-character, regardless of either or both being tagged as "weak"".
• Section 13.1.2 defines If-None-Match = "*" / #entity-tag and says "A recipient MUST use the weak comparison function when comparing entity tags for If-None-Match". The condition is false if the field is *, false if one of a list of entity tags matches, and "Otherwise, the condition is true." A false condition gets 304 for GET or HEAD and 412 for any other method. A list holding * and other values "is syntactically invalid".
• Section 5.2 makes several lines of one field one list, joined with commas. Section 5.6.1.2 has a recipient accept empty list elements, and section 5.6.3 allows spaces and tabs around the commas.
• Section 15.4.5 says "The server generating a 304 response MUST generate any of the following header fields that would have been sent in a 200 (OK) response to the same request: Content-Location, Date, ETag, and Vary; Cache-Control and Expires."
• RFC 9111 section 4.3.4 has a cache that gets a 304 pick the stored response by the 304's validator, the most recent one with a matching weak validator, and replace that response's header fields with the 304's as section 3.2 says. That is why the 304 carries the Cache-Control of the 200 it stands for. When processing fails, the 304 carries private, no-cache and, as Shorten the Location Doxygen titles and keep the generated index pages out of search #53 asks, no ETag. A cache that follows section 4.3.4 then has no validator to pick the stored response by and may leave its headers as they were. Either way the stored response is past its max-age and is checked again before its next use.

What changed

All in pipeline/pipeline.web.packages/pipeline.web. Line numbers are at 902290d for items 1 to 8.

1. A weak ETag of the supplied Evidence. getETag returns W/"<SHA-256 in lower case hex>". The values come from the WebRequestEvidence record that the web integration adds to Evidence as server.http-request on every request, which holds each value as the request supplied it, before any Flow Element runs. Only values whose keys are in the Pipeline's Evidence key filter count. Keys are lower cased and sorted, and each key and value is written as its length in four bytes, big endian, then its UTF-8 bytes, so two different sets of values cannot give the same bytes. The digest has no per-process seed, so the same Evidence gives the same ETag in every process. A FlowData whose Evidence was added without the web integration has no record, so its own Evidence is used.
2. If-None-Match is evaluated as section 13.1.2 says. ifNoneMatchMatches reads every If-None-Match line with getHeaders and joins them into one list. * matches. Otherwise the field has to be a comma separated list of entity tags, with spaces, tabs and empty elements allowed around the commas, and the list matches when one tag's quoted part equals the ETag's, with or without W/ on either side. Any other field does not match, such as a tag followed by something that is not a tag, two tags with no comma between them, a space inside the quotes, * with a tag, a lower case w/, or an unquoted value.
3. 304 and 412. A matching If-None-Match on a GET or HEAD gets a 304 with no content that carries what the 200 to the same request would, being Vary, Cache-Control, the ETag unless processing recorded errors, and Access-Control-Allow-Origin. The 200 sends no Content-Location or Expires, so neither does the 304, and Date is the servlet container's to add. Any other method gets 412 with no content and the CORS header, so a script on another origin can read the 412. The headers on the 304 are what Write the usage sharing record as the specification says #81, still a draft, adds to the specification, so for now they follow RFC 9110 section 15.4.5.
4. A failed response. A 200 whose flow data holds errors carries Cache-Control: private, no-cache and no ETag, written as the specification writes it. Vary, Content-Type, Content-Length and the CORS header are set as before.
5. No content. A null content is not written, and the response is sent with a length of 0.
6. Vary. init reads any filter that is an EvidenceKeyFilterWhitelist or derives from one, as its comment already said, so a parallel group's headers are in Vary.
7. Content-Length. setHeaders sets the length of the content in bytes in the response's character encoding, which the servlet specification makes ISO-8859-1 when none is set.
8. WebRequestEvidence's class comment says the ETag is made from it too.
9. A configurable max-age. At 4e42723, ClientsidePropertyServiceCore.Default.setCacheMaxAge sets the max-age in seconds and refuses a negative number. The default stays 1800 seconds, DEFAULT_CLIENTSIDE_CACHE_MAX_AGE_SECONDS in fiftyone.pipeline.web.Constants, and a failed response still carries private, no-cache.
   • The servlet filter reads the clientside-cache-max-age init parameter (PipelineFilter.getCacheMaxAgeSeconds) and fails to start on a value that is not a whole number of seconds, 0 or more.
   • The Spring integration reads FiftyOneInterceptorConfig.getClientsideCacheMaxAge(), a default method that returns 1800, so existing implementations still compile. FiftyOneInterceptorConfigDefault.setClientsideCacheMaxAge sets it, and ClientsidePropertyService.setCacheMaxAge is a default method too, which ClientsidePropertyService.Default takes from the core class.
10. Review follow-ups. At 7a2686b, this branch's head:
    • serveContent finds the builder element for the endpoint before it tests If-None-Match. So a Pipeline without the JavaScriptBuilderElement or JsonBuilderElement fails with a PipelineConfigurationException on every request, rather than answering 304 to a client that already holds an ETag. .NET's ServeContent answers the 304 before it looks for the builder, so for a Pipeline missing its builder the two differ. No specification page covers this order.
    • setCachingHeaders sets Vary only when some header affects the response, rather than sending an empty Vary header, as .NET's SetHeaders does.
    • FiftyOneInterceptorConfigDefault.setClientsideCacheMaxAge refuses a negative max-age with the message the service gives, so the error points at the setting.
    • The clientside-cache-max-age init parameter is described in the Javadoc of INIT_PARAM_CLIENTSIDE_CACHE_MAX_AGE in fiftyone.pipeline.web.Constants. The other init parameters, config-file and clientside-properties-enabled, have no documentation beyond their use in the example web.xml files, so none is added for this one.

Overlap with #86. #86 reads the Vary headers through a new getHeadersAffectingJavaScript(), so that they are worked out again until every element has loaded. Both change the line that sets Vary. Whichever merges second keeps both, by reading through getHeadersAffectingJavaScript() and setting Vary only when the list is not empty.

rc was merged in at 330f707, bringing #68. Its Revalidation_SameETag_Gets304 test worked the expected ETag out from generateKey(...).hashCode(). That test now checks that the 304 carries the ETag that was sent, and its mocked request answers getHeaders as well as getHeader.

The supplied Evidence key that was replaced

1f92896 and 36f2526 on this branch met the supplied-Evidence rule another way. They added a default method FlowData.generateSuppliedKey, backed by a copy of the Evidence that FlowDataDefault.process took before processing, and made the ETag from that key's hash code. A DataKey holds only the exclusive or of its values' hash codes, without the keys, so it cannot give the digest above. 902290d therefore removes generateSuppliedKey and the copy, so pipeline.core is as it is on rc, the public FlowData interface is unchanged and no FlowData copies its Evidence. Their test, which built its FlowData without the web integration, is replaced by Json_ETag_UnchangedWhenSequenceElementAddsSessionId, which covers the same case with the real Sequence Element through the web integration.

For the release notes

• The client-side JavaScript and JSON endpoints send their ETag as a weak entity tag, W/"<SHA-256 in hex>", worked out from the Evidence the request supplied. A browser that revalidates the JavaScript now gets a 304, where before the new session id added to every request changed the ETag each time. An ETag kept from an earlier version does not match, so each browser gets one full response after the upgrade.
• If-None-Match accepts *, a list of entity tags, several If-None-Match lines and weak tags, as RFC 9110 section 13.1.2 says.
• A matching If-None-Match gets a 304 for GET and HEAD, carrying the Vary, Cache-Control, ETag and CORS headers a 200 would, and a 412 for any other method.
• A response built while processing recorded errors sends Cache-Control: private, no-cache and no ETag, so a browser does not keep it.
• The Vary header names the headers that elements in a parallel group read.
• Content-Length is the length of the content in bytes.
• The max-age of the client-side endpoints' Cache-Control header can be set, with the clientside-cache-max-age init parameter of the servlet filter or FiftyOneInterceptorConfigDefault.setClientsideCacheMaxAge for Spring. The default stays 1800 seconds.
• A response whose builder element failed is sent with no content rather than throwing a NullPointerException.
• A Pipeline without the JavaScript or JSON builder fails with a PipelineConfigurationException even for a request whose If-None-Match matches, rather than answering 304.
• No Vary header is sent when no header affects the response, rather than an empty one.
• FiftyOneInterceptorConfigDefault.setClientsideCacheMaxAge refuses a negative number where it is called.

How to verify

The tests and the fix are separate commits.

• f42344a adds the tests on top of rc merged in, with the old service code. Twelve tests are written to fail there: JavaScript_VaryHeader_ParallelElements, JavaScript_ETag_IsWeakEntityTag, Json_ETag_IsWeakEntityTag, JavaScript_ETag_SameInEveryProcess, Json_ETag_SameAcrossPipelines, Json_ETag_UnchangedWhenSequenceElementAddsSessionId, IfNoneMatch_Star_304, IfNoneMatch_ListHoldingTheTag_304, IfNoneMatch_TagWithoutWeakPrefix_304, IfNoneMatch_TabBetweenElements_304, IfNoneMatch_SeveralFieldLines_304 and Json_ContentLength_InBytes. The _200 If-None-Match tests are written to pass on both sides, because the old exact comparison never matched those values either. They keep the new parser from matching too much.
• 902290d holds the fix.

JavaScript_ETag_SameInEveryProcess and Json_ETag_SameAcrossPipelines compare the ETag with SHA-256 values worked out apart from the Java code, with Python's hashlib over the same encoding.

The command, with the JavaScript templates submodule checked out because PipelineFilterTests and UsageSharingEvidenceTests build a JavaScriptBuilderElement:

mvn -f pipeline/pom.xml -pl pipeline.web.packages/pipeline.web,pipeline.web.packages/pipeline.web.mvc -am test -Dtest='ClientsidePropertyServiceTests,UsageSharingEvidenceTests,PipelineFilterTests,UACHServiceCoreTests,FiftyOneInterceptorTests,FiftyOneInterceptorConfigDefaultTests' -Dsurefire.failIfNoSpecifiedTests=false

The full build below ran every test in these classes on 7a2686b.

4e42723 adds the configurable max-age with its nine tests in one commit, because they use methods and constants that did not exist before it, so on 902290d they would not compile. In ClientsidePropertyServiceTests they are JavaScript_CacheMaxAge_Configured, Json_CacheMaxAge_Configured_304, JavaScript_CacheMaxAge_FailedResponseNoCache and CacheMaxAge_Negative_Refused. In PipelineFilterTests they are CacheMaxAge_NotSet_IsTheDefault, CacheMaxAge_Set, CacheMaxAge_Invalid_Refused and PipelineFilter_Initialise_InvalidCacheMaxAge. In FiftyOneInterceptorTests it is FiftyOneInterceptor_SetsTheCacheMaxAge.

Earlier commits on this branch were checked on ubuntu-latest with JDK 17, building the same two modules and running the four classes other than UsageSharingEvidenceTests, which came later with rc.

• The failed response and the null content. Red at 012e70a, run 42394657601, with 17 passed, 1 error and 2 failures. Green at b8b53dc, run 42394664423, with 20 passed.

• The 412 and the 304's headers. Red at 4f62ea4, run 42416471804, with 21 passed and 3 failures. Green at 6a74ed1, run 42416575313, with 24 passed.

• 013146b holds the review follow-up tests alone, on 4e42723's code.

  • JavaScript_MissingBuilder_MatchingIfNoneMatch_Throws and Json_MissingBuilder_MatchingIfNoneMatch_Throws expect a PipelineConfigurationException rather than a 304.
  • JavaScript_VaryHeader_PSEUDO now expects no Vary header where the filter names no real header.
  • The new FiftyOneInterceptorConfigDefaultTests holds setClientsideCacheMaxAge_Negative_Throws and setClientsideCacheMaxAge_Zero_Kept.
  • All but setClientsideCacheMaxAge_Zero_Kept are written to fail there. In run 42726888694 (branch tmp/red-67) these four failed, as expected: JavaScript_MissingBuilder_MatchingIfNoneMatch_Throws and Json_MissingBuilder_MatchingIfNoneMatch_Throws with "A PipelineConfigurationException should have been thrown", JavaScript_VaryHeader_PSEUDO with Mockito's "Never wanted here" for setHeader("Vary", <any string>), and setClientsideCacheMaxAge_Negative_Throws with "An IllegalArgumentException should have been thrown".

• 7a2686b, this branch's head, holds the follow-up fixes.

Full build

The full build and test matrix passed every job other than macOS on head 7a2686b in run 42773789376 on 10 November 2026, 11 jobs succeeded, run as a dry run so nothing merged. macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.