[jwrosewell]

Usage sharing finds the values it shares from the Evidence and, when the web integration supplies it, the web request. Since #68 it copied every Evidence value into a new case-insensitive map twice on every call, whether or not the call was shared. This draft finds the values only when something needs them, and without a copy when the Evidence holds no web request.

The problem

rc measured slower than main in the usage sharing overhead test, and swapping #68 alone moved the cost both ways. These are the figures of run 43309004655, one ubuntu-22.04-arm runner, 32 rounds, every tree once a round with the order turning, milliseconds per call, median (lowest to highest).

Tree	ShareUsageOverhead_HundredEvidence_ms	ShareUsageOverhead_SingleEvidence_ms
main f69da52	1.6120 (1.5790 to 1.7170)	0.0058 (0.0056 to 0.0062)
rc 36c153d	1.8310 (1.7560 to 1.9390)	0.0065 (0.0061 to 0.0070)
rc without #68	1.4160 (1.3830 to 1.4840)	0.0059 (0.0056 to 0.0065)
main with #68	1.7925 (1.7500 to 1.8390)	0.0064 (0.0060 to 0.0079)

The figure named ShareUsageOverhead_HundredEvidence_ms comes from ShareUsageOverhead_ThousandEvidence, which gives each call 1,001 Evidence values (ShareUsageOverheadTests.java lines 158 to 184 at 36c153d5), so this description calls it the 1,001 value figure. The single value figure is 10,000 calls with the User-Agent alone (lines 123 to 146).

On rc, ShareUsageBase.processInternal calls ShareUsageEvidence.find on every call, and ShareUsageTracker.generateKey calls it again. find copies every value into a new TreeMap ordered by String.CASE_INSENSITIVE_ORDER. TrackerBase.track makes the key once to look the call up and, for a call it has not seen, again to store it (TrackerBase.java lines 63 and 67). So a call that is not shared copied every value twice, and a call that is shared three times.

What the specification says

Evidence, line 10 at 48e28393:

Keys MUST be case-insensitive and will usually be split into <prefix>.<field>.

Usage sharing element, Accepted Evidence, lines 18 to 29 at 48e28393:

It shares instead what it finds, from two places:

• every Evidence value in the Flow Data, and
• every header other than Cookie, every cookie, query string parameter and session value of the web request, [...] when the web integration supplies the request as server.http-request [...]

A key found in both places is shared once, with the value from the Evidence.

Session tracking, lines 262 to 264:

These requests will usually all generate exactly the same usage data. We have no need for this duplicate data, so it MUST be discarded as early as possible.

#68 copied the values so that keys from the two places are compared without regard to case and a key in both keeps the Evidence value. Both still hold, as below.

What changed and why

1. ShareUsageEvidence.find copies only when there is a web request (lines 50 to 63 at ee271a37). When the Evidence holds no server.http-request, it returns the Evidence's own read-only map. FlowData's Evidence is a ConcurrentSkipListMap ordered by String.CASE_INSENSITIVE_ORDER (EvidenceDefault.java line 42), so its keys are case-insensitive as the specification requires, and a ConcurrentSkipListMap cannot hold a null value. So the map holds the same keys and values, in the same order, as the copy did, which left out null values and the web request and was keyed without regard to case. Every FlowData a Pipeline creates gets an EvidenceDefault from FlowDataFactoryDefault, as PipelineDefault, its constructor, EvidenceDefault and PipelineBuilderBase.getFlowDataFactory are all package-private (PipelineBuilderBase.java lines 218 to 219). With a web request, find copies and merges exactly as before.
2. The element finds the values only when it needs them (processInternal, lines 645 to 672, and processData, lines 793 to 840). It finds them in processInternal only when an ignore filter is set, and otherwise in processData once tracker.track has allowed the call to be shared. A call the tracker turns down, which is most calls, no longer finds them in the element at all. The ignore filter, the cancel check, the share percentage and the tracker run in the same order as before, and on each the same values decide.
3. ShareUsageTracker is as on rc. It finds the values for its key in generateKey, which without a web request no longer copies.

Every exit was read. processInternal leaves through the ignore filter's break, the cancel check, or processData. processData returns when the share percentage or the tracker says no, and otherwise finds the values if the ignore filter had not, and queues or discards them as before. find returns the Evidence's map, or the copy.

Nothing public is removed or renamed. ShareUsageEvidence is new on rc since #68 and is not on main.

A shape that was measured and not kept

The draft's history first gave the element's copy to the tracker for each call, through a thread-local value, so that a call with a web request copied once whether shared or not (db808b00). It cleared the 1,001 value figure but not the single value one. In run 43332442847 (job 131580208280) the single value figure was main 0.0058 (0.0057 to 0.0061), rc 0.0066 (0.0063 to 0.0070), db808b0 0.0066 (0.0064 to 0.0074), and the same with a holder made once for each thread (883ae89b) 0.0064 (0.0061 to 0.0068). Commits c736ca5c and ee271a37 replace it, and ShareUsageTracker.java at the head is the same file as on rc (blob 8bfa52d3).

Tests

• ShareUsageEvidenceReadTests gives the element Evidence, and a web request, that count how often all their values are read. Each test makes one call that is shared and a repeat that the default tracker turns down, checks which was shared from the queue, and counts the reads in the repeat. NoWebRequest_EvidenceReadOnceWhenNotShared asks for one read of 100 Evidence values, for the session tracking key. WebRequest_EachCopiedOnceWhenNotShared asks for one read each of the Evidence and of a web request of 100 values, for the one copy the key is made from.
• ShareUsageEvidenceTests checks that find without a web request returns the Evidence uncopied and keyed without regard to case, and that with a web request it adds the web request's values, keeps the Evidence value for a key in both, and leaves out the web request itself.

Both run on ubuntu-22.04 with JDK 17 in run 43337686624, with every test #68 added or changed, through mvn test -pl pipeline.engines.fiftyone,pipeline.cloudrequestengine,pipeline.web.packages/pipeline.web,pipeline.web.packages/pipeline.web.mvc -am.

• Red, job 131597924393 on 04680bd6, being rc 36c153d with the two test files only. Both files are the same blobs as at the head (d3d919e7 and fee9b726). pipeline.cloudrequestengine ran 48 with no failures, and pipeline.engines.fiftyone printed Tests run: 48, Failures: 3, Errors: 0, Skipped: 1, the three below, so Maven stopped before the web modules.

ShareUsageEvidenceTests.Find_NoWebRequest_ReadsTheEvidenceWithoutCopying:68 The Evidence was copied, so every request pays for a copy of every Evidence value. ==> expected: <later> but was: <null>
ShareUsageEvidenceReadTests.NoWebRequest_EvidenceReadOnceWhenNotShared:143 A call that is not shared read all the Evidence more than once, when the session tracking key needs it once. ==> expected: <1> but was: <2>
ShareUsageEvidenceReadTests.WebRequest_EachCopiedOnceWhenNotShared:177 A call that is not shared copied the web request more than once. ==> expected: <1> but was: <2>

• Green, job 131597924343 on b197ca15, whose tree 178476d5 is the head's tree. pipeline.engines.fiftyone ran 48 with 1 skipped, the @Disabled ShareUsageBuilder_IgnoreData_InvalidFilter. pipeline.cloudrequestengine ran 48, pipeline.web 26 and pipeline.web.mvc 9, with no failures. Accept no Evidence for usage sharing, and give it the whole web request #68's own tests are unchanged and pass.

Timing

The head's tree, 178476d5, was measured as b197ca1 in run 43337686624, job 131597923970. It used the measuring steps of run 43309004655 on one ubuntu-22.04-arm runner, with 32 rounds, every tree once a round and the order turning, and *ShareUsageOverheadTests* only. The figures are milliseconds per call, as the log prints them, median (lowest to highest), and the last column is from the log's own comparison of each tree with main.

Tree	1,001 values	1 value	1 value, lower than main in
main f69da52	1.6135 (1.5680 to 1.6650)	0.0059 (0.0055 to 0.0062)
rc 36c153d	1.8280 (1.7200 to 1.9240)	0.0067 (0.0064 to 0.0071)	0 of 32
rc without #68	1.4005 (1.3700 to 1.5060)	0.0059 (0.0057 to 0.0065)	9 of 32
find change alone, 637d5a0	1.4240 (1.3790 to 1.4950)	0.0063 (0.0059 to 0.0070)	1 of 32
this draft, b197ca1	1.4225 (1.3780 to 1.4850)	0.0062 (0.0059 to 0.0067)	2 of 32

On 1,001 values the draft is 11.8 per cent below main and lower in all 32 rounds, and its range is clear of main's. On 1 value its median is inside main's range and the middle halves overlap (0.0060 to 0.0063 against 0.0058 to 0.0060), but it is 5.1 per cent above main and lower in only 2 of 32 rounds.

That 1 value test times 10,000 calls in a new JVM. So run 43342968543, job 131615467187, ran it for 100,000 calls, the one change being int iterations = 10_000; to 100_000 in ShareUsageOverheadTests.java, made the same way in every tree. A cost on every call would show at 100,000 calls as it does at 10,000.

Tree	1 value at 100,000 calls	lower than main in
main	0.0038 (0.0032 to 0.0043)
rc	0.0032 (0.0027 to 0.0040)	30 of 32
rc without #68	0.0030 (0.0027 to 0.0039)	30 of 32
this draft	0.0031 (0.0027 to 0.0034)	32 of 32

At 100,000 calls the draft is 19.3 per cent below main and 2.9 per cent above rc without #68, with the middle halves overlapping and the draft lower in 13 of 32 rounds. So the 0.0003 ms the draft shows over main at 10,000 calls is not a cost on every call, as it would then show at 100,000 calls too. Where in the 10,000 calls it is spent has not been measured.

Full build

Run 43338657365, started by the full build script without macOS on tmp/pr-matrix-120-lazy from rc ef34a03, finished with success on the head ee271a3. Configure, job 131601182237, printed "Merging in any changes from rc", "Merge made by the 'ort' strategy." and "Left out 6 macOS entries". All ten Setting 'PullRequestSha' lines read ee271a3. There is one in Configure and one in each of the nine Build and Test jobs, which are Ubuntu Java 8, 11, 17 and 21, Ubuntu ARM Java 17, and Windows Java 8, 11, 17 and 21. In each of the nine jobs ShareUsageEvidenceTests ran 3 and ShareUsageEvidenceReadTests ran 2, with no failures and none skipped. The full build of the first shape, run 43329495458, was cancelled when that shape was replaced.

macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

What remains

1. A call with a web request still copies the Evidence and the web request once for the tracker's key when it is not shared. A call that is shared copies them three times, twice for the key and once for the values shared, as on rc. With an ignore filter set, the element also copies them once for the filter, as on rc. The usage sharing overhead test adds no web request, so neither figure measures that path.
2. rc moved to ef34a03 (#117) after this branch was made from 36c153d. Ask the IP Intelligence benchmark only for the property it reads #117 changed only PerformanceBenchmark.java and its test in the IP Intelligence console example. The full build merged rc in, and the draft is mergeable with no conflict.