Wait for the test proxies to bind, and stop ConnectionTests leaking clients - #1863
Conversation
…lients Three test harness problems, salvaged from PR 1856. The SDK half of that PR was fixed independently in PR 1859, and the timeout change it proposed is superseded by PR 1862, but these three were never picked up. HttpProxyServer.startAsync only initiates the bind and returns a CompletionStage that completes once the port is listening. All four test classes that stand up a local proxy discarded it. Because the e2e tests run in parallel, tests can start sending traffic before the proxy is accepting, and get a connection refused from a proxy that is about to be perfectly healthy. ProxyServerTools.startProxyServer waits on that future, with a 30 second cap so a proxy that never binds fails the run with a clear error rather than hanging. ConnectionTests.ConnectionTestInstance.dispose was dead code. Nothing ever called it, so every test in the class leaked the client it opened along with its identity. Those clients keep retrying for the rest of the JVM's life, and the ones configured with proxy settings keep retrying through the proxies this class runs locally, competing with the tests still running. Once stopProxy closes those proxies they retry against a dead port instead, for the remainder of the job. This is the same leak that PR 1861 fixed in TokenRenewalTests, and it works against the parallelism cap added in PR 1862, since the whole point of that cap was to stop the proxies being starved. It is now called from an @after. While wiring that up, the ECC identities need care. Unlike every other identity in this class they are created by the test rather than taken from the shared pool, and they carry a self signed certificate that only this test knows about. disposeTestIdentity recycles rather than deletes when RECYCLE_TEST_IDENTITIES is set, and these are SELF_SIGNED, so recycling one would put a device with an unknown thumbprint into the x509 pool for a later test to fail on. They are deleted from the registry instead. ContractAPIMqttTest declared @mocked Object mockSendLock and @mocked Integer mockedInteger, and ContractAPIAmqpTest declared @mocked Object mockSendLock. None of the three is referenced anywhere. Mocking java.lang.Object makes JMockit retransform it, which is a hazard to the whole JVM rather than to one test. To be clear about the evidence: this is not currently failing. The full provisioning suite passes on JDK 8 with reruns disabled, and no ContractAPI failure appears in the last seven CI builds. These are removed because they are unused and risky, not because they are breaking something today. Verified on JDK 8: iot-e2e-common test compilation succeeds, and provisioning-device-client runs 544 tests with no failures and no reruns, the same count as before the removal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fresh evidence from
|
More evidence from
|
| Job | Result | Wall |
|---|---|---|
| Linux JDK 8 | 2589 passed, 0 failed | 12.8 min |
| Linux JDK 11 | 316 passed, 0 failed | 12.7 min |
| Linux JDK 17 | 316 passed, 0 failed | 12.7 min |
| Linux JDK 21 | 316 passed, 0 failed | 12.7 min |
Note 316 executed on main versus 158 in PR builds, so these runs include the @FlakeyTest and @ContinuousIntegrationTest cases that PR builds skip. Wall clock is unchanged at 12.7–12.8 min, confirming again that threadCount=6 costs nothing.
Java Windows 162204 failed, and it is directly relevant here.
The Windows failure is the ECC test, and it has always been the ECC test
Every Windows failure in the last 15 main builds is the same test:
162204 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_false, _true_true]
162122 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_false]
162052 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_true]
161876 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_false, _true_true]
161823 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_true]
161523 failed CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_false, _true_true]
Nothing else has failed on Windows in that window, and this predates all of the recent work — 161523 and 161876 are on 8abba69bc and d1583cc9c. The trailing _true_* is useHttpProxy, so this is the proxied path again.
It also explains a pattern that had been bothering me: this test carries @FlakeyTest, so it is skipped in PR builds and only runs on main. That is why PRs go green while main goes red on the same commit.
This PR's leak fix is firing in that exact run
The Windows log shows 15 ECC devices created and 0 removed:
Successfully added device ecc-test-device-... x15
Removing device ecc-test-device-... x0
CanOpenConnectionWithECCCertificates calls setupEccDevice(), which registers a device and a module directly, and with dispose() never being called none of them are ever cleaned up. Three per failing test across three rerun attempts, plus the passing variants. Every one is left in the registry, and every one leaves a client behind retrying through the local proxies.
This PR fixes both halves of that: the @After closes the client, and the ECC branch deletes the identity from the registry rather than recycling it.
What this PR will not fix
Being clear so this is not oversold. The failing test itself shows 60 seconds of complete silence — the registry operations complete in about 70 ms and then there is not a single connection status transition before the timeout. So the client never gets far enough to report anything, on all three attempts. Removing the leaked clients reduces the contention that plausibly causes that, but I cannot claim from this evidence that it will resolve it.
What the evidence does establish is that the leak is real, it is happening on every Windows nightly, and it is worse than I described when opening this PR, because for ECC identities it leaks registry entries as well as clients.
CanOpenConnectionWithECCCertificates[MQTT_WS_SELF_SIGNED_MODULE_CLIENT_true_false] and its true_true counterpart are the only tests that have failed on the Java Windows nightly in the last 15 main builds. They fail intermittently, roughly a third of runs, and always the same way: the registry work finishes in about 140 milliseconds and then there is 60 seconds of complete silence with no connection status transition at all before the JUnit timeout fires. setupEccDevice ignored useHttpProxyAuth. setup, which every other test in this class uses, picks between the authenticated proxy on 8899 and the unauthenticated one on 9000 based on that flag. setupEccDevice had only the first branch, so every proxied ECC variant went to the authenticated proxy. Two consequences. Both proxied ECC variants piled onto one embedded proxy while the other sat idle, doubling the demand on a server that runs inside the same JVM as the tests. And the true_false variant never tested what its name says: it claims to cover ECC certificates through a proxy that does not require authentication, and it actually exercised the authenticated one, so that combination had no coverage at all. The two tests that fail are exactly the two that were misrouted. Rather than adding the missing branch to the second copy, both call sites now share applyProxySettings. Having the same decision written out twice is what allowed them to drift, and the copy that drifted was the one used by the test that fails. CanOpenMultiplexingConnection keeps its own copy because it builds MultiplexingClientOptions rather than ClientOptions, so it cannot share the helper. This should reduce the failure rate rather than being guaranteed to eliminate it. The underlying condition is contention for the embedded proxies, and this removes one contributor to it. Even on a passing Windows run these tests take about 11.4 seconds against a 60 second budget, so the headroom is smaller than the pass result suggests. Tracked by work item 39365585. Verified with mvn -pl iot-e2e-tests/common -am test-compile on JDK 8. The behaviour of setup() is unchanged, so the only functional difference is which proxy the ECC variants use. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Added the fix for the Windows ECC failurePushed The defect
if (this.useHttpProxy) {
if (this.useHttpProxyAuth) { /* authenticated proxy, 8899, with credentials */ }
else { /* unauthenticated proxy, 9000, no credentials */ }
}
Two consequences. Both proxied ECC variants piled onto one embedded proxy while the other sat idle, doubling demand on a server running inside the same JVM as the tests. And The two tests that fail are exactly the two that were misrouted. Why this is on the Windows pipeline specificallyEvery Windows failure across the last 15 The misrouting itself is platform independent, so the likely explanation is that Windows agents have less headroom and the extra load on one proxy is enough to push these past the 60s budget there but not on Linux. Even on a passing Windows run these take about 11.4s of the 60s budget: ~5s before the registry work, ~2.5s of registry calls, ~3.7s to connect. On the shape of the fixI added the missing branch by extracting
ScopeI want to be careful not to oversell this. The underlying condition is contention for the embedded proxies, and this removes one contributor. It should reduce the failure rate rather than being guaranteed to eliminate it, and the nightlies will show whether more is needed. What it definitely does fix is the coverage gap, which is a real defect regardless of the timing. |
There was a problem hiding this comment.
Pull request overview
Improves e2e test reliability and prevents leaked clients and risky unused mocks.
Changes:
- Waits up to 30 seconds for local proxies to bind.
- Cleans up
ConnectionTestsclients and ECC identities. - Removes unused JMockit fields.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
ContractAPIMqttTest.java |
Removes unused mocked fields. |
ContractAPIAmqpTest.java |
Removes an unused mocked field. |
TokenRenewalTests.java |
Waits for proxy startup. |
MultiplexingClientTests.java |
Waits for proxy startup. |
FileUploadTests.java |
Waits for proxy startup. |
ConnectionTests.java |
Adds cleanup, proxy waits, and shared proxy configuration. |
ProxyServerTools.java |
Adds bounded proxy-start helper. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
setupEccDevice registers the ECC device before it has anything to assign to identity, and for module variants it then registers a module and constructs a client, either of which can throw. When that happened the new @after reached dispose() with identity still null, took the early return, and left the device behind in the registry - the same leak the rest of this change is removing. Record the device id as soon as the registration succeeds and drive the cleanup off that instead of off identity, so a half provisioned ECC identity is still deleted. Deleting the device deletes its module too, so the module needs no separate tracking. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Gate failure on this PR, and where build 162204 actually stands1. The red check on this PR is not caused by this PR
Line 74 is the first Why this is not attributable to this change:
I have requested a re-queue of 2. Build 162204: this PR helps, but does not fully fix itThe failure in 162204 is Pulling the actual log for 162204 shows the failure has two different shapes across the three reruns of the same test, which matters for how much this PR can claim: Two things follow. Run 1 hung inside Runs 2 and 3 hung after the module was registered, with no status transition ever emitted. The status callback is installed immediately after The proxied Verdict. This PR removes two real contributors: the leaked clients that keep retrying through the embedded proxies, and the misrouting that piled both proxied ECC variants onto one proxy while the other sat idle. Both are demonstrably firing in that build. But it does not explain run 1 hanging in 3. Review commentsBoth Copilot comments are answered inline. The first one found a real bug — |
…sing
Build 162413 failed on Linux JDK 17 and JDK 21 with
NullPointerException: Cannot invoke TestIdentity.getClient() because
this.testInstance.identity is null
at ConnectionTests.CanOpenConnection(ConnectionTests.java:336)
Line 336 is the closing client.close(). Every test in this class is
bounded by @test(timeout = 60000), and JUnit runs the method body on a
separate thread that it abandons, still running, when the timeout
fires. @after is outside that timeout, so the dispose() added by this
change ran on the main thread while the abandoned thread was still
partway through the test body, and the identity = null in dispose()
pulled the field out from under it.
The four other classes that dispose from an @after do not clear the
field, and none of them bound their tests with a timeout. Clearing it
was hygiene rather than a requirement - setup() assigns the field on
every attempt - so it is dropped, which restores the existing
convention.
The two timeout bounded test bodies now also take the client once into
a local rather than re-reading it off the shared instance for each
call, so they no longer depend on that field surviving the test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build 162413: the JDK 11 flake cleared, but it exposed a real bug in this PRI re-queued
So the The bug: the new
|
|
| Class changed here | Android runner | Group | Result |
|---|---|---|---|
MultiplexingClientTests |
MultiplexingClientAndroidRunner |
TestGroup6 | passed |
FileUploadTests |
FileUploadAndroidRunner |
TestGroup10 | passed |
ConnectionTests |
ConnectionTestsAndroidRunner |
TestGroup11 | passed |
TokenRenewalTests |
TokenRenewalAndroidRunner |
TestGroup12 | passed |
ProvisioningServiceClientTests is not in this PR's diff. The only provisioning files here are ContractAPIAmqpTest and ContractAPIMqttTest, which are JMockit unit tests in provisioning-device-client — they never run on the emulator.
The host was resolvable in the same emulator run. TestGroup1 also contains ProvisioningClientSymmetricKeyAndroidRunner, which is the other 12 of the 17 tests, and all 12 passed. That class goes through ProvisioningCommon, which constructs new ProvisioningServiceClient(provisioningServiceConnectionString) against the same javasdkgatebgpks-dps... host and creates enrollments through it. So within one 80-second process, the same hostname resolved for 12 tests and failed to resolve for 5. That is transient DNS inside the emulator, not a missing or misdeployed resource — DeployCloudTestResources succeeded, and the DPS instance is created fresh per build with a random suffix, so a not-yet-propagated record early in the run fits the evidence.
Android has been green on this branch. 162185 and 162268 passed on earlier commits of this PR, and 162390 passed on main earlier today.
What I am not claiming
I cannot prove the ordering — whether the 5 failures ran before the 12 successes, which would make it a startup propagation window, or were interleaved, which would make it flaky emulator resolution. The log timestamps are all flushed at task end, so relative ordering within the run is not recoverable from it.
Recommendation: re-run Java Android. Nothing in the diff can affect it. I do not have permission to queue that build myself; it needs approval, or a maintainer can hit re-run on 162434.
Android re-run 162511: same failure, and it is a known environmental flakeRe-ran on the same commit. Cause is unchanged: Evidence that it is not this PR:
Correction to my earlier note: I called it transient after one occurrence. Two in a row on the same commit is worth more than that, so the base-rate check above is the actual basis, not the single observation. This needs an infrastructure fix (DNS resolution for freshly created DPS instances inside the emulator), not a change to this PR. Happy to open a separate issue for it. |
Two gaps in the teardown this change introduced, both leaving clients retrying for the life of the JVM - the leak this class is trying to stop. 1. Identities produced after teardown had already run @after sits outside the @test(timeout = 60000) wrapper, so it can execute while the thread JUnit abandoned at the timeout is still inside setup(). The identity that thread went on to acquire had no owner and leaked. Teardown now takes ownership of the tracked fields and clears them, which makes it idempotent, and each setup path re-checks afterwards and disposes what it produced if teardown had already run. Nothing blocks in either direction: waiting on a setup that is itself hung, which is how these tests have actually timed out, would stall the rest of the run. The public identity field is still never cleared, so an abandoned thread can keep reading it. Cleanup ownership moved to a separate field so that clearing it cannot resurrect that NPE, and so a second teardown cannot requeue the same identity into the shared pool twice. 2. CanOpenMultiplexingConnection kept its clients in locals Its MultiplexingClient and the three DeviceClients registered to it are local to the method, so the new @after never saw them, and close() sat after open() rather than in the finally. A failed or timed out open skipped it and left all four retrying. close() moved into the finally, guarded so it cannot mask what the test threw. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
The new failures are environmental.
|
What is actually failing hereThree red checks, two different causes, and one of them says something uncomfortable about this PR. All three builds are from 09-01 17:37 on merge commit Windows 163074 and Android 163075: never reached the testsBoth failed in
It also appears to have stopped on its own: Linux on Linux 163073, JDK 21: two real test failuresLine 424 on this branch is The uncomfortable partThat second failure happened on this branch, with the proxy bind wait and the What did change: the earlier signature of the leak is gone. The run that motivated this PR ended with 58 What is left is slower and different: the client cannot complete an MQTT-over-WebSocket connect through the local embedded proxy inside 60s, with no connection status transition logged before the timeout. The proxy accepts and then does not answer. That points at the embedded proxy itself under load, not at bind timing and not at leaked clients. On the orderingThese two PRs are not actually circular.
The blocker on this PR is a re-run, not a code change. Requesting one; it needs approval before it starts. |
The red checks here are stale. Re-runs are better, but not clean.The three failing checks point at builds 163073/163074/163075 from 09-01 17:37. The re-runs I queued on the same commit Re-run results, same commit, no code change:
Resource deployment succeeded in all three. That failure was transient and has cleared; it was never related to this PR. What the re-runs actually failed on
An important caveat on the Windows one: a manually queued build has no PR target branch, so it runs the fuller suite - 316 e2e tests executed versus 158 in a gate run, 2589 versus 2273 on JDK 8. So, translated to what the gate would have shown: Android green, Windows green, Linux still red on The part that matters for this PRThat Linux failure is a proxied variant, timing out in What it does fix is real and visible: the 58 What remains is a different failure - the client cannot complete an MQTT-over-WebSocket connect through the local embedded proxy inside 60s, with no status transition logged before the timeout. The proxy accepts and then does not answer. That is the embedded proxy under load, and nothing in this PR addresses it. Refreshing these checksOnly a PR-triggered build updates them, so the stale entries will persist until something pushes to this branch. Say the word if an empty commit to re-trigger is wanted; it will reset the review state, which is already |
CanOpenConnection, CanOpenConnectionWithECCCertificates and CanOpenMultiplexingConnection each declared @test(timeout = 60000). That is exactly Mqtt.CONNECTION_TIMEOUT, the budget the client gives one MQTT CONNECT round trip, so the test and the operation it measures expired on the same tick. Linux build 163287 shows what that produces: 05:04:58.384 Starting test: CanOpenConnection[MQTT_WS_SAS_DEVICE_CLIENT_true_true] 05:04:58.386 Acquiring test device from testSasDeviceQueue ... 60 seconds, no connection status transition at all ... 05:05:58.389 Closing device client... 05:05:58.390 Test failed on run 1, test timed out after 60000 ms Not one status transition in the whole minute. The client was still inside connectToken.waitForCompletion when JUnit killed the thread, on the same tick that Mqtt.connect would have thrown and let the retry policy try again. A test bounded by the same number as the component it exercises cannot observe that component retrying, and cannot report why it failed. Removing the override lets IntegrationTest's two minute rule apply, which leaves room for one stalled attempt to expire and a retry to follow it. The stalls are transient - the other proxied variants connect in under a second in the same runs - so a retry has a real chance of succeeding. Verified the mechanics against JUnit rather than assuming them: with the override, work that needs 75s dies at 60s with exactly the TestTimedOutException seen above; without it, the same work completes under the inherited two minute rule. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root cause of the residual proxied timeout, and a fixPushed What the timeout actually wasFrom Linux build 163287, the failing test verbatim: Not a single connection status transition in the whole minute. The status callback is registered before That rules out the theories we had been working from. It is not the bind race, it is not the leaked clients, and it is not a proxy that accepts and then goes quiet - any of those would have produced a transition or an error. The numbers explain it:
They are the same number. When a connect attempt stalls, the client sits in A test bounded by the same number as the component it exercises cannot observe that component retrying. The changeDropped the The stalls are transient - the other proxied variants connect in under a second in the same runs - so a retry has a real chance. Correcting the record: the description of this PR dismissed removing this override, on the grounds that the retry policy has VerificationCompiles on JDK 8. Mechanics checked against JUnit rather than assumed: with a 60s method override plus a 120s rule, work needing 75s dies at 60s with exactly What I cannot do locally is prove it against a live hub and a real proxy. The gate does that. Being straight about the ceiling: this makes a stalled connect survivable rather than fatal, it does not make the stall stop happening. If a stall lasts beyond two minutes, or several stack up, this test can still fail. Whatever makes the local proxy stall in the first place is still unexplained and still worth chasing separately. |
There was a problem hiding this comment.
🟡 Changes recommended
The cleanup lifecycle breaks reruns and can still leak or incorrectly recycle identities, while timeout removals contradict the stated scope.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
iot-e2e-tests/common/src/test/java/tests/integration/com/microsoft/azure/sdk/iot/iothub/connection/ConnectionTests.java:259
@Afteris not outside this timeout.IntegrationTestinstallsTimeoutas an inner@Rule(IntegrationTest.java:74-77), so it wraps the statement that already includes@After; meanwhileRerunFailedTestRulere-evaluates that same statement and test instance. After an ordinary failed attempt,dispose()leavesdisposed=true, so the retry'ssetup()immediately disposes its newly acquired identity before the test uses it. After a timeout, the abandoned attempt's@Afterhas not run yet, so the retry can instead race with it on these shared fields. Cleanup needs per-attempt ownership/generation rather than this sticky instance flag.
* <p>Every test in this class is bounded by a timeout, the two minute one that {@link IntegrationTest}
* applies. JUnit runs the test body on a
* separate thread and, when the timeout fires, abandons that thread while it is still running. {@code @After}
* is outside the timeout, so teardown can execute while setup on the abandoned thread has not finished
* acquiring its identity. Without this, the identity that setup goes on to produce would have no owner and
- Files reviewed: 7/7 changed files
- Comments generated: 3
- Review effort level: Balanced
…lex setup Two cleanup holes. An ECC identity could still reach the shared x509 pool. dispose() claims and clears eccDeviceIdToDelete, so when teardown lands between the device being registered and the identity being published, the late disposeIfTeardownAlreadyRan saw an identity with no ecc id and fell through to Tools.disposeTestIdentity. With RECYCLE_TEST_IDENTITIES set that requeues it, handing a self signed device with a certificate no other test knows about to the next test that takes an x509 identity. Classification now lives in identityIsEcc, set at the top of setupEccDevice and never cleared, so it outlives the id it was derived from. A late ECC identity is discarded rather than recycled; the device itself is already gone, deleted by the dispose that claimed the id. CanOpenMultiplexingConnection acquired its three identities before entering the try. Failing on the second or third leaked whatever the first calls had already appended, and this method never assigns testInstance.identity, so the @after could not reclaim them either. The loop moved inside the protected scope. Verified the lifecycle against the interleavings rather than by inspection: teardown between registration and publication, teardown before registration, normal ECC ordering, non ECC recycling, and repeated dispose. The first of those is the case being fixed here, and it reproduces as a recycle before the change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Salvages the three test-harness fixes from #1856 that are still relevant. That PR is now
CONFLICTINGand most of it has been overtaken — this carries forward only the parts that were never picked up, rebased onto currentmain.Relationship to #1856
ProxiedSSLSockethonors the connect timeoutHttpProxySocketFactoryreturns an unconnected socketProxiedSSLSocket.close()null-safetymain@Test(timeout = 60000)ConnectionTestsclient leak@Mocked ObjectOn the timeout: #1856 was right and my original reasoning here was wrong. I argued that because the retry policy is
ExponentialBackoffWithJitterwithretryCount = Integer.MAX_VALUEthe client never gives up and never throws, so no timeout value could matter. That conflates the overall policy with a single attempt.Mqtt.connectbounds one CONNECT round trip withconnectToken.waitForCompletion(Mqtt.CONNECTION_TIMEOUT), andCONNECTION_TIMEOUTis 60000 - exactly the@Test(timeout = 60000)these tests declared. A stalled attempt and the test therefore expired on the same tick, so JUnit killed the thread precisely when the client would have thrown and let the retry run. Linux build 163287 shows the consequence: 60 seconds with not one connection status transition logged, because nothing was ever allowed to happen. The three overrides are removed here soIntegrationTests two minute rule applies, which leaves room for one stalled attempt to expire and a retry to follow. This does not stop the stalls, it stops a single stall being fatal.1. Wait for the test proxies to bind
HttpProxyServer.startAsync(int)returnsCompletionStage<Void>that completes once the port is listening. All four classes that stand up a local proxy discarded it. Because the e2e tests run in parallel, a test can start sending traffic before the proxy is accepting and get a connection refused from a proxy that is about to be perfectly healthy.ProxyServerTools.startProxyServerwaits on that future, capped at 30s so a proxy that never binds fails the run with a clear error instead of hanging.Applied to all five call sites across
ConnectionTests,FileUploadTests,MultiplexingClientTestsandTokenRenewalTests.2.
ConnectionTestswas leaking every client it openedConnectionTestInstance.dispose()was dead code — defined, never called. Four other test classes calltestInstance.dispose(); this one didn't. So everyCanOpenConnectionvariant leaked its client and its identity.Those clients keep retrying for the life of the JVM, and the proxied ones keep retrying through the proxies this class runs locally, competing with tests that are still running. Once
stopProxy()closes them they retry against a dead port for the rest of the job.This is the same leak #1861 fixed in
TokenRenewalTests, and it works directly against the parallelism cap added in #1862 — the entire point of that cap was to stop these proxies being starved. Now called from an@After.ECC identities have to be deleted, not recycled
Worth calling out, because it is a trap. Unlike every other identity here, the ECC ones are created by the test rather than drawn from the shared pool, and carry a self-signed cert only this test knows about.
disposeTestIdentityrecycles rather than deletes whenRECYCLE_TEST_IDENTITIESis set, and these areSELF_SIGNED— so recycling one would drop a device with an unknown thumbprint into the x509 pool for a later test to fail on. They are removed from the registry instead.3. Unused
@Mocked Objectin the provisioning testsContractAPIMqttTestdeclared@Mocked Object mockSendLockand@Mocked Integer mockedInteger;ContractAPIAmqpTestdeclared@Mocked Object mockSendLock. None of the three is referenced anywhere. Mockingjava.lang.Objectmakes JMockit retransform it, which is a hazard to the whole JVM rather than to one test.Being straight about the evidence: this is not currently failing. #1856 cited build 161517, but I could not reproduce it — the full provisioning suite passes on JDK 8 with reruns disabled, and no
ContractAPI*failure appears in the last seven CI builds. These are removed because they are unused and risky, not because they are breaking something today. If you would rather not touch them, that commit can be dropped without affecting the other two fixes.Verification
On JDK 8:
mvn -pl iot-e2e-tests/common -am test-compile— BUILD SUCCESSmvn -pl provisioning/provisioning-device-client test -Dsurefire.rerunFailingTestsCount=0— 544 tests, 0 failures, the same count as before the removalThe proxy bind wait and the client leak fix only show their effect in a live gated run, so those are exercised by the gate rather than locally.
Closes #1856.