[Better Tests] Adds the GCE metadata fake

[IvanBM18]

Bug: b/555370359

Creates a GCE metadata emulator for our future integration test suite, built on Testcontainers and WireMock. It lets tests resolve credentials and read instance and project metadata without a real GCE host or a local gcloud login, and lets them distinguish a trusted from an untrusted environment.

google-auth and our vendored oauth2client evaluate the GCE_METADATA_* environment variables once, at import time, and bake the resulting URL in. Importing this module therefore claims two loopback addresses up front; the addresses never move, only what listens on them does. A test picks its in-process identity by picking which fixture answers on the default address:

with gce_metadata_emulator.trusted_as_default() as tworker:
  ...
with gce_metadata_emulator.untrusted_as_default() as uworker:
  ...
with gce_metadata_emulator.trusted_untrusted_pair() as (tworker, uworker):
  ...  # uworker sits on the secondary address

Nothing is patched to make that work. An emulator on the secondary address is reachable through the client its context manager yields, or by handing its env to a child process:

subprocess.run(argv, env={**os.environ, **uworker.env})

This emulator was based off the pre-existing local emulator go wrap + what is detailed at the GCP metadata server docs

Changes

• Add gce_metadata_emulator.py, a WireMock Testcontainer fake GCE metadata server. Supports:
  • Serving fake service account credentials, so a test can tell a trusted from an untrusted environment.
  • Serving project- and instance-scoped metadata values, including attribute shadowing, seeded from the JSON configs added in [Better Tests] Add GCE metadata emulator worker/uworker configs #5485.
  • Rejecting requests that omit Metadata-Flavor: Google, the way the real server does.
  • Failing loudly when two emulators want the same address.

The tests covering all of this land in #5486, next in the stack.

📚 Stacked PRs

Step	PR	Title	Base Branch
0	#5475	[Better Tests] Adds new test dependencies	master
1	#5479	[Better Tests] Support GCE_METADATA_HOST with port in compute_metadata	better-test/dependencies
2	#5485	[Better Tests] Add GCE metadata emulator worker fixtures	better-tests/01-compute-metadata-host
3	#5480	[Better Tests] Add the WireMock GCE metadata emulator (👉 This PR)	better-tests/01a-metadata-emulator-configs
4	#5486	[Better Tests] Bootstrap and cover the GCE metadata emulator	better-tests/02-metadata-emulator-core
5	#5481	[Better Tests] Add fault injection and runtime overrides to metadata emulator	better-tests/02a-metadata-emulator-tests

[dylanjew]

This is a bit hard to read. Could you focus this documentation on what this fake does and how it should be used?

[IvanBM18]

Sure, done!

[dylanjew]

i've also seen test servers hardcode ports and throw an error if it's already in use. There's also a bunch of handling for if the port is in use in this class. Do we need to do both? would it be simpler to also hardcode here?

[IvanBM18]

mmm no we don't need both port finding + port error handling, I added them so we could have better error msgs, but I guess its overcomplicating this code. SO in favor of simplicity I'll also add hardcoded ports as well, maybe we can come back to this idea later if we want to speed up testing times by launching multiple concurrent tests.

[IvanBM18]

Made the selection of hardcoded ports based on:
https://github.com/google/clusterfuzz/blob/master/src/local/butler/constants.py#L93

[dylanjew]

thanks, this is simpler! is there an error message thrown if the port is in use?

[IvanBM18]

Yes, when standing up the server it will throw an exception, theres a test for that in our test pr

[IvanBM18]

Needed so:
Imports like from test_suites.fixtures import ...
Resolve cleanly, we can later remove this once we figure out the butler.py command for the new test suite

[IvanBM18]

Dito ^
Needed so:
Imports like from test_suites.fixtures import ...
Resolve cleanly, we can later remove this once we figure out the butler.py command for the new test suite

[dylanjew]

LGTM