Add mongo evalya full-coverage fixture

[NouemanKHAL]

What does this PR do?

TL;DR: Adds a mongo-full evalya fixture (mongo/tests/evalya.yaml): the smallest sharded MongoDB 8.0 cluster that exercises both the mongos and the replica-set collectors, a seed, and a continuous workload that includes lock contention. Stacked on #25153 (onboard-evalya skill). Test fixtures only.

• Topology: 1-member config replica set; shard01 with a primary (shard-a) and a hidden delayed secondary (shard-b: priority 0, votes 0, secondaryDelaySecs 5) so replication lag and repl.* counters move; mongos; a socat entrypoint forwarding 27017 (mongos), 27018 (shard-a), 27019 (shard-b), same pattern as Add rabbitmq evalya full-coverage fixture #25224.
• seed.sh: adds the shard, creates the datadog user, seeds a pre-split sharded collection with indexes.
• activity-gen.sh: a mixed workload through mongos (inserts, indexed and scan queries, getMores, updates, aggregates, deletes, periodic DDL, a long-lived session), plus lock contention in lockdrill:
  • on shard-a: a slow $where writer holding intent locks, two fast writers and dbStats, and a holder every 3s running createIndexes/dropIndexes, dbHash, a cross-database renameCollection, and an fsync lock;
  • through mongos: a capped insert (Metadata X) and setUserWriteBlockMode on then off (Global X).
• logicalSessionRefreshMillis=10000, so sessions.count appears within a short run.
• README: inputs, the three-instance check config, zero-by-nature metrics, and the lock workload.

Knobs (Compose interpolation, read from the environment of the evalya run, e.g. LOCK_DRILL=0 evalya run ... or evalya run -e LOCK_DRILL=0 ...):

Variable	Default	Effect
MONGO_VERSION	8.0	Image tag for every mongo service (6.0+).
DB_USERNAME / DB_PASSWORD	datadog / datadog	Monitoring user created by seed.
ACTIVITY_GEN	1	0 keeps activity-gen up but idle.
LOCK_DRILL	1	0 skips the fsync lock and the setUserWriteBlockMode holder; the rest of the lock contention still runs.

Side effect of the lock drill: setUserWriteBlockMode with global: true blocks all user writes cluster-wide for its window, every 3s. That includes the fixture's own activity-gen iteration, the shard holder's index build (logged, restarted 5s later), and any consumer writing through the forwarded ports. Read-only scrapers are not blocked. LOCK_DRILL=0 opts out. An exit trap kills the holders and runs fsyncUnlock then setUserWriteBlockMode off, best effort, so stopping activity-gen mid-window does not leave the cluster blocked. ACTIVITY_DURATION (script-only, not passed by the compose file) is documented in the README.

Motivation

semantic-core's FTF federates *-full fixtures (redis-full, rabbitmq-full #25224) to compare the Datadog mongo check with the OTel mongodbreceiver. Mongo had no evalya fixture, and the existing mongo-shard.yaml compose (11 mongods, no healthchecks, docker exec init) is not usable as one. The lock workload exists because serverStatus reports locks.<type>.acquireWaitCount and timeAcquiringMicros only after an acquisition waits, which one-at-a-time traffic never causes, so the comparator had no data on either side for 27 lock metric pairs.

Validation

• Lock pairs: 21 of the 27 are non-zero on the shard primary, from both the mongo check and the mongodbreceiver (MongoDB 8.0.32, three 15s windows, LOCK_DRILL=1). The other 6 read modes 8.0 never acquires in steady state (Metadata R, oplog R, and waits on oplog w); the README records why.
• Coverage, measured before the lock workload was added (not re-measured on this head):
  • dashboard and monitor metrics: 29/29 emitted, 25 non-zero with agent check mongo (Agent 7.83.1, three instances: mongos, primary, secondary). The 4 zeros: chunks.jumbo, globallock.currentqueue.{readers,writers}, extra_info.page_faultsps. The two queue gauges are point-in-time, and the lock workload queues operations only in ~100ms bursts, so they are still expected to read 0 at scrape time; that expectation is reasoned, not measured.
  • metadata.csv: 221/328 emitted, 162 non-zero.
  • OTel mongodbreceiver (otelcol-contrib 0.161.0, all metrics enabled): 50/52 emitted, 43 non-zero.
• LOCK_DRILL, live on this head:
  • LOCK_DRILL=1: shard-a serverStatus().locks showed non-zero acquireWaitCount for Global, Database, and Collection in all four modes. With an fsync lock held by hand, docker stop on activity-gen returned in 0.5s, logged lock drill released, the fsyncLockWorker lock was gone, and a write through mongos succeeded.
  • LOCK_DRILL=0: the container got LOCK_DRILL=0, the log showed lock drill disabled and 40+ iterations with no UserWritesBlocked and no failures. Database, Collection, and Metadata lock fields were still populated; the Global wait fields were absent, since the drill's holders are the only Global S/X requests.
• evalya manifest validate --path mongo/tests/evalya.yaml: OK (warnings: ignored restart: policies, no registry mirror).
• ddev --no-interactive test --lint mongo: pass. Tests-only change, no changelog.
• Not verified: consumption through an OCI federation context, non-default MONGO_VERSION, ACTIVITY_GEN=0.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

🤖 Generated with Claude Code

[cit-pr-commenter-54b7da]

evalya-impact-summary

evalya impact analysis

Impact analysis: 0 selected, 0 skipped (of 0 test tasks)
Publish tasks:   3 (always emitted)
Diff (11 files):
  .claude/skills/onboard-evalya/README.md
  .claude/skills/onboard-evalya/SKILL.md
  .claude/skills/onboard-evalya/references/coverage-loop.md
  .claude/skills/onboard-evalya/references/redis-exemplar.md
  .claude/skills/onboard-evalya/scripts/asset_metrics.py
  .gitignore
  mongo/tests/README.md
  mongo/tests/activity-gen.sh
  mongo/tests/compose/full-coverage.compose
  mongo/tests/evalya.yaml
  mongo/tests/seed.sh

Debug a specific task: evalya plan impact --path <path> --task <task>

Learn more about CI impact filtering

[datadog-official]

 

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 91.85%

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6715aea | Docs | View more details | Give us feedback!}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-30T14:10:24.397961Z	b2e9c68	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2e9c68581

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run