database-monitoring: run Elastio's encryption detector beside the agent (content_analysis)

[anchoo2kewl]

Tracking: elastio/blue-stack#17198

What changed

Adds content_analysis (default false) to aws-elastio-database-monitoring-agent and bumps the module to 0.2.0.

With it on, Elastio's encryption detector (ED) runs as a second container in the agent's task:

• Where it comes from: ed_image, built by elastio/database-monitoring-agent from its pinned elastio/ed commit for arm64 and amd64. The task stays ARM64.
• User: it runs as the agent's user (65532), which alone can read the samples.
• Network: it listens on 127.0.0.1:50051, loopback only.
• Scratch volume: it reads a task-local volume at /var/lib/elastio-dbmon-ed, read-only, mounted at the same path as in the agent, which writes it. It is never EFS.
• Not essential: if ED stops, the agent judges without a verdict (which can raise a finding and never withholds one), and restartPolicy brings ED back.
• Memory:
  • ED gets 384 MiB, measured on the arm64 image with one worker: 158 MiB idle and 231 MiB after a window.
  • The agent's container is limited to the rest, and GOMEMLIMIT is 80% of that share, not of the task.
  • content_analysis needs task_memory >= 1024, enforced by a validation.

Defaults move to agent 0.1.7 and elastio-database-monitoring-ed:0.1.7. With content_analysis = false the task definition is unchanged apart from the image tag.

Order

Merge after agent 0.1.7 is released. The new default image tags only exist once agent-v0.1.7 is tagged, and the ED image also needs the ECR repo from elastio/infraworld#2399. The agent change is elastio/database-monitoring-agent#5.

Verified

• terraform fmt -check, terraform validate, and terraform test: 17/17, 3 of them new:
  • on: two containers, the detector runs as 65532 and is not essential, it mounts the scratch volume read-only, the scratch volume is task-local, the agent's memory is 640, GOMEMLIMIT is 512MiB, and ELASTIO_DBMON_ED_ADDR is set;
  • off: one container, no ED variables, no scratch volume;
  • task_memory = 512 with content_analysis is refused.
• Seeded defect: putting GOMEMLIMIT back to 80% of the task fails the "on" run.
• terraform-docs 0.19.0 plus prettier regenerated the README, so the terraform-docs job has no diff.
• The same ED image ran natively on arm64 as uid 65532 with --workers 1 and passed the agent's gRPC contract tests.

Not verified

• It hasn't been applied anywhere yet. UAT is next, after the agent release.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[maksvet]

@anchoo2kewl which ED server does the image run, Python or Rust?
Is the metrics port also on loopback only?
The Rust server binds metrics to --host, so it is fine.
The Python server calls start_http_server(port) with no address, so it listens on 0.0.0.0.
If it is Python, 50052 is open on the task ENI to anything the security group lets in.
Then we can fix it in ED by passing the host, or change the "loopback only" comment and say that the security group must block the port.

[anchoo2kewl]

It's the Python server: the image is ED's own Dockerfile at 624e66b, which ships grpc_server. You're right about metrics. On the image with these flags, 50051 is on 127.0.0.1 and 50052 is on 0.0.0.0, from start_http_server(port) in grpc_server/metrics.py. ED master has the same line. I've fixed the comment and README in 1146458: the metrics port is on all interfaces, so the security groups must not allow inbound 50052 (the agent needs no inbound rules; UAT's task SG has none). I'll open the one-line fix in elastio/ed to bind metrics to --host, and we pick it up with the next pin bump.

[anchoo2kewl]

@maksvet the ED fix is open: elastio/ed#42. It binds the Prometheus metrics server to --host (and serves the collector's own registry). I built the image from that branch and ran it with this module's flags: 50051 and 50052 are both on 127.0.0.1. Once it merges, the pin bump comes through as a Dependabot PR on elastio/database-monitoring-agent (third_party/ed), and CI tests it. Until then, the README says the security groups must not allow inbound 50052. Noted on merge order: after agent 0.1.7 is released and the ED ECR repo exists.

[maksvet]

Thanks, 1146458 looks good.
Please link the elastio/ed fix here when it is open, so we remember to bump the pin.
Merge only after agent 0.1.7 is released and the ED ECR repo exists.