fix(kubernetes): terminate idle computing units

[yrenat]

What changes were proposed in this PR?

Following discussion #6264, this PR adds backend-side cleanup for idle Kubernetes computing units.

The main change is a scheduled cleanup task in the computing unit managing service that periodically scans active Kubernetes computing units and terminates units that have been inactive longer than a configurable timeout.

The implementation includes the following changes:

• Added new Kubernetes configuration entries for:
  • computing unit idle timeout
  • computing unit idle check interval
• Exposed both settings through environment-variable-based configuration so deployment-side overrides can be applied without code changes.
• Added a scheduled background task in ComputingUnitManagingService that runs the idle cleanup logic at a fixed interval.
• Added idle Kubernetes computing unit termination logic in ComputingUnitManagingResource:
  • only considers Kubernetes computing units that are not already terminated
  • checks whether the computing unit has any active workflow executions
  • computes the latest execution activity timestamp from existing execution metadata
  • terminates the Kubernetes pod when the computing unit is considered idle past the configured timeout
  • updates the computing unit termination time in the database after cleanup

The timeout and check interval are configurable through environment variables, so the behavior can be tuned for different deployment or testing needs without modifying the code.

Any related issues, documentation, discussions?

Fixes #5362

How was this PR tested?

Tested locally on the Kubernetes deployment flow.

fix-idle-CU-demo.mp4

Was this PR authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex GPT-5

[github-actions]

👋 Thanks for opening this pull request, @yrenat!

It looks like the pull request description doesn't quite follow our template yet:

• The What changes were proposed in this PR? section is missing; please keep the template's headings.
• The How was this PR tested? section is missing; please keep the template's headings.
• The Was this PR authored or co-authored using generative AI tooling? section is missing; please keep the template's headings.

Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.

You can find the template prompts by editing the description, or see CONTRIBUTING.md for the full contribution flow.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Ma77Ball, @aglinxinyuan, @aicam
  You can notify them by mentioning @Ma77Ball, @aglinxinyuan, @aicam in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.63%. Comparing base (1677e43) to head (e551284).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6046      +/-   ##
============================================
- Coverage     60.68%   60.63%   -0.06%     
- Complexity     3367     3386      +19     
============================================
  Files          1134     1134              
  Lines         44167    44002     -165     
  Branches       4827     4779      -48     
============================================
- Hits          26802    26679     -123     
+ Misses        15907    15862      -45     
- Partials       1458     1461       +3     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	44.59% <ø> (ø)		Carriedforward from 82a57ee
amber	66.13% <100.00%> (-0.01%)	⬇️
computing-unit-managing-service	27.31% <100.00%> (+27.31%)	⬆️
config-service	52.30% <ø> (ø)
file-service	63.97% <ø> (ø)
frontend	51.87% <ø> (-0.63%)	⬇️	Carriedforward from 82a57ee
notebook-migration-service	78.57% <ø> (ø)
pyamber	91.17% <ø> (-0.02%)	⬇️	Carriedforward from 82a57ee
workflow-compiling-service	55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 3 better · 🔴 3 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main 1677e43 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	384	0.234	26,328/33,878/33,878 us	🔴 +19.7% / 🔴 +116.5%
🟢	bs=100 sw=10 sl=64	820	0.501	120,104/147,636/147,636 us	🟢 -8.7% / 🔴 +35.3%
⚪	bs=1000 sw=10 sl=64	968	0.591	1,028,719/1,123,023/1,123,023 us	⚪ within ±5% / 🔴 +7.4%

Baseline details

Latest main 1677e43 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	384 tuples/sec	445 tuples/sec	760.25 tuples/sec	-13.7%	-49.5%
bs=10 sw=10 sl=64	MB/s	0.234 MB/s	0.272 MB/s	0.464 MB/s	-14.0%	-49.6%
bs=10 sw=10 sl=64	p50	26,328 us	21,992 us	12,873 us	+19.7%	+104.5%
bs=10 sw=10 sl=64	p95	33,878 us	32,991 us	15,648 us	+2.7%	+116.5%
bs=10 sw=10 sl=64	p99	33,878 us	32,991 us	19,338 us	+2.7%	+75.2%
bs=100 sw=10 sl=64	throughput	820 tuples/sec	776 tuples/sec	974.75 tuples/sec	+5.7%	-15.9%
bs=100 sw=10 sl=64	MB/s	0.501 MB/s	0.474 MB/s	0.595 MB/s	+5.7%	-15.8%
bs=100 sw=10 sl=64	p50	120,104 us	131,587 us	102,470 us	-8.7%	+17.2%
bs=100 sw=10 sl=64	p95	147,636 us	154,654 us	109,144 us	-4.5%	+35.3%
bs=100 sw=10 sl=64	p99	147,636 us	154,654 us	115,530 us	-4.5%	+27.8%
bs=1000 sw=10 sl=64	throughput	968 tuples/sec	964 tuples/sec	1,000 tuples/sec	+0.4%	-3.2%
bs=1000 sw=10 sl=64	MB/s	0.591 MB/s	0.589 MB/s	0.61 MB/s	+0.3%	-3.2%
bs=1000 sw=10 sl=64	p50	1,028,719 us	1,040,789 us	1,005,545 us	-1.2%	+2.3%
bs=1000 sw=10 sl=64	p95	1,123,023 us	1,148,667 us	1,045,968 us	-2.2%	+7.4%
bs=1000 sw=10 sl=64	p99	1,123,023 us	1,148,667 us	1,076,800 us	-2.2%	+4.3%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,520.68,200,128000,384,0.234,26327.83,33878.45,33878.45
1,100,10,64,20,2438.88,2000,1280000,820,0.501,120103.98,147636.00,147636.00
2,1000,10,64,20,20667.69,20000,12800000,968,0.591,1028718.84,1123023.26,1123023.26

[yrenat]

/request-review @aicam

[chenlica]

In addition to @aicam , @kunwp1 please review it as well.

[aicam]

Please add a new column to workflow_computing_unit which is "termination_reason" and we should have two types of termination (as of this PR):

• USER_REQUESTED
• GARBAGE_COLLECTED

[aicam]

Please add a new column to workflow_computing_unit which is "termination_reason" and we should have two types of termination (as of this PR):

• USER_REQUESTED
• GARBAGE_COLLECTED

[yrenat]

this video shows the new effect:
I create 2 Computing Units, then delete 1 CU, and wait for another one to be auto-deleted. The log shows the output message for each case.

texera-PR6046.mp4

[chenlica]

@aicam @yrenat : we want to follow the practice of having a discussion before raising a PR. Can you create a discussion first?

[Yicong-Huang]

@yrenat please also have more tests. the coverage is too low.

[yrenat]

@aicam @yrenat : we want to follow the practice of having a discussion before raising a PR. Can you create a discussion first?

discussion added at: #6264

[yrenat]

@yrenat please also have more tests. the coverage is too low.

done. The coverage is now 100%

[chenlica]

@yrenat Before we continue this PR, can we first finish the discussion #6264 and draw a design conclusion?

[yrenat]

@yrenat Before we continue this PR, can we first finish the discussion #6264 and draw a design conclusion?

Yeah I'm working on that. Sorry for not replying promptly.