refactor(workflow-core): remove unused hadoop Iceberg catalog

[anthonychengit]

What changes were proposed in this PR?

This PR removes the unused hadoop Iceberg catalog type.

Changes:

• Remove the hadoop catalog branch from IcebergCatalogInstance.
• Remove IcebergUtil.createHadoopCatalog.
• Remove the unused HadoopCatalog import while keeping HadoopFileIO.
• Update IcebergTableWriterSpec to use the supported postgres catalog helper.
• Update the storage.conf catalog-type comment from hadoop, rest, or postgres to rest or postgres.

Any related issues, documentation, discussions?

Closes #6150

How was this PR tested?

Tested with:

• sbt "WorkflowCore / scalafmtOnly"
• sbt "WorkflowCore / compile"

Also attempted:

• sbt "WorkflowCore / testOnly org.apache.texera.amber.core.storage.result.iceberg.IcebergTableWriterSpec"

Was this PR authored or co-authored using generative AI tooling?

Generated-by: ChatGPT was used for planning and review assistance. I reviewed and tested the final code before submission.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @mengw15, @aglinxinyuan, @Ma77Ball
  You can notify them by mentioning @mengw15, @aglinxinyuan, @Ma77Ball in a comment.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Ma77Ball, @mengw15, @Yicong-Huang
  You can notify them by mentioning @Ma77Ball, @mengw15, @Yicong-Huang in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.56%. Comparing base (39db110) to head (03db543).

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6315      +/-   ##
============================================
- Coverage     70.57%   70.56%   -0.02%     
+ Complexity     3425     3424       -1     
============================================
  Files          1146     1146              
  Lines         45062    45050      -12     
  Branches       4960     4959       -1     
============================================
- Hits          31802    31788      -14     
+ Misses        11613    11612       -1     
- Partials       1647     1650       +3     

Flag	Coverage Δ
access-control-service	70.00% <ø> (ø)
agent-service	76.76% <ø> (ø)
amber	66.71% <ø> (-0.01%)	⬇️
computing-unit-managing-service	17.72% <ø> (ø)
config-service	52.30% <ø> (ø)
file-service	66.80% <ø> (ø)
frontend	69.34% <ø> (-0.03%)	⬇️
notebook-migration-service	78.94% <ø> (ø)
pyamber	91.74% <ø> (ø)
workflow-compiling-service	55.14% <ø> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 5 worse · ⚪ 10 noise (<±5%) · 0 without baseline

Compared against main 39db110 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	404	0.247	24,154/35,389/35,389 us	🔴 -8.8% / 🔴 +146.4%
⚪	bs=100 sw=10 sl=64	824	0.503	120,496/150,629/150,629 us	⚪ within ±5% / 🔴 +52.0%
⚪	bs=1000 sw=10 sl=64	945	0.577	1,059,841/1,095,129/1,095,129 us	⚪ within ±5% / 🔴 +16.5%

Baseline details

Latest main 39db110 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	404 tuples/sec	443 tuples/sec	831.47 tuples/sec	-8.8%	-51.4%
bs=10 sw=10 sl=64	MB/s	0.247 MB/s	0.27 MB/s	0.507 MB/s	-8.5%	-51.3%
bs=10 sw=10 sl=64	p50	24,154 us	22,446 us	11,844 us	+7.6%	+103.9%
bs=10 sw=10 sl=64	p95	35,389 us	33,445 us	14,359 us	+5.8%	+146.4%
bs=10 sw=10 sl=64	p99	35,389 us	33,445 us	18,888 us	+5.8%	+87.4%
bs=100 sw=10 sl=64	throughput	824 tuples/sec	810 tuples/sec	1,089 tuples/sec	+1.7%	-24.3%
bs=100 sw=10 sl=64	MB/s	0.503 MB/s	0.495 MB/s	0.665 MB/s	+1.6%	-24.3%
bs=100 sw=10 sl=64	p50	120,496 us	121,593 us	92,503 us	-0.9%	+30.3%
bs=100 sw=10 sl=64	p95	150,629 us	153,862 us	99,121 us	-2.1%	+52.0%
bs=100 sw=10 sl=64	p99	150,629 us	153,862 us	112,422 us	-2.1%	+34.0%
bs=1000 sw=10 sl=64	throughput	945 tuples/sec	944 tuples/sec	1,117 tuples/sec	+0.1%	-15.4%
bs=1000 sw=10 sl=64	MB/s	0.577 MB/s	0.576 MB/s	0.682 MB/s	+0.2%	-15.4%
bs=1000 sw=10 sl=64	p50	1,059,841 us	1,056,240 us	910,046 us	+0.3%	+16.5%
bs=1000 sw=10 sl=64	p95	1,095,129 us	1,143,709 us	950,716 us	-4.2%	+15.2%
bs=1000 sw=10 sl=64	p99	1,095,129 us	1,143,709 us	985,832 us	-4.2%	+11.1%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,494.75,200,128000,404,0.247,24154.41,35388.60,35388.60
1,100,10,64,20,2427.48,2000,1280000,824,0.503,120495.64,150628.51,150628.51
2,1000,10,64,20,21170.34,20000,12800000,945,0.577,1059841.03,1095129.35,1095129.35