Antalya 26.6 - Fixed refreshing of glue metadata after ALTER calls

[subkanthi]

#2085

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fixed an issue where Glue DataLakeCatalog tables could retain stale schema metadata after ALTER TABLE operations.

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [61c7e4a]

[subkanthi]

@blau-ai

[blau-ai]

CI triage — PR #2272

Verdict: 0 failures. CI is not red — it just hasn't produced results yet.

The current head (15fbfac, the "Merge branch 'antalya-26.6' into …" commit) triggered a fresh run (32988139141) that started only a few minutes ago. As of now:

• Config Workflow → ✅ OK
• Fast test, Source upload → 🟡 running
• All builds / stateless / integration / unit / stress / fuzzer jobs → ⏳ pending (or SKIPPED for the arm/tsan/msan variants this PR's config excludes)
• DCO → ✅ pass

There are no failure/error check-runs, and the praktika report (result_pr.json) shows the workflow as RUNNING with nothing failed. The earlier commit 6c7b9aa has no uploaded report, so this is the first real CI pass for the branch — there's simply nothing to classify as PR-caused vs. pre-existing/flaky yet. Re-run this triage once the run finishes and I'll break down any failures.

---

One-paragraph health check of the change

The diff is tightly scoped to the reported bug (src/Databases/DataLake/GlueCatalog.cpp + one integration test). GlueCatalog::updateSchema previously only rewrote metadata_location and dropped new_schema on the floor (/*new_schema*/), which is exactly why Glue kept serving a stale StorageDescriptor.Columns after ALTER … ADD/DROP COLUMN; the fix now rebuilds the TableInput and pushes the columns via the new setGlueColumnsFromIcebergSchema / icebergTypeToGlueType helpers (with recursive array/map/struct handling and iceberg.field.* column parameters), and createTable now also stops discarding metadata_content. The change looks coherent and the added test_schema_evolution_show_create_and_drop is a real reproducer (SHOW CREATE + DROP COLUMN round-trip). It's gated behind the Glue integration test, which needs a full build + running server — so correctness will only be proven by CI on this run, not statically. Two things worth a glance while CI runs: (1) getCurrentSchemaFromMetadata calls metadata->getValue<Int32>(f_current_schema_id) / getArray(f_schemas) unguarded — fine for well-formed Iceberg metadata, but it will throw on a metadata object missing those keys; (2) updateSchema now constructs a fresh TableInput rather than read-modify-write, so make sure it isn't dropping any table parameters that the previous updateMetadata path preserved (it sets metadata_location + table_type, matching createTable, so this is likely intentional). Neither is a blocker — just verify the test_database_glue integration shard goes green.

Automated triage by @blau-ai — no build/test run locally; verdict is based on GitHub check-runs + the praktika S3 report for 15fbfac.

[subkanthi]

TESTING

alter table datalake_db.`db1.table22` add column column_c Nullable(String);

ALTER TABLE datalake_db.`db1.table22`
    (ADD COLUMN `column_c` Nullable(String))

Query id: 6d3a20fc-21f7-45e5-a87b-239aa14bad8c

Ok.

0 rows in set. Elapsed: 0.148 sec. 

Ubuntu-2404-noble-amd64-base :) show create table datalake_db.`db1.table22`;

SHOW CREATE TABLE datalake_db.`db1.table22`

Query id: 5b699d67-2300-49ea-9fe3-91199ff0cff1

   ┌─statement──────────────────────────────────────────────────────┐
1. │ CREATE TABLE datalake_db.`db1.table22`                        ↴│
   │↳(                                                             ↴│
   │↳    `name` Nullable(String),                                  ↴│
   │↳    `double` Nullable(Float64),                               ↴│
   │↳    `integer` Nullable(Int64),                                ↴│
   │↳    `column_a` Nullable(String),                              ↴│
   │↳    `column_b` Nullable(String),                              ↴│
   │↳    `column_c` Nullable(String)                               ↴│
   │↳)                                                             ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/db1/table22/') │
   └────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.004 sec. 

[DimensionWieldr]

CI Failures Analysis

Verdict: no PR-caused regressions. Head 61c7e4a6, run 33412728540.

The PR’s own test test_database_glue/test.py::test_schema_evolution_show_create_and_drop PASSED in targeted integration.

Related to this PR

None.

Pre-existing / unrelated

• test_schema_inference (12 params, asan/ubsan) — known #2216. First param kills the server; the rest are cascades. Same fails on sibling PRs / MasterCI.
• test_cas_gcs/...::test_a_write_whose_response_carries_no_generation_is_refused — CAS GCS assertion, also on PR Antalya 26.6: Added support for alter add column with first/after #2289.
• Settings export_merge_tree_part_ignore_extra_source_columns / export_merge_tree_part_schema_match_mode — SnapshotNotFoundError after #2284; regression snapshots not updated yet. Same 2 fails on Antalya 26.6: Added support for alter add column with first/after #2289.
• Stress test (amd_debug) Cannot start clickhouse-server — Hybrid currentDatabase() restart bug, #1993. asan_ubsan stress passed.
• 04068_lwd_minmax_projection_recovery_rmt + Server died — CAS S3 asan LWD mutation (FILE_DOESNT_EXIST on dropRef). Cascade after that. amd_binary CAS S3 was green.

Flaky / infra

• /tiered storage/with cas/background move — other tiered jobs passed.
• cas_s3_cache_alter_attach_1 — 157 attach-partition fails; non-CAS alter_attach_1 passed.
• Cancelled long jobs (cas_alter_attach_1, LWD 2/2 shards, stateless CAS S3 1/2) — timeouts, not test failures.

Recommendations

1. Merge-ready from this PR’s Glue catalog changes.
2. Do not expect iceberg test_schema_inference to go green on rerun (antalya-26.6: test_schema_inference kills the server on amd_asan_ubsan — upstream Decimal-bounds overflow exposed by #2145 #2216).
3. Settings red clears once clickhouse-regression snapshots include the two export-part settings from Antalya 26.6 allow non matching schema export partition by name #2284.

[DimensionWieldr]

Ran https://github.com/Altinity/clickhouse-regression/blob/main/iceberg/tests/iceberg_engine/alter_support.py on Glue catalog, and all passed.

LGTM