Repository navigation

feat(iceberg): native Iceberg scans of HDFS-backed tables via iceberg-rust's hdfs-native backend - #5898

Open

mixermt wants to merge 44 commits into

apache:mainfrom

mixermt:feat/iceberg-hdfs-native

Open

feat(iceberg): native Iceberg scans of HDFS-backed tables via iceberg-rust's hdfs-native backend#5898
mixermt wants to merge 44 commits into
apache:mainfrom
mixermt:feat/iceberg-hdfs-native

Commits on Sep 13, 2026
- feat: read and write Iceberg tables on HDFS via iceberg-rust's hdfs-native backend
  
  mixermt
  and
  claude
  committed
  1e818b0
- docs: trim duplicated commentary around the HDFS NameNode resolution
  
  mixermt
  and
  claude
  committed
  4d80b05
- test: skip the Iceberg HDFS suite when MiniDFSCluster cannot start
  
  mixermt
  and
  claude
  committed
  9c06f1d
- fix: match hadoop-client-minicluster to the hadoop each Spark profile supplies
  
  mixermt
  and
  claude
  committed
  c87dcf9
Commits on Sep 15, 2026
- Merge branch 'main' into feat/iceberg-hdfs-native
  mixermt
  authored
  5cadaa5
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  mixermt
  authored
  e6668ea
Commits on Sep 19, 2026
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  cursoragent
  committed
  abc4945
- fix: report task input metrics for fused and shuffled native Iceberg scans
  
  mixermt
  and
  claude
  committed
  19ee804
- fix: register the native shuffle input bridge ahead of nested inputs
  
  mixermt
  and
  claude
  committed
  2742d8e
- feat: show Iceberg planning metrics on the native scan node in the SQL UI
  
  mixermt
  and
  claude
  committed
  91f0587
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  4b04395
Commits on Sep 21, 2026
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  2ef6512
Commits on Oct 6, 2026
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  8f22fd1
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  1e2bcd9
- style: wrap the resolve-needed fallback reason under the 100-column scalastyle limit
  
  mixermt
  and
  claude
  committed
  b8ec1fa
Commits on Oct 7, 2026
- chore: move the iceberg-rust pin to the latest head of apache/iceberg-rust#3111
  
  mixermt
  and
  claude
  committed
  29c92e3
Commits on Oct 11, 2026
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  5f03cf6
Commits on Oct 23, 2026
- Merge branch 'apache:main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  c906878
- feat(iceberg): read Iceberg tables on HDFS natively via iceberg-rust's hdfs-native backend
  
  mixermt
  and
  claude
  committed
  70df82a
- fix(iceberg): fall back for name-mapped tables that read list or map columns
  
  mixermt
  and
  claude
  committed
  1a610fa
- Merge branch 'main' into feat/iceberg-hdfs-native
  mixermt
  authored
  b493b62
- fix(iceberg): decline scans that mix storage backends or read HDFS encryption zones
  
  mixermt
  and
  claude
  committed
  12f9d33
Commits on Oct 24, 2026
- Merge remote-tracking branch 'mixermt/feat/iceberg-hdfs-native' into feat/iceberg-hdfs-native
  mixermt
  committed
  5858078
- Merge branch 'main' into feat/iceberg-hdfs-native
  mixermt
  authored
  d562bc8
Commits on Oct 27, 2026
- Merge branch 'main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  bb871c7
- fix(iceberg): claim HDFS scans only as the Spark user, batch the encryption-zone probe, default the NameNode port
  
  mixermt
  and
  claude
  committed
  17c0872
- feat(iceberg): move the iceberg-rust pin past apache/iceberg-rust#3437
  
  mixermt
  and
  claude
  committed
  fd1cf58
Commits on Oct 28, 2026
- chore(iceberg): drop the iceberg-rust#3437 note from the pin comment
  
  mixermt
  and
  claude
  committed
  c6c7312
Commits on Oct 30, 2026
- Merge branch 'main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  81d5e29
Commits on Oct 31, 2026
- Merge branch 'main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  5ea3ba4
- test(iceberg): create the HDFS encryption zone with the non-deprecated overload
  
  mixermt
  and
  claude
  committed
  d35847b
- test(iceberg): prove the native HDFS client reads as the Spark user
  
  mixermt
  and
  claude
  committed
  ffbdf18
- feat(iceberg): forward the HDFS client settings the session sets to the native reader
  
  mixermt
  and
  claude
  committed
  c1e96d3
- fix(iceberg): keep the HDFS encryption-zone listings per parent, with a TTL and a budget
  
  mixermt
  and
  claude
  committed
  a302824
- test(iceberg): read a merge-on-read table's position deletes over HDFS
  
  mixermt
  and
  claude
  committed
  4bf68ae
Commits on Nov 5, 2026
- Merge branch 'main' into feat/iceberg-hdfs-native
  
  mixermt
  and
  claude
  committed
  94ee908
- fix(iceberg): read the executors' HADOOP_USER_NAME from the SparkContext, not the session
  
  mixermt
  and
  claude
  committed
  984e91d
- fix(iceberg): keep HDFS encryption-zone listings per table and for an hour by default
  
  mixermt
  and
  claude
  committed
  a5d2680
Commits on Nov 6, 2026
- Merge remote-tracking branch 'origin/main' into feat/iceberg-hdfs-native
  mixermt
  committed
  ce24a0c
- Merge remote-tracking branch 'origin/main' into feat/iceberg-hdfs-native
  mixermt
  committed
  04cfc9e
- fix(iceberg): give getMetadataLocation its Scaladoc back
  
  mixermt
  and
  claude
  committed
  8a3a6b7
- fix(iceberg): probe encryption zones in linear time and bound what is kept by entries
  
  mixermt
  and
  claude
  committed
  f2c8a73
- feat(iceberg): add spark.comet.scan.icebergNative.hdfs.enabled
  
  mixermt
  and
  claude
  committed
  ce10dde
- Merge remote-tracking branch 'origin/main' into feat/iceberg-hdfs-native
  mixermt
  committed
  6bca678

Commits

Commits on Sep 13, 2026

Commits on Sep 15, 2026

Commits on Sep 19, 2026

Commits on Sep 21, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026

Commits on Oct 11, 2026

Commits on Oct 23, 2026

Commits on Oct 24, 2026

Commits on Oct 27, 2026

Commits on Oct 28, 2026

Commits on Oct 30, 2026

Commits on Oct 31, 2026

Commits on Nov 5, 2026

Commits on Nov 6, 2026