Skip to content

Commit b2aed36

Browse files
lfranckelfrancke
committed
feat(hive): Restore the Thrift methods removed by HIVE-26537 for 4.2.0
Hive 4.0.1 removed get_table and get_table_objects_by_name from the metastore Thrift interface. Any client built against a Hive 2.3 metastore client still calls them, and that is what Spark bundles: 2.3.9 in Spark 3.5, 2.3.10 in Spark 4.0 through 4.2. Iceberg's HiveCatalog reaches the metastore through IMetaStoreClient and therefore uses whichever client Spark supplies, so an Iceberg job against a Hive 4.0.1 or newer metastore fails. This restores the methods as they were in Hive 4.0.0.
1 parent 9ee33bf commit b2aed36

4 files changed

Lines changed: 2585 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ All notable changes to this project will be documented in this file.
1010
- nifi: Backport NIFI-15958 to log periodic progress while waiting for the content archive scan and provenance re-index, for `2.6.0`, `2.7.2`, and `2.9.0` ([#1611]).
1111
- hbase: Add an SBOM for the web UI (npm) dependencies, which are unpacked from webjars and therefore not covered by the CycloneDX Maven plugin ([#1620]).
1212
- trino: Add SBOMs for the web UI, both for the two npm projects behind it and for the pre-built JavaScript vendored into the source tree ([#1620]).
13+
- hive: Restore the `get_table` and `get_table_objects_by_name` Thrift methods that HIVE-26537 removed in Hive 4.0.1, for `4.2.0` ([#1636]).
1314
- hadoop, spark: Add SBOMs for the pre-built JavaScript that is vendored into the source tree for the HDFS and Spark web UIs ([#1620]).
1415

1516
### Changed
@@ -22,6 +23,8 @@ All notable changes to this project will be documented in this file.
2223
- superset: Fix the broken builds by excluding the `cypress-base` end-to-end test project from the frontend SBOM ([#1616]).
2324
- superset: Fix the broken 4.1.4 build by also excluding `packages/superset-ui-switchboard` from the frontend SBOM ([#1620]).
2425
- superset: Install nvm into `/opt/nvm` so that Node and npm, which are only needed to build the frontend, are no longer shipped in the final image (about 161 MB) ([#1623]).
26+
- hive: Build against the Hive modules built alongside each other (in the same reactor) rather than the ones published on Maven Central, for `4.2.0`.
27+
Upstream fixed one instance of this for Hive 4.3.0 in [HIVE-29827](https://issues.apache.org/jira/browse/HIVE-29827) but there are others. ([#1636]).
2528

2629
### Removed
2730

@@ -1321,3 +1324,4 @@ All notable changes to this project will be documented in this file.
13211324
[#249]: https://github.com/stackabletech/docker-images/pull/249
13221325
[#250]: https://github.com/stackabletech/docker-images/pull/250
13231326
[#268]: https://github.com/stackabletech/docker-images/pull/268
1327+
[#1636]: https://github.com/stackabletech/docker-images/pull/1636

‎hive/Dockerfile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,9 +83,13 @@ elif [[ "${PRODUCT_VERSION}" == 4.0.* ]]; then
8383
)
8484
else
8585
# Starting with 4.1.0 the build process changed again in https://github.com/apache/hive/pull/5936 (HIVE-29062)
86+
# The main reactor pins its dependencies on the standalone-metastore modules through `standalone-metastore.version`.
87+
# This is _not_ changed by `mvn versions:set` above.
88+
# If we do not change it here it'd download those dependencies (without the -stackableXXXX suffix) from Maven central.
8689
mvn \
8790
clean package \
8891
-Dhadoop.version=${HADOOP_VERSION}-stackable${RELEASE_VERSION} \
92+
-Dstandalone-metastore.version=${NEW_VERSION} \
8993
-DskipTests \
9094
-Pdist
9195
# Looks like we can not filter the projects using "--projects standalone-metastore/metastore-server --also-make",

‎hive/stackable/patches/4.2.0/0005-Resolve-every-in-reactor-dependency-against-the-modu.patch‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
From a8aada47897543df0ba008655852e16aa5ede477 Mon Sep 17 00:00:00 2001
2+
From: Lars Francke <git@lars-francke.de>
3+
Date: Fri, 11 Sep 2026 14:48:51 +0200
4+
Subject: Resolve every in-reactor dependency against the modules built
5+
alongside it
6+
7+
standalone-metastore refers to several modules built in the same reactor through the literal
8+
hive.version property rather than ${project.version}.
9+
We use "mvn versions:set" to stamp our Stackable release version and we also expect Hive to
10+
then honor that. This patch makes sure it does (plus a change in the Dockerfile)
11+
12+
Upstream fixed one instance of this, the metastore-server to metastore-client reference, for
13+
Hive 4.3.0: https://issues.apache.org/jira/browse/HIVE-29827.
14+
---
15+
standalone-metastore/metastore-rest-catalog/pom.xml | 10 +++++-----
16+
standalone-metastore/metastore-server/pom.xml | 2 +-
17+
.../metastore-tools/metastore-benchmarks/pom.xml | 2 +-
18+
standalone-metastore/metastore-tools/pom.xml | 2 +-
19+
standalone-metastore/packaging/pom.xml | 6 +++---
20+
5 files changed, 11 insertions(+), 11 deletions(-)
21+
22+
diff --git a/standalone-metastore/metastore-rest-catalog/pom.xml b/standalone-metastore/metastore-rest-catalog/pom.xml
23+
index ad6cecb535..8a4054422f 100644
24+
--- a/standalone-metastore/metastore-rest-catalog/pom.xml
25+
+++ b/standalone-metastore/metastore-rest-catalog/pom.xml
26+
@@ -34,31 +34,31 @@
27+
<dependency>
28+
<groupId>org.apache.hive</groupId>
29+
<artifactId>hive-standalone-metastore-server</artifactId>
30+
- <version>${hive.version}</version>
31+
+ <version>${project.version}</version>
32+
<scope>provided</scope>
33+
</dependency>
34+
<dependency>
35+
<groupId>org.apache.hive</groupId>
36+
<artifactId>hive-standalone-metastore-common</artifactId>
37+
- <version>${hive.version}</version>
38+
+ <version>${project.version}</version>
39+
</dependency>
40+
<dependency>
41+
<groupId>org.apache.hive</groupId>
42+
<artifactId>hive-iceberg-catalog</artifactId>
43+
- <version>${hive.version}</version>
44+
+ <version>${project.version}</version>
45+
</dependency>
46+
<!-- Test dependencies -->
47+
<dependency>
48+
<groupId>org.apache.hive</groupId>
49+
<artifactId>hive-standalone-metastore-common</artifactId>
50+
- <version>${hive.version}</version>
51+
+ <version>${project.version}</version>
52+
<classifier>tests</classifier>
53+
<scope>test</scope>
54+
</dependency>
55+
<dependency>
56+
<groupId>org.apache.hive</groupId>
57+
<artifactId>hive-standalone-metastore-server</artifactId>
58+
- <version>${hive.version}</version>
59+
+ <version>${project.version}</version>
60+
<classifier>tests</classifier>
61+
<scope>test</scope>
62+
</dependency>
63+
diff --git a/standalone-metastore/metastore-server/pom.xml b/standalone-metastore/metastore-server/pom.xml
64+
index 0e24b3ebc9..5ffdb4047f 100644
65+
--- a/standalone-metastore/metastore-server/pom.xml
66+
+++ b/standalone-metastore/metastore-server/pom.xml
67+
@@ -28,7 +28,7 @@
68+
<dependency>
69+
<groupId>org.apache.hive</groupId>
70+
<artifactId>hive-standalone-metastore-client</artifactId>
71+
- <version>${hive.version}</version>
72+
+ <version>${project.version}</version>
73+
</dependency>
74+
<dependency>
75+
<groupId>org.apache.httpcomponents</groupId>
76+
diff --git a/standalone-metastore/metastore-tools/metastore-benchmarks/pom.xml b/standalone-metastore/metastore-tools/metastore-benchmarks/pom.xml
77+
index 44281104b1..b6a1037574 100644
78+
--- a/standalone-metastore/metastore-tools/metastore-benchmarks/pom.xml
79+
+++ b/standalone-metastore/metastore-tools/metastore-benchmarks/pom.xml
80+
@@ -46,7 +46,7 @@
81+
<dependency>
82+
<groupId>org.apache.hive.hcatalog</groupId>
83+
<artifactId>hive-hcatalog-server-extensions</artifactId>
84+
- <version>${hive.version}</version>
85+
+ <version>${project.version}</version>
86+
<exclusions>
87+
<exclusion>
88+
<groupId>org.apache.zookeeper</groupId>
89+
diff --git a/standalone-metastore/metastore-tools/pom.xml b/standalone-metastore/metastore-tools/pom.xml
90+
index 4da5ea9d44..b184241f3e 100644
91+
--- a/standalone-metastore/metastore-tools/pom.xml
92+
+++ b/standalone-metastore/metastore-tools/pom.xml
93+
@@ -39,7 +39,7 @@
94+
<dependency>
95+
<groupId>org.apache.hive.hcatalog</groupId>
96+
<artifactId>hive-hcatalog-server-extensions</artifactId>
97+
- <version>${hive.version}</version>
98+
+ <version>${project.version}</version>
99+
<exclusions>
100+
<exclusion>
101+
<groupId>org.apache.zookeeper</groupId>
102+
diff --git a/standalone-metastore/packaging/pom.xml b/standalone-metastore/packaging/pom.xml
103+
index 5a526cf722..733a203db6 100644
104+
--- a/standalone-metastore/packaging/pom.xml
105+
+++ b/standalone-metastore/packaging/pom.xml
106+
@@ -176,7 +176,7 @@
107+
<dependency>
108+
<groupId>org.apache.hive</groupId>
109+
<artifactId>hive-metastore</artifactId>
110+
- <version>${hive.version}</version>
111+
+ <version>${project.version}</version>
112+
</dependency>
113+
<dependency>
114+
<groupId>org.apache.hive</groupId>
115+
@@ -192,13 +192,13 @@
116+
<dependency>
117+
<groupId>org.apache.hive</groupId>
118+
<artifactId>hive-exec</artifactId>
119+
- <version>${hive.version}</version>
120+
+ <version>${project.version}</version>
121+
<classifier>core</classifier>
122+
</dependency>
123+
<dependency>
124+
<groupId>org.apache.hive</groupId>
125+
<artifactId>hive-iceberg-handler</artifactId>
126+
- <version>${hive.version}</version>
127+
+ <version>${project.version}</version>
128+
<exclusions>
129+
<exclusion>
130+
<groupId>org.apache.hive</groupId>

0 commit comments

Comments
 (0)