Add hive.kerberos-service-host configuration option - #3864
Conversation
| properties.get("ugi"), | ||
| property_as_bool(properties, HIVE_KERBEROS_AUTH, HIVE_KERBEROS_AUTH_DEFAULT), | ||
| properties.get(HIVE_KERBEROS_SERVICE_NAME, HIVE_KERBEROS_SERVICE_NAME_DEFAULT), | ||
| properties.get(HIVE_KERBEROS_SERVICE_HOST), |
There was a problem hiding this comment.
Let's try setting the default value here as properties.get(HIVE_KERBEROS_SERVICE_HOST, urlparse(uri).hostname)
That way, we don't have to do any logic in hive.py:175 and can just pass in the property directly.
There was a problem hiding this comment.
@rambleraptor Thanks for the review! Good idea, I moved the branching out of _init_thrift_transport. (80571f7)
I put the default in _HiveClient.__init__ instead of in _create_hive_client, because _HiveClient can also be constructed directly. With the default only at the call site, test_create_hive_client_with_kerberos_using_context_manager and test_kerberized_client_uses_fresh_transport_on_reuse fail, since they construct _HiveClient without a host.
Happy to move it to the call site if you prefer. That would just need those two tests to pass the host explicitly.
Closes #3787
Rationale for this change
The Hive catalog derives the Kerberos SASL
host(the hostname component of the service principal) from the metastore URI, with no way to override it. Authentication therefore fails whenever the service principal's hostname component differs from the host being connected to, for example in an HA setup whereurihas to list the real metastore hosts while the principal uses a single fixed hostname.This adds an optional
hive.kerberos-service-hostproperty, mirroring the existinghive.kerberos-service-namefor theservicecomponent. When it is not set, the URI host is used, so existing behavior is unchanged.Are these changes tested?
Yes.
test_create_hive_client_with_kerberos_service_hostverifies the property is passed through to_HiveClient.test_kerberized_client_uses_configured_service_hostverifies at the transport level that the SASL host is the configured value, and falls back to the URI host when the property is not set._create_hive_clientassertions were updated for the new argument.I also verified it end-to-end against a Hive Metastore with Kerberos: without the property the connection fails with a KDC
LOOKING_UP_SERVERerror, and with it the correct service ticket is issued and the catalog works.Are there any user-facing changes?
Yes, a new optional Hive catalog property
hive.kerberos-service-host. It defaults to the metastore URI host, so there is no change for existing users. The documentation table has been updated.