Repository navigation
feat(controller): let the module controller authorize its metrics itself - #235
Merged
Merged
Conversation
duckhawk
marked this pull request as ready for review
November 15, 2026 10:51
kkozoriz
self-requested a review
November 15, 2026 10:51
kkozoriz
approved these changes
Nov 15, 2026
helm_lib_module_controller_manifests put a kube-rbac-proxy container next to the controller whenever controllerMetricsProxyPort was set. That is one more process, with its own memory request, in every controller replica on master nodes, only to check a bearer token before forwarding /metrics. controllerMetricsInProcessAuthorization (default false) drops the sidecar and its VPA policy. The controller container then carries the https-metrics port itself, and the template emits METRICS_BIND_ADDRESS=:<controllerMetricsProxyPort>, METRICS_AUTHORIZATION_NAMESPACE and METRICS_AUTHORIZATION_DEPLOYMENT, so the controller can make the same checks the proxy made: TokenReview, then SubjectAccessReview for get on deployments/prometheus-metrics of its Deployment. RBAC stays as it is: the d8:rbac-proxy binding of the controller ServiceAccount and the Prometheus grant on prometheus-metrics. Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
duckhawk
force-pushed
the
feat/controller-metrics-in-process-authorization
branch
from
November 15, 2026 11:00
6b278ac to
8191c70
Compare
2 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
helm_lib_module_controller_manifestsgetscontrollerMetricsInProcessAuthorization(defaultfalse). Together withcontrollerMetricsProxyPortit changes how the controller's metrics are served:kube-rbac-proxycontainer and its VPA policy are not rendered;https-metricsport (controllerMetricsProxyPort);METRICS_BIND_ADDRESS=:<controllerMetricsProxyPort>,METRICS_AUTHORIZATION_NAMESPACE=d8-<module>andMETRICS_AUTHORIZATION_DEPLOYMENT=<fullname>.The controller is then expected to serve HTTPS on that address and authorize each request the way the proxy did: a TokenReview, then a SubjectAccessReview for
getondeployments/prometheus-metricsof its own Deployment.RBAC is unchanged:
helm_lib_module_controller_rbacstill binds the controller ServiceAccount tod8:rbac-proxy(TokenReview, SubjectAccessReview), and the module keeps its Prometheus grant onprometheus-metrics. The Service port namehttps-metricsand the ServiceMonitor stay as they are.Without the option nothing changes. The chart version goes to 1.72.36; it is independent of #234.
Why do we need it, and what problem does it solve?
Part of reducing the memory requests of the storage modules on master nodes. In every replica of every module controller, a kube-rbac-proxy runs as a separate process with its own VPA request, only to check a bearer token before forwarding
/metrics. On a test cluster with three masters that is 55 containers and about 1 GiB of working set. Each one also costs a request of 25Mi on DKP 1.72–1.75, 64Mi on 1.76.0–1.76.11 and about 32Mi on 1.76.12+.The option keeps the authorization contract of the proxy, so Prometheus needs no new permissions on any DKP version. controller-runtime's own
filters.WithAuthenticationAndAuthorizationwas not used for two reasons. It authorizes the non-resource URL/metrics, a cluster-wide grant that is wider than the per-Deploymentprometheus-metricssubresource the proxy checked: any identity allowed to read/metricsof any component could read the controller. It also pullsk8s.io/apiserverinto the binary.The pilot module is csi-nfs.
What is the expected result?
controllerMetricsInProcessAuthorization: truehas nokube-rbac-proxycontainer. The controller listens onhttps-metricsand gets the three environment variables above.Checklist