From cc2f9eeb420f9f97201c189e5b7a3ae4b29d86fb Mon Sep 17 00:00:00 2001 From: "Christoph Engelbert (noctarius)" Date: Mon, 31 Aug 2026 22:27:16 +0200 Subject: [PATCH 1/3] Improved DHCHAP Kubernetes page --- .../security/authentication-encryption.md | 105 +++++++++++++----- 1 file changed, 80 insertions(+), 25 deletions(-) diff --git a/docs/kubernetes/operations/security/authentication-encryption.md b/docs/kubernetes/operations/security/authentication-encryption.md index 0972191f..49df89e4 100644 --- a/docs/kubernetes/operations/security/authentication-encryption.md +++ b/docs/kubernetes/operations/security/authentication-encryption.md @@ -1,19 +1,21 @@ --- title: Host Authentication and Encryption -description: "Configure NVMe-oF host access control, DH-HMAC-CHAP authentication, and TLS/PSK encryption on Kubernetes through the StoragePool custom resource." +description: "Host access control and DH-HMAC-CHAP authentication for the NVMe-oF transport on Kubernetes, configured on a StoragePool and reconciled by the operator." weight: 10710 --- -Simplyblock supports NVMe-oF transport security to protect data in transit and restrict host access to storage -subsystems. This includes: +Simplyblock protects the NVMe-oF transport between a worker node and a storage node with host access control and +DH-HMAC-CHAP authentication. Only a host registered as an allowed host of a storage pool can connect to the NVMe-oF +subsystem of a volume in that pool, and every connection is authenticated in both directions with the NVMe standard +authentication protocol (TP8018). -- **Host access control:** restrict which hosts (by NQN) can connect to a volume's NVMe-oF subsystem. -- **DH-HMAC-CHAP authentication:** mutual authentication between host and target using the NVMe standard - authentication protocol (TP8018). -- **TLS/PSK encryption:** encrypt data in transit using TLS 1.3 with Pre-Shared Keys. +On Kubernetes, both are configured declaratively on the `StoragePool` custom resource and reconciled by the +Simplyblock Operator. No host NQN has to be registered, and no key has to be provisioned by hand. -On Kubernetes, transport security is configured declaratively on the `StoragePool` custom resource and reconciled -by the Simplyblock Operator. No host NQN has to be registered and no key has to be provisioned by hand. +!!! note "Transport encryption and volume encryption" + TLS/PSK encryption of the NVMe-oF transport is not exposed through the `StoragePool`. Encryption of the volume + data at rest is an unrelated, per-volume feature and is described in + [Volume Encryption](../../usage/volume-encryption.md). ## Enable Host Authentication and Encryption @@ -34,31 +36,84 @@ spec: - worker-2 ``` -The keys are generated as soon as `dhchap` is set, but authentication is only enforced once `allowedNodes` is -non-empty. +The DH-HMAC-CHAP keys of the pool are generated as soon as `dhchap` is set. Authentication is only enforced once +`allowedNodes` is non-empty. + +Both fields belong into the manifest that creates the pool. The `StorageClass` generated for the pool is only +restricted to the allowed nodes when `dhchap` is `true` and `allowedNodes` is non-empty at the moment the class is +created, and `parameters` and `allowedTopologies` cannot be patched afterward. A pool created with `dhchap: true` and +an empty `allowedNodes` therefore keeps an unrestricted `StorageClass` for the rest of its life, even once nodes are +added to the list. Recreating the pool is the only way to correct this. ## Reconciliation by the Operator Once the storage pool is created, host registration and node scheduling are reconciled by the operator: -- Each node in `allowedNodes` is registered as an allowed host of the pool, under a deterministic NQN derived - from that node's Kubernetes UID (`nqn.2014-08.io.simplyblock:uuid:`). -- Each allowed node is labeled `simplyblock.io/pool...: allowed`, and the generated - `StorageClass` is restricted to that label through `allowedTopologies`. The first `Pod` to consume a - `PersistentVolumeClaim` of this pool can therefore only be scheduled onto an allowed node. -- The same label is written into the `nodeAffinity` of the `PersistentVolume` when the volume is created, which - restricts every later scheduling decision on the already-bound volume. -- The node's own NQN and the pool's DHCHAP secrets are presented by the CSI node plugin on connect, so no host - NQN has to be supplied anywhere in the Kubernetes flow. +- **Allowed hosts:** each node in `allowedNodes` is registered as an allowed host of the pool, under a deterministic + NQN derived from that node's Kubernetes UID (`nqn.2014-08.io.simplyblock:uuid:`). +- **Node labels:** each allowed node is labeled `simplyblock.io/pool...: allowed`, and the + label is removed again from every node that leaves the list. +- **First scheduling decision:** the generated `StorageClass` is restricted to that label through `allowedTopologies`, + so the first `Pod` to consume a `PersistentVolumeClaim` of this pool can only be scheduled onto an allowed node. +- **Every later scheduling decision:** the same label is written into the `nodeAffinity` of the `PersistentVolume` + when the volume is created, which restricts every scheduling decision on the already-bound volume, including a + restart, a recreate, and a drain. +- **Host NQN:** the node's own NQN and the pool's DHCHAP secrets are presented by the CSI node plugin on connect, so + no host NQN has to be supplied anywhere in the Kubernetes flow. ## Managing Allowed Nodes -`dhchap` is immutable, because the `parameters` and `allowedTopologies` of the generated `StorageClass` cannot -be patched in the Kubernetes API once it exists. `allowedNodes` stays mutable. Changing it relabels the nodes -and updates the pool's allowed hosts, and it never rewrites the `StorageClass`. +`dhchap` is immutable, because the `parameters` and `allowedTopologies` of the generated `StorageClass` cannot be +patched in the Kubernetes API once it exists. `allowedNodes` stays mutable. Changing it relabels the nodes and updates +the pool's allowed hosts, and it never rewrites the `StorageClass`. + +A node removed from `allowedNodes` loses its label, and its NQN is removed from the allowed hosts of the pool and of +every volume in it. The node is rejected on its next connect attempt. A volume already connected on that node is not +disconnected by the removal. + +## Verifying the Configuration + +`status.allowedNodes` carries the node names last registered on the control plane. A difference to `spec.allowedNodes` +means the pool has not converged yet. + +```bash title="Reading the allowed nodes registered for a storage pool" +kubectl get storagepool pool-a -n simplyblock \ + -o jsonpath='{.status.allowedNodes}' +``` + +The nodes carrying the pool's label are listed through a label selector. The result has to match `status.allowedNodes`. + +```bash title="Listing the nodes labeled as allowed for a storage pool" +kubectl get nodes \ + -l simplyblock.io/pool.simplyblock.cluster-a.pool-a=allowed +``` + +Whether the generated `StorageClass` restricts scheduling at all is visible in its `allowedTopologies`. An empty result +means the class was created while `allowedNodes` was empty. + +```bash title="Checking the topology restriction of the generated storage class" +kubectl get storageclass simplyblock-simplyblock-cluster-a-pool-a \ + -o jsonpath='{.allowedTopologies}' +``` + +## Pods on a Disallowed Node + +`allowedTopologies` and the `nodeAffinity` of the `PersistentVolume` keep a `Pod` off a node outside `allowedNodes`. +If one lands there regardless, no `nvme connect` is ever built. `NodeStageVolume` derives the host NQN of its own node +and requests the connection information from the control plane, which rejects the unknown NQN with an HTTP `404`. The +`Pod` stays unscheduled with a `FailedMount` event. + +```plain title="Example of a FailedMount event on a node outside the allowed nodes" +MountVolume.MountDevice failed for volume "pvc-...": rpc error: code = Internal +desc = failed to fetch connection: GET 404: Host NQN +nqn.2014-08.io.simplyblock:uuid: not found in allowed hosts for volume +``` + +The node is either missing from `allowedNodes` or the pool has not converged yet. Both are checked as described in +[Verifying the Configuration](#verifying-the-configuration). -See the [Operator Reference](../../../reference/operator/reference.md) for the full `StoragePool` field list, -and [Storage Class](../../usage/storage-class.md) for the `dhchap_node_label` parameter this generates. +See the [Operator Reference](../../../reference/operator/reference.md) for the full `StoragePool` field list, and +[Storage Class](../../usage/storage-class.md) for the `dhchap_node_label` parameter this generates. For a detailed explanation of the security mechanisms and configuration, see [NVMe-oF Security](../../../architecture/concepts/nvmf-security.md). The equivalent flow for a plain Linux From 0e1a97d1a15db92a31948f9d89fd0885a0d94d26 Mon Sep 17 00:00:00 2001 From: noctarius aka Christoph Engelbert Date: Mon, 31 Aug 2026 22:33:54 +0200 Subject: [PATCH 2/3] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- .../kubernetes/operations/security/authentication-encryption.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes/operations/security/authentication-encryption.md b/docs/kubernetes/operations/security/authentication-encryption.md index 49df89e4..3f2ba4b1 100644 --- a/docs/kubernetes/operations/security/authentication-encryption.md +++ b/docs/kubernetes/operations/security/authentication-encryption.md @@ -39,7 +39,7 @@ spec: The DH-HMAC-CHAP keys of the pool are generated as soon as `dhchap` is set. Authentication is only enforced once `allowedNodes` is non-empty. -Both fields belong into the manifest that creates the pool. The `StorageClass` generated for the pool is only +Both fields belong in the manifest that creates the pool. The `StorageClass` generated for the pool is only restricted to the allowed nodes when `dhchap` is `true` and `allowedNodes` is non-empty at the moment the class is created, and `parameters` and `allowedTopologies` cannot be patched afterward. A pool created with `dhchap: true` and an empty `allowedNodes` therefore keeps an unrestricted `StorageClass` for the rest of its life, even once nodes are From aa6633c6f412d00037671b48d43380cc342be92b Mon Sep 17 00:00:00 2001 From: noctarius aka Christoph Engelbert Date: Mon, 31 Aug 2026 22:34:24 +0200 Subject: [PATCH 3/3] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- .../kubernetes/operations/security/authentication-encryption.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes/operations/security/authentication-encryption.md b/docs/kubernetes/operations/security/authentication-encryption.md index 3f2ba4b1..4b2d11d6 100644 --- a/docs/kubernetes/operations/security/authentication-encryption.md +++ b/docs/kubernetes/operations/security/authentication-encryption.md @@ -51,7 +51,7 @@ Once the storage pool is created, host registration and node scheduling are reco - **Allowed hosts:** each node in `allowedNodes` is registered as an allowed host of the pool, under a deterministic NQN derived from that node's Kubernetes UID (`nqn.2014-08.io.simplyblock:uuid:`). -- **Node labels:** each allowed node is labeled `simplyblock.io/pool...: allowed`, and the +- **Node labels:** each allowed node is labeled `simplyblock.io/pool...=allowed`, and the label is removed again from every node that leaves the list. - **First scheduling decision:** the generated `StorageClass` is restricted to that label through `allowedTopologies`, so the first `Pod` to consume a `PersistentVolumeClaim` of this pool can only be scheduled onto an allowed node.