docs: add migration instruction for sparkapplicationtemplate resources

docs/modules/spark-k8s/pages/usage-guide/app_templates.adoc

@@ -12,16 +12,10 @@ Application templates are available for the `v1alpha1` version of the SparkAppli
 4. Application template references are immutable in the sense that once applied to an application they cannot be changed again. Currently templates are applied upon the creation of the application, and any changes to the template references after that will be ignored.
 5. Application and template CRDs must have the exact same versions. Currently only `v1alpha1` is supported.
 
-== Migrating from cluster-scoped templates
+IMPORTANT: Application templates were cluster-scoped when they were first released in SDP 26.3, and are namespace-scoped from SDP 26.7 onwards.
+Upgrading across that change requires manual steps, because a CustomResourceDefinition cannot change its scope in place.
+See xref:usage-guide/upgrade.adoc[] before upgrading.
 
-IMPORTANT: Application templates used to be cluster wide resources when they were first released. This was a mistake. Many users do not have the access rights to create cluster scoped resources and so the templates are now namespace scoped.
-
-If you are migrating from older installations where templates were treated as cluster-wide resources, account for the following:
-
-1. Recreate each template in every namespace where SparkApplications use it.
-2. Keep template names consistent per namespace if you want the same application annotations to continue working.
-3. Cross-namespace template references are no longer resolved; templates and applications must be in the same namespace.
-4. Update GitOps/automation manifests to create templates as namespace-targeted resources before reconciling dependent SparkApplications.
 == Examples
 
 Applications use `metadata.annotations` to reference application templates as shown below:

docs/modules/spark-k8s/pages/usage-guide/upgrade.adoc

@@ -0,0 +1,93 @@
+= SDP upgrade notes
+:description: Instructions for upgrading the SDP versions.
+
+== Upgrade from SDP 26.3 to 26.7
+
+=== Application templates changed from cluster-scoped to namespace-scoped
+
+`SparkApplicationTemplate` was a cluster-scoped resource when it was first released in SDP 26.3.
+As of SDP 26.7 the resource is namespace-scoped because many users do not have the access rights to create cluster-scoped resources.
+
+WARNING: This change requires manual intervention *before* the upgrade.
+A Kubernetes CustomResourceDefinition cannot change its scope in place, so the 26.7 operator cannot perform this migration for you, and it does not start until the old CustomResourceDefinition is gone.
+
+==== What you see if you upgrade without preparing
+
+The operator maintains its own CustomResourceDefinitions.
+On startup it tries to reconcile `sparkapptemplates.spark.stackable.tech` to the namespace-scoped definition, the API server rejects the change because `spec.scope` is immutable, and the operator exits:
+
+[source]
+----
+Error: failed to run webhook server
+
+Caused by:
+    0: failed to update certificate
+    1: conversion webhook error
+    2: failed to patch CRD "sparkapptemplates.spark.stackable.tech"
+    3: ApiError: CustomResourceDefinition.apiextensions.k8s.io
+       "sparkapptemplates.spark.stackable.tech" is invalid:
+       spec.scope: Invalid value: "Namespaced": field is immutable: Invalid
+----
+
+The operator Pod then enters `CrashLoopBackOff` and no Spark resources are reconciled — not just templates, but `SparkApplication`, `SparkConnectServer` and `SparkHistoryServer` as well, because the operator never finishes starting.
+
+When installed through OLM on OpenShift, the ClusterServiceVersion stays in the `Installing` phase with `waiting for deployment spark-k8s-operator-deployment to become ready`.
+
+=== Migration procedure
+
+Deleting the CustomResourceDefinition deletes every `SparkApplicationTemplate` in the cluster.
+Export them first if you want to keep them.
+
+. Export the existing templates, while the 26.3 operator is still running:
++
+[source,shell]
+----
+kubectl get sparkapptemplates.spark.stackable.tech -o yaml > spark-app-templates.yaml
+----
++
+NOTE: Because the resource is still cluster-scoped at this point, this single command captures every template in the cluster.
+Note which namespaces the SparkApplications that reference them live in — you need that in step 4.
+
+. Delete the CustomResourceDefinition.
+This also deletes all `SparkApplicationTemplate` objects:
++
+[source,shell]
+----
+kubectl delete crd sparkapptemplates.spark.stackable.tech
+----
+
+. Upgrade the operator to 26.7.
+It recreates `sparkapptemplates.spark.stackable.tech` as a namespace-scoped resource on startup.
+Confirm it did:
++
+[source,shell]
+----
+kubectl get crd sparkapptemplates.spark.stackable.tech -o jsonpath='{.spec.scope}'
+----
++
+This must print `Namespaced`.
+
+. Recreate the templates from your export, once per namespace that needs them.
+Strip the `metadata.resourceVersion`, `metadata.uid` and `metadata.creationTimestamp` fields first, otherwise the apply is rejected.
+The target namespace comes from `-n`, so the manifests themselves do not need a `metadata.namespace`:
++
+[source,shell]
+----
+kubectl apply -n <namespace> -f spark-app-templates.yaml
+----
++
+If only some templates belong in a given namespace, split the exported file accordingly rather than applying all of them everywhere.
+
+If the operator is already crashlooping because the upgrade happened first, the recovery is the same, except that the templates are no longer readable and cannot be exported.
+Delete the CustomResourceDefinition, let the operator restart, and recreate the templates from your source manifests or GitOps repository.
+
+=== Consequences for existing manifests
+
+A template must now exist in the same namespace as the SparkApplications that reference it:
+
+. Recreate each template in every namespace where SparkApplications use it.
+. Keep template names consistent per namespace if you want the same application annotations to keep working.
+. Cross-namespace template references are no longer resolved; templates and applications must be in the same namespace.
+. Update GitOps and automation manifests to create templates as namespace-targeted resources before reconciling dependent SparkApplications.
+
+See xref:usage-guide/app_templates.adoc[] for the current behaviour of application templates.

docs/modules/spark-k8s/partials/nav.adoc

@@ -13,6 +13,7 @@
 ** xref:spark-k8s:usage-guide/spark-connect.adoc[]
 ** xref:spark-k8s:usage-guide/examples.adoc[]
 ** xref:spark-k8s:usage-guide/overrides.adoc[]
+** xref:spark-k8s:usage-guide/upgrade.adoc[]
 ** xref:spark-k8s:usage-guide/operations/index.adoc[]
 *** xref:spark-k8s:usage-guide/operations/applications.adoc[]
 *** xref:spark-k8s:usage-guide/operations/pod-placement.adoc[]