Controller upgrade (0.13.x -> 0.14.x) orphans pre-upgrade AutoscalingListener per scale set; indefinite crashes with RunnerScaleSetNotFoundException

[cypher7682]

Checks

• I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
• I am using charts that are officially provided

Controller Version

0.14.2 (upgraded in place from 0.12.1)

Deployment Method

ArgoCD

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions).
• I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Install `gha-runner-scale-set-controller` and one or more `gha-runner-scale-set` releases at 0.12.1 (or any 0.13.x). Let the AutoscalingRunnerSets, AutoscalingListeners, and runners become healthy.
2. Upgrade the controller in place to 0.14.x/0.14.2 (chart version bump only; no change to the AutoscalingRunnerSet spec).
3. Observe that for every existing scale set there are now *two* AutoscalingListener objects in the controller namespace:
   - a new one named `<ars-name>-<newhash>-listener` (created by 0.14.x), which is healthy, and
   - the pre-upgrade one named `<ars-name>-<oldhash>-listener`, which is never deleted.
4. Observe that the old listener's pods crashing indefinitely, because it still holds the pre-upgrade `spec.runnerScaleSetId`, which no longer exists after re-registration:
   `createSession failed: ... StatusCode 404, RunnerScaleSetNotFoundException: No runner scale set found with identifier <old-id>`
5. Confirm the count invariant is broken: number of AutoscalingListener objects = 2 × number of AutoscalingRunnerSet objects.

Describe the bug

Upgrading the controller across the 0.13.x → 0.14.x boundary orphans the pre-upgrade AutoscalingListener for every existing scale set, and nothing ever cleans the orphan up. The orphaned listener pods crash forever, causing high pod churn and orphaned resources; though the runners themselves are unaffected.

Root cause

The listener object's name is derived from a hash, and the hash inputs changed in 0.14.0:

• 0.12.1 / 0.13.x — scaleSetListenerName() uses hash(namespace):

  func scaleSetListenerName(ars *v1alpha1.AutoscalingRunnerSet) string {
      namespaceHash := hash.FNVHashString(ars.Namespace)[:8]
      return fmt.Sprintf("%v-%v-listener", ars.Name, namespaceHash)
  }

• 0.14.0+ — it uses hashSuffix(namespace, runnerGroup, gitHubConfigUrl):

  func scaleSetListenerName(ars *v1alpha1.AutoscalingRunnerSet) string {
      return fmt.Sprintf("%v-%v-listener", ars.Name,
          hashSuffix(ars.Namespace, ars.Spec.RunnerGroup, ars.Spec.GitHubConfigUrl))
  }

  (introduced by the "use combination of namespace, GitHub URL, and runner group when hashing the listener name" change in 0.14.0)

So for the same AutoscalingRunnerSet, the computed listener name differs between versions.

The AutoscalingRunnerSet reconciler only ever finds "its" listener by a single Get on the freshly-computed name — there is no List-by-owner/label for listeners:

• autoscalingrunnerset_controller.go L236 (create path): r.Get(..., Name: scaleSetListenerName(ars)) → IsNotFound → creates a brand-new listener.
• autoscalingrunnerset_controller.go L404 (cleanup path): also r.Get(..., Name: scaleSetListenerName(ars)).

After the upgrade the controller computes the new name, gets NotFound, and creates the new listener. It never looks up the old-named listener, so it never deletes it. The orphan then cannot be reaped by anything else:

• No Kubernetes garbage collection. The listener lives in the controller namespace while its AutoscalingRunnerSet lives in the runner namespace; cross-namespace owner references are disallowed, so the listener has no ownerReferences.
• Not reachable by GitOps prune. The listener is created by the controller, not rendered by the chart, so GitOps tooling does not track or prune it.

The orphaned listener still carries the autoscalinglistener.actions.github.com/finalizer, so the AutoscalingListener controller keeps recreating its pod. That pod calls createSession for the stale runnerScaleSetId and gets 404 RunnerScaleSetNotFoundException, exits, and is recreated — an indefinite crash state.

Notably, the reconciler already uses the robust pattern for the sibling resource: it finds EphemeralRunnerSets by owner index rather than by reconstructed name —
autoscalingrunnerset_controller.go L757: r.List(ctx, list, client.InNamespace(ars.Namespace), client.MatchingFields{resourceOwnerKey: ars.Name}).
The listener is the odd one out, discovered by reconstructed name instead of by identity.

No migration shipped with the 0.14.0 name change to rename or remove pre-existing listeners, so every scale set that existed before the upgrade is left with a persistently crashing twin.

Describe the expected behavior

Upgrading the controller should not orphan the previous listener. The AutoscalingRunnerSet reconciler should discover its listener by a stable identity (owner reference where possible, or the stable scale-set identity labels actions.github.com/scale-set-name + actions.github.com/scale-set-namespace, which are already stamped and are identical across versions) and delete any listener for the scale set that is not the current desired one, rather than looking the listener up solely by a version-dependent computed name.

Equivalently: after reconcile there should be exactly one AutoscalingListener per AutoscalingRunnerSet, regardless of any change to the name-hash inputs across versions. A change to the naming scheme should self-heal on the next reconcile instead of stranding the prior object.

Additional Context

# These are the only relevant lines:
githubConfigUrl: https://github.com/<ORG>
runnerGroup: <runner-group>
runnerScaleSetName: <scale-set-name>

Controller Logs

https://gist.github.com/cypher7682/9d9113d18458e5d46c6505a191717027

Runner Pod Logs

N/A - no relevant logs here for this.

[github-actions]

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.