Retry starts after concurrent job limits

[Silvren]

Summary

• recognize the unofficial ConcurrentJobLimit API error as a retryable job-start outcome
• return affected jobs to the persisted created state instead of marking them start_failed
• re-submit created jobs during a later manager polling cycle, providing non-blocking backoff without sleeping in worker threads
• factor start-task submission into one path so initial starts and retries share authentication and accounting behavior
• add unit and end-to-end coverage for a rejected first start followed by a successful retry and completed job

Testing

• python -m pytest tests/extra/job_management -q -k "not resilient" (179 passed, 6 deselected)
• targeted job-start and worker tests (35 passed)
• pre-commit checks on all changed files

Fixes #838

[soxofaan]

@HansVRP can you give this a review?

[HansVRP]

@soxofaan To me this seems more of a patch to the unclear behavior of the parallel jobs setting in the job manager vs a user account available nr of concurrent jobs.

The more elegant solution is probably going to be make the job manager aware of the user-s max concurrency.

[Silvren]

Thanks, that distinction makes sense. My intent with this PR is to provide the runtime fallback requested in #838: when the backend rejects a start with ConcurrentJobLimit, keep the job recoverable and retry it in a later manager cycle instead of marking it as permanently failed.

I agree that making the manager aware of the effective per-user concurrency limit would be a better scheduling-layer improvement. At the moment that limit is not exposed through a standardized openEO API contract, and it can also differ from the manager's configured parallel-job limit, so I don't think it fully replaces handling the backend error.

Would you be comfortable keeping this error-driven fallback as a safety net, while treating concurrency-limit discovery/configuration as a separate follow-up? If you prefer that this PR also changes the scheduling model, I can rework it once we agree on where the effective limit should come from.

[HansVRP]

We will take the PR under further consideration, the concurrent joblimit is also not an official error message yet and may lead to a brittle design. So I definitely see the use of the solution, but I want to investigate how easy/hard it is to tackle the actual underlying issues and avoid the error all together