Skip to content

[feat] Skip the test if its associated Slurm job has expired - #3728

Open
vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/slurm-deadline-graceful
Open

[feat] Skip the test if its associated Slurm job has expired#3728
vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/slurm-deadline-graceful

Conversation

@vkarak

@vkarak vkarak commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

If the job's state is DEADLINE, then the associated test is skipped. Here's an example:

Previously, the test was continuing and failed in sanity checking because stdout/stderr were not present:

FAILURE INFO for stream_build_test (run: 1/1)
  * Description:
  * System partition: pseudo-cluster:compute
  * Environment: gnu-mpi
  * Test file: /home/admin/reframe-examples/tutorial/stream/stream_build_run.py
  * Stage directory: /scratch/rfm-stage/stage/pseudo-cluster/compute/gnu-mpi/stream_build_test
  * Node list: None
  * Job type: batch job (id=70)
  * Dependencies (conceptual): []
  * Dependencies (actual): []
  * Maintainers: []
  * Failing phase: sanity
  * Rerun with '-n /6c084d40 -p gnu-mpi --system pseudo-cluster -r'
  * Reason: sanity error: rfm_job.out: No such file or directory
--- rfm_job.out (last 10 lines) ---
--- rfm_job.out (ERROR: [Errno 2] No such file or directory: '/scratch/rfm-stage/stage/pseudo-cluster/compute/gnu-mpi/stream_build_test/rfm_job.out') ---
--- rfm_job.err (last 10 lines) ---
--- rfm_job.err (ERROR: [Errno 2] No such file or directory: '/scratch/rfm-stage/stage/pseudo-cluster/compute/gnu-mpi/stream_build_test/rfm_job.err') ---

Now the test is skipped with an informative message:

[----------] start processing checks
[ RUN      ] stream_build_test /6c084d40 @pseudo-cluster:compute+gnu
[     SKIP ] (1/2) stream_build_test /6c084d40 @pseudo-cluster:compute+gnu [the associated job 71 has expired]
[ RUN      ] stream_build_test /6c084d40 @pseudo-cluster:compute+gnu-mpi
[     SKIP ] (2/2) stream_build_test /6c084d40 @pseudo-cluster:compute+gnu-mpi [the associated job 72 has expired]
[----------] all spawned checks have finished

Closes #3700.

Signed-off-by: Vasileios Karakasis <vkarak@gmail.com>
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.91%. Comparing base (00ffd41) to head (0032738).

Files with missing lines Patch % Lines
reframe/core/schedulers/slurm.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3728      +/-   ##
===========================================
- Coverage    91.92%   91.91%   -0.01%     
===========================================
  Files           62       62              
  Lines        13808    13810       +2     
===========================================
+ Hits         12693    12694       +1     
- Misses        1115     1116       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Treat Slurm job deadlines gracefully

1 participant