Skip to content

[feat] Extend allowed syntax for system/partitions/environment names and enforce strict syntax checking for extra_resources keys - #3725

Open
vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/expand-alphanum-ext-string
Open

[feat] Extend allowed syntax for system/partitions/environment names and enforce strict syntax checking for extra_resources keys#3725
vkarak wants to merge 1 commit into
reframe-hpc:developfrom
vkarak:feat/expand-alphanum-ext-string

Conversation

@vkarak

@vkarak vkarak commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Broaden the alphanum_ext_string schema type so that system, partition, environment and feature names may:

  • start with a digit;
  • contain dots (.) and plus signs (+), in addition to the already-allowed dash (-) and underscore (_).

The valid_systems / valid_prog_environs mini-language is updated in lockstep so that every name accepted by the configuration schema can also be targeted precisely in a test's valid_systems / valid_prog_environs.

This PR also enforces strict syntax in the extra_resources keys as these are expanded in kwargs and if they are not valid identifiers, ReFrame will crash, e.g., here:

**self.extra_resources

Replaces #3720
Closes #3645
Closes #3723

Implementation details

Schema (reframe/schemas/config.json)

  • alphanum_ext_string pattern: ^[a-zA-Z_](?:[a-zA-Z0-9_-])*$^[a-zA-Z0-9_](?:[a-zA-Z0-9_.+-])*$.
    A name still may not start with ., + or -; this invariant is what keeps the mini-language tokenizer unambiguous (a bare name can never be mistaken for a +feat / -feat / %key=val token).
  • The inline propertyNames patterns of systems.partitions.extras, environments.extras and environments.resources now reference the shared #/defs/alphanum_string def (no behavioural change — the inline patterns were already identical).
  • systems.partitions.resources[].name is now validated against alphanum_string instead of accepting an arbitrary string.

Mini-language (reframe/core/pipeline.py)

  • _N (the name production used by _VALID_SYS_SYNTAX / _VALID_ENV_SYNTAX) gains +. Leading digits and . were already accepted by _N; with this change the schema pattern and _N are equivalent for ASCII input.

Parser (reframe/core/runtime.py)

  • _is_valid_part() / _is_valid_env() now split %key=val specs with maxsplit=1, so a value containing = (e.g. %key=a=b) no longer raises an uncaught ValueError.

Docs (docs/config_reference.rst)

  • Updated the character rules for systems.name, systems.partitions.name, environments.name (the last one previously said nothing) and the two features lists.
  • Documented the deliberate two-tier model: names and features are permissive; extras keys and resource names stay plain POSIX-style identifiers (^[a-zA-Z_][a-zA-Z0-9_]*$).
  • versionchanged:: 4.11.0 markers throughout.

Grammar: before → after

Input Before After
sys.x:part.y
0sys:1part ❌ (schema)
g++, gcc-11.2+omp
sys:gpu+mps, +gpu+mps
.sys, +sys, -sys
sys x, a:b:c, a@b

Signed-off-by: Vasileios Karakasis <vkarak@gmail.com>
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.53%. Comparing base (00ffd41) to head (cc696c3).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3725      +/-   ##
===========================================
- Coverage    91.92%   91.53%   -0.40%     
===========================================
  Files           62       62              
  Lines        13808    13808              
===========================================
- Hits         12693    12639      -54     
- Misses        1115     1169      +54     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@RiMillo

RiMillo commented Sep 10, 2026

Copy link
Copy Markdown

Hello @vkarak,

I'm reporting back from #3720.

First of all, thanks for taking the time, it is really appreciated.

I had some tests with my config. My original need was to allow version-like names for environments, i.e., 1.0.0, and they worked. I also tested system names with leading digits and they worked, too! Thanks!

I then tried adding +<stuff>, e.g., 1.0.0+alpha to environment names, and had some unexpected (at least to me, since I do not practice reframe that much) results. The name is validated just fine (that's an improvement wrt master). But if I provide the full name when launching reframe, e.g., --prgenv '1.0.0+alpha', no test is matched (hence, run neither), even if tests have valid_prog_environs = ["*"]. It works if I leave out the +alpha: since it is a regex pattern, the right environment is selected. To sum, with environment 1.0.0+alpha and valid_prog_environs = ["*"]:

  • reframe --prgenv=1.0.0+alpha: test is not selected ❌
  • reframe --prgenv=1.0.0: test is run ✅

Just to be clear, I'm already happy with the current status, and if this is, in fact, an expected behaviour, well, sorry for the noise, and do not hesitate to just merge 🙃

@vkarak

vkarak commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@RiMillo This is expected behaviour because the --prgenv interprets its argument as a Python regex. So you have to escape the + if you want it to be interpreted literally. Can you try --prgenv=1\.0\.0\+alpha. Note that technically you have to escape . as well: the fact that it works is because it matches any character. It could match also an environment named 1_0_0.

@RiMillo

RiMillo commented Sep 10, 2026

Copy link
Copy Markdown

Ah, you are right, of course. I did not think about that!

Indeed it works when escaping \+!

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Allow feature names to start with a digit Relax alphanum_ext_string schema to support numeric and version-based environment names

2 participants