Skip to content

Input: reject DSP runs where kpar*bndpar != NPROC at input time - #8238

Merged
mohanchen merged 3 commits into
deepmodeling:developfrom
DanielZhao0432:fix-dsp-kpar-guard
Oct 30, 2026
Merged

mohanchen merged 3 commits into
deepmodeling:developfrom
DanielZhao0432:fix-dsp-kpar-guard

Conversation

@DanielZhao0432

@DanielZhao0432 DanielZhao0432 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
  • I have requested any needed governance exception below.

Linked Issue

Fix #8228

Unit Tests and/or Case Tests for my changes

  • Commands run:
    1. Local (non-DSP build): recompiled the affected translation unit
      (read_inp_sys.cpp.o) successfully; the new check is inside #ifdef __DSP
      and is not compiled in non-DSP builds.
    2. Real DSP build on Tianhe (this branch, based on develop after v3.11.0-beta11,
      commit 64dcb68), case 001_4GaAs, 4 MPI processes x 4 OpenMP threads:
      • kpar=1 (default): ABACUS aborts during input reading with the intended
        diagnostic: "On DSP hardware, the mtblas/C_PACK libraries require kpar
        (kpar_lcao for LCAO) * bndpar to equal the number of MPI processes, but got
        NPROC=4, kpar=1, kpar_lcao=1, bndpar=1. Please set kpar=4 or adjust the
        process count." (exit code 1)
      • kpar=4: the calculation runs and converges normally.
    3. python3 tools/03_code_analysis/agent_governance_check.py --staged → no findings.
    4. New unit test source/source_io/unittests/test_read_inp_sys.cpp (target
      MODULE_IO_test_read_inp_sys) covering dsp_kpar_layout_valid: PW k-point
      pools, the LCAO kpar→kpar_lcao reset, the bndpar exemption from Fix: relax the parameter constraints of device DSP to allow bndpar+kpar #7420, and
      degenerate zero/negative values — 4/4 tests pass locally.
  • Result summary: all checks above pass.
  • Checks not run, with reason: no unit/case test added — the check exists only in
    __DSP builds and CI has no DSP toolchain/hardware; the behavior was verified
    manually on Tianhe as listed above.

What's changed?

  • Background: on DSP hardware, the mtblas/C_PACK vendor libraries require exactly
    one MPI process per k-point pool. Any other configuration (e.g. the default
    kpar=1 with 4 processes) makes the vendor routines return inconsistent results
    across ranks, and the SCF silently diverges (see the linked issue for logs).
    [Fix] Report and exit for unsupported DSP KPAR configuration #6771/Fix DSP count check problem #7035 previously rejected such configurations with a WARNING_QUIT in
    Driver::init_hardware, but Fix: relax the parameter constraints of device DSP to allow bndpar+kpar #7420 removed the guard entirely to allow
    bndpar+kpar combinations, leaving the invalid configuration unguarded.
  • This PR reinstates the guard as a kpar input check compiled only under
    __DSP: kpar (kpar_lcao for LCAO) * bndpar must equal the number of MPI
    processes. Unlike the removed check, the bndpar factor keeps the bndpar+kpar
    use case that Fix: relax the parameter constraints of device DSP to allow bndpar+kpar #7420 intended to allow. Because the check runs during input
    reading, it also triggers under --check-input.
  • The kpar INPUT description now documents the DSP restriction (visible via
    abacus -h kpar and in the generated documentation).
  • User-visible behavior change: DSP builds abort at startup with a clear
    diagnostic for configurations the vendor libraries cannot handle, instead of
    silently diverging mid-SCF. Non-DSP builds are completely unaffected.

Governance Notes

  • INPUT/docs changes: kpar description updated (metadata only; no parameter
    added/renamed; the new validation applies to DSP builds only).
  • Core module impact: none — input validation only, in source_io.
  • Exceptions requested: none for the global dependency budget (the check reads
    Parameter::sys.nproc, populated by Parameter::set_pal_param in main.cpp
    before input reading). One expected warning: the header read_input_tool.h
    gains an <algorithm> include, required by the new inline helper's std::max.
    The low code_quality_score.py scores for read_inp_sys.cpp (0) and
    read_input_tool.h (48) are pre-existing historical debt (legacy macros, file
    length); the new test file scores 95.

On DSP hardware the mtblas/C_PACK vendor libraries require exactly one
MPI process per k-point pool; other configurations make the vendor
routines return inconsistent results across ranks and the SCF silently
diverges. deepmodeling#6771/deepmodeling#7035 guarded this with a WARNING_QUIT in
Driver::init_hardware, but deepmodeling#7420 removed the guard entirely to allow
bndpar+kpar combinations.

Reinstate the guard as a kpar input check under __DSP, requiring
kpar (kpar_lcao for LCAO) * bndpar == NPROC, which keeps the bndpar+kpar
use case from deepmodeling#7420 working. Also document the restriction in the kpar
INPUT description.

Fixes deepmodeling#8228
@DanielZhao0432 DanielZhao0432 added Bugs Bugs that only solvable with sufficient knowledge of DFT GPU & DCU & HPC GPU and DCU and HPC related any issues labels Oct 29, 2026
@DanielZhao0432

Copy link
Copy Markdown
Collaborator Author

@ABACUS-PR-BOT Could you please take a look to see why two tests failed? If the issue is simple, help me fix it.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One DSP input-validation defect and one nonblocking test suggestion remain. This was a static source review; no builds or runtime tests were run, and the reported Tianhe results were not independently verified.

@ABACUS-PR-BOT

Copy link
Copy Markdown
Collaborator

For the two CI failures, I couldn't retrieve the diagnostics. Please share the test names and the first failure message from each so I can assess the cause and whether a simple fix is possible.

- Extract the DSP k-point/band layout arithmetic into the inline
  ModuleIO::dsp_kpar_layout_valid (read_input_tool.h) so it is covered
  by unit tests without a DSP build.
- Skip the check when bndpar <= 0 (rejected by the bndpar input check),
  so the diagnostic message never divides by zero.
- Add source/source_io/unittests/test_read_inp_sys.cpp covering PW
  k-point pools, the LCAO kpar->kpar_lcao reset, the bndpar exemption
  from deepmodeling#7420, and degenerate (zero/negative) values.
@DanielZhao0432 DanielZhao0432 added Input&Output Suitable for coders without knowing too many DFT details and removed Bugs Bugs that only solvable with sufficient knowledge of DFT labels Oct 30, 2026
@mohanchen mohanchen added the Refactor Refactor ABACUS codes label Oct 30, 2026
@mohanchen
mohanchen merged commit 5b31a6a into deepmodeling:develop Oct 30, 2026
37 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

GPU & DCU & HPC GPU and DCU and HPC related any issues Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DSP][LCAO] SCF diverges with default kpar=1 on multiple MPI processes; the DSP guard for this configuration was removed in #7420

3 participants