Feat big renaming - #64
Open
oliviermattelaer wants to merge 29 commits into
Open
Conversation
Hard rename: the old name is gone, no symlink and no compatibility
shim. Every in-repo reference to the executable path is updated --
CI workflows, INSTALL, the release packer, the plugin launcher
generator, the mg7 madevent runtime helper, the docs, the Template
proc_card header and newprocess_mg5, and all test/comparator
subprocess invocations.
The ASCII banner header written by banner.py (and the shipped
Template/LO/Cards/proc_card_mg5.dat) keeps its 61-column box: one
padding space was dropped to absorb the longer name.
Deliberately untouched: UpdateNotes.txt (changelog of past releases)
and tests/input_files/{tt_banner.txt,madspin_event.lhe} (recorded LHE
banners used as parser fixtures). The MG5_aMC / MadGraph5_aMC@NLO
product name, python identifiers and config keys are out of scope.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This script drove the pre-2.0 workflow of generating a process by editing Cards/proc_card_mg5.dat inside a Template copy and re-running the generator. It has been non-functional for years: it execs $mgdir/mg5, an executable removed long ago (UpdateNotes.txt:611, "OM: remove ./bin/mg5 executable"), so every invocation dies at "Error no mg5 executable found" before doing any work. The script prints its own deprecation notice announcing removal in version 2.0; we are well past that. Nothing references it: no Python exporter copies or names it, and the bin/newprocess and bin/newprocess_nlo calls in the parallel tests refer to separate scripts in external MG4/NLO trees. Because Template/LO is copied wholesale into generated process directories, this only served to ship a broken script into every output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First half of the standalone/standalone_mg7 swap: free the name `standalone` by renaming the Fortran standalone exporter to `standalone_fortran`. Nothing yet answers to `standalone`; the second commit gives that name to the current `standalone_mg7`. Renamed everywhere the token names the Fortran exporter: the MG5 _v4_export_formats / _export_formats lists and the do_output config dict, find_output_type, get_default_path, the launch completion and dispatch, the citation "runnable" list, ProcessExporterFortranSA's `format` attribute and its default, the UFO_model_to_mg4 makefile selection, and the MadLoop (ML5) interface, whose `standalone` is the same Fortran standalone family. Internal callers that mean the Fortran exporter are repointed too: MadSpin's matrix-element generation and the Fortran arm of the Fortran-vs-C++ language check in process_checks. find_output_type cannot distinguish standalone_msP/msF/rw from standalone_fortran on disk, so check_launch now accepts those explicitly instead of relying on the old common `standalone` prefix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second half of the swap. `output standalone` now produces the MadMatrix
(C++/CUDA) standalone tree that `output standalone_mg7` produced before;
the Fortran one, renamed in the previous commit, is `standalone_fortran`.
Renamed in the export-format lists, the do_output config dict,
find_output_type, get_default_path (PROCMG7_SA_*), the launch dispatch,
the citation lists, the ExportCPPFactory dispatch and the MG7 arm of the
language check in process_checks.
Two guards needed a decision rather than a rename:
* complete_launch excluded the madmatrix export from the SALauncher
option set with `mode != 'standalone_mg7'`. That exclusion now reads
`mode != 'standalone'`, which looks like a tautology next to the
`startswith('standalone')` on the same line but is not; a comment
spells that out.
* the MadLoop interface has no madmatrix output, so `output standalone`
there would have been silently taken as the output *path*. It now
raises and points at standalone_fortran. The aMC@NLO interface's
forbidden-format list gained `standalone` for the same reason.
Test methods named after the old format are renamed to test_madmatrix_*,
together with the three workflows that invoke them by name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mg7 is the default output format, so its auto-named process directories now take the plain PROC_ prefix instead of PROCMG7_, and the madmatrix standalone takes PROC_SA_ instead of PROCMG7_SA_. Both prefixes are now shared with a legacy backend: PROC_ with madevent and PROC_SA_ with the Fortran standalone. That is deliberate. auto_path() hands out the first unused index, so the two backends never collide on disk, they just interleave -- verified by running all four formats into one directory and getting PROC_sm_0 (mg7), PROC_sm_1 (madevent), PROC_SA_sm_0 (madmatrix) and PROC_SA_sm_1 (standalone_fortran). Comments record the intent, since the branches now build identical names and would otherwise look like a bug. Nothing matches these directories by name, so output-type detection, which works off directory contents, is unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The merged-flavor particles print their internal model names ('_quark',
'_anti_quark', '_lepton', ...) in drawn Feynman diagrams, which is
unreadable. Substitute short labels at drawing time:
81 -> Q -81 -> Qx
82 -> L- -82 -> L+
83 -> v -83 -> vx
The lepton signs follow the model: the merged '_lepton' is built from the
electron (PDG 11, charge -1), so the particle is L- and the antiparticle
L+.
The map is applied in FeynmanLine.get_name, the single point through which
both the EPS and the SVG/JSON drawers fetch a line label, and only for the
'name' lookup, so the polarization suffix path still yields Q_R / L-_L.
The particle name/antiname themselves are deliberately NOT renamed: they
feed model['name2pdg'] and name2part, i.e. how the interface resolves user
input ('define p = g _quark _anti_quark'). Process strings are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mg7 output is no longer called madevent7, so naming its driver module
madevent.py only invited confusion with the legacy madevent tooling it
reuses. The module is imported, not copied, so this is a plain module
rename: the generated bin/generate_events now does
"from madgraph.iolibs.template_files.mg7.launch import main".
Logger names ("madevent", via logging.getLogger) are deliberately left
alone: those are shared with the legacy madevent tool drivers that the mg7
launcher reuses, and renaming them would detach the log handlers.
Verified end to end: the module imports, a generated process directory has a
bin/generate_events pointing at the new module and it runs, and
test_relaunch_switch_defaults_mg7 (which imports the module by dotted path)
passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two leftovers in madgraph/iolibs/template_files/mg7/ are no longer reachable
by any code path:
- simd/{api_simd.cpp,cudacpp.mk,Makefile} was wired to an `output --simd=<path>`
option added in 3e452ad. That option, along with the exporter attributes it
drove (matrix_element_path / from_template_simd / to_link_simd), has since
been removed from ProcessExporterMG7; the only surviving references were the
commented-out from_template_simd/to_link_simd block, which is dropped here so
no comment points at files that no longer exist. Nothing else copies, links
or includes these three files: madmatrix builds its own cudacpp.mk from
template_files/madmatrix/madmatrix.mk, and the `simd_vector_size` run_card key
is a madspace runtime setting (ms.set_simd_vector_size), unrelated to this
directory.
- generate_events did `from internal.madevent import main`. No exporter has ever
produced a bin/internal layout for mg7 (dirs_to_create is bin/src/lib/Cards/
SubProcesses), the string `internal.madevent` occurs nowhere else in the tree,
and the file appears in no from_template list. The real bin/generate_events of
an mg7 output is written inline by ProcessExporterMG7.copy_template and points
at madgraph.iolibs.template_files.mg7.launch.
gridpack.py and train_madnis.py were also examined and are kept: gridpack.py is
copied into the gridpack bin/generate_events by launch.py, and train_madnis.py
is imported by launch.py as `from .train_madnis import train_madnis`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Since standalone was given to the MadMatrix (C++/CUDA) exporter, the MadLoop and aMC@NLO interfaces stopped recognising it as a format at all, so the generic check_output would have taken it as the output *path* and quietly created a directory named standalone. The previous commit guarded that by refusing the format outright; accept it as an alias instead. ML5: output standalone now generates the MadLoop Fortran standalone, exactly as output standalone_fortran does. Verified by diffing the full recursive file lists of two generated trees for g g > t t~ [virt=QCD]: identical. aMC@NLO: standalone_fortran is itself a forbidden format there (an NLO process cannot be exported standalone in any form), so the alias is applied before the forbidden-format check and standalone is dropped from that list. The result is the single canonical error, "You generated a NLO process, which cannot be exported in standalone_fortran mode", rather than a second parallel message that could drift from it. Unit suite: 892 tests, only the two known pre-existing failures (testIO_UnitProcOutputIOTests, test_DensityMatrixObservables22 / no scipy). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
9 tasks
…fortran The two acceptance tests that validate the madmatrix (`standalone`) C++/CUDA output against an independent backend used `standalone_cpp` as the reference. Repoint them at `standalone_fortran` so the coverage survives the upcoming removal of the user-facing `standalone_cpp` format. - test_madmatrix_vs_cpp -> test_madmatrix_vs_fortran (p p > e+ e- QCD=0): the reference arm now builds Source + `make check` and runs ./check, the same convention the other Fortran cross-checks use. - test_standalone_cpp: the MSSM `g g > go go` cross-check now takes its 5000 GeV reference from standalone_fortran, and additionally asserts the hard-coded 6.4739191 reference against the Fortran backend at the auto-bumped default energy. Verified by hand that standalone_fortran reproduces the C++ numbers exactly for MSSM g g > go go: 6.4739191277160320 (default) vs the C++ 6.4739191e+00, and 5.7733902125148244 vs 5.7733902e+00 at 5000 GeV. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`output standalone_cpp` is gone; `standalone` (MadMatrix C++/CUDA) and `standalone_fortran` cover the two standalone outputs a user wants. The exporter classes are NOT removed: ProcessExporterCPP / OneProcessExporterCPP / UFOModelConverterCPP in madgraph/iolibs/export_cpp.py are the base classes of the madmatrix exporter, and `check language` drives ProcessExporterCPP directly through the internal API. Both keep working unchanged. Interface: - check_output now raises InvalidCmd for `output standalone_cpp` with a message pointing at `standalone` / `standalone_fortran`, the same way `pythia8` and `madweight` already do. Without that guard the removed name would silently be taken as the output *path*. - dropped from _export_formats (class level and setup()), from the do_output config table, from the `path == 'auto'` whitelist, from the get_default_path prefix table (the PROC_SA_CPP_ branch; the remaining standalone branches all produce PROC_SA_ and are collapsed), from the 'Output to directory ... done' and write_generation_citations lists, and from the do_output/launch help. - ExportCPPFactory loses its 'standalone_cpp' branch. find_output_type's `os.path.isdir(src_path)` catch-all is *not* deleted: it is what makes `launch <dir>` work for any C++-family tree with a src/ directory, matchbox_cpp in particular (which is reachable from ./bin/madgraph -- the class-level _export_formats includes it, and setup() only runs when switching back from ML5/aMC@NLO). It now returns 'standalone_cpp_family', with master_interface.do_launch updated in the same commit; the name still starts with 'standalone' so do_launch keeps routing it to the SALauncher. group_subprocs.DecayChainSubProcessGroup.group_amplitudes normalised its `criteria` against a list of *output format* names, but criteria is an exporter's grouped_mode -- only False / 'madevent' / 'madweight' ever reach it. Replaced by the same `if not criteria` normalisation its non-decay sibling uses. Tests: - test_standalone_cpp, test_standalone_cpp_output, test_standalone_cpp_output_consistency and the C++ arm of test_standalone_flavor_mask now go through a new TestCmdShell2._output_standalone_cpp helper that drives ProcessExporterCPP through its internal API, so the scalar-C++ coverage (including the only scalar-C++ flavor-mask coverage) survives. The helper reuses cmd._curr_matrix_elements the way do_output does: rebuilding a HelasMultiProcess from the same _curr_amps does not give the same matrix elements back (decay chains are lost), which is what test_standalone_cpp_output_consistency checks. - tests/parallel_tests/me_comparator.MG5_CPP_Runner is deleted: it drives an out-of-process `output standalone_cpp` proc_card, which cannot be converted to the internal API. Its only caller, sample_script.py, constructed it and never passed it to set_me_runners, and no workflow runs that script, so no live coverage is lost. Test names are unchanged, so no workflow step needed updating. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bin/madgraph wrote its readline history to mg5history in the shared state
directory ($XDG_STATE_HOME, ~/.local/state, or the legacy ~/.mg5). An
MG5_aMC install writes bin/mg5_aMC history to that same path, so anyone with
both installed had the two command sets interleaved into one file, each run
truncating the other to the last 100 entries.
MG7 now writes mg7history instead. mg5history is read once as a fallback
when mg7history does not yet exist, so an existing MG5 user does not start
from an empty history, and mg5history is never written to again.
Left alone deliberately: the me5history and mw5history files used by
Template/{LO,NLO}/bin and MadWeight. Those launchers are madevent, aMCatNLO
and MadWeight, which are the same tools in an MG5 and an MG7 install, so
sharing a history between them is correct rather than a collision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mg7 run_card already defaulted dynamical_scale_choice to half_transverse_mass, but that setting was inert: fixed_ren_scale and fixed_fact_scale both defaulted to true, so every default mg7 run used a fixed scale of 91.188 GeV. The launcher passes these through as ren_scale_fixed / fact_scale_fixed, which gate whether the dynamical choice is consulted at all. Flip both defaults to false so HT/2 is what a default run actually uses. The 91.188 values stay as the fallback for users who turn either switch back on. A lepton-collider process still pins fixed_fact_scale (set in check_validity): with no PDF there is no meaningful factorisation scale. The renormalisation scale goes dynamical there too. The cross-section references in check_xsec_processes_reference.json were generated with the fixed scale. They remain valid because test_check_xsec_processes_mg7._edit_run_card rewrites the two switches back to true before running -- until now that rewrite matched nothing and was dead defensive code, and it is what keeps roughly forty reference values honest, so its comments are updated to say so. The mg7 cross-section tests in test_cmd_madevent.py are unaffected: their helper already forced both switches to false, so their reference values were always taken with HT/2 and that rewrite simply becomes a no-op. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_dyn_scale_legacy translates an mg7 dynamical_scale_choice name back to the legacy integer code used by the madevent-side tools. It mapped transverse_energy to 3, which is HT/2, rather than to 1, the sum of transverse energy. The inverse map _LO_DYNSCALE_MAP has always had 1 -> transverse_energy, so an LO run_card carrying dynamical_scale_choice = 1 round-tripped through mg7 and came back as 3, silently changing the scale. The comment above the map said "mapping to confirm"; it is now confirmed against the codes documented on the LO run_card and both maps round-trip for all four values. This was latent until the previous commit: with fixed scales as the mg7 default, _legacy_compat returned -1 before ever consulting this map. Now that a default mg7 run is dynamical, the map is live. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
madspace writes events.lhe uncompressed. LHE is large and very
compressible, and madevent has always stored its events gzipped, so
compress the file once combine_to_lhe has produced it. On a 2000-event
e+ e- > mu+ mu- run this takes 1569291 bytes down to 135001, a factor
11.6, and the archive passes gzip -t with all 2000 events intact.
Nothing downstream needed changing: _find_event_file already looked for
both events.lhe and events.lhe.gz, the time-of-flight post-processing
already keys off need_zip = lhe_path.endswith('.gz') to re-compress what
it rewrites, and the mg7 acceptance tests either gunzip the .gz when the
plain file is absent or iterate over both names. misc.gzip is used rather
than a local implementation so that files over 256 MB go through the
external multithreaded tool, as elsewhere in the codebase.
Verified: test_systematics_mg7 and test_add_time_of_flight_mg7 both pass
with this change; the latter exercises the gunzip/rewrite/re-gzip round
trip.
Known interaction, not introduced here but now on the default path:
reweight_interface.do_import gunzips a .gz input WITHOUT keep=True, so it
consumes the event file it was given, and if the banner then comes back
empty it prompts with default 0 under timeout=0 and fails on open('0').
That branch used to be rare for mg7 output and is now the normal one.
test_mass_reweighting_mg7 cannot arbitrate this locally: it fails on this
machine both with and without this change (the known macOS f2py/lhapdf
breakage), differing only in which symptom it reports.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MG7RunCmd._prepare_run_dir materialises the unweighted_events.lhe.gz name
that the madevent post-processing tools expect, by gzipping the mg7 event
file. That assumed the source was the plain events.lhe. Since events.lhe is
now written gzipped, it produced a doubly-gzipped file: each consumer
decompressed it once, got gzip bytes rather than LHE text, and parsed zero
events. lhe_parser.split then raised
UnboundLocalError: cannot access local variable 'i'
because its event loop never ran. This took out every tool that reads the
events -- Pythia8, Rivet, reweighting and both MadSpin jobs.
Copy the file when the source is already compressed, and keep gzipping only
a plain source, so both forms still work.
Verified by driving _prepare_run_dir directly on both source forms: each
produces an unweighted_events.lhe.gz that lhe_parser.EventFile reads as a
9271-byte banner and 2000 events. Reproduced the original fault the same
way first: the doubly-gzipped file gives banner length 0 and 0 events.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ~40 reference cross-sections in check_xsec_processes_reference.json were produced with NNPDF23_lo_as_0130_qed, but the test only pinned the scale and took the PDF from whatever the run_card.toml template defaults to. Changing that default therefore silently invalidated every reference value. Pin the set explicitly into each generated Cards/run_card.toml, next to the existing fixed-scale pins, and route the skip guard through the same _REFERENCE_PDF constant so the set the test requires on disk and the set it runs with cannot drift apart. No reference value changes: the pinned set is what the template still defaults to, so this is a no-op today and pure protection for tomorrow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mg7 run_card.toml shipped NNPDF23_lo_as_0130_qed, a 2012 LO set with alpha_s(M_Z) = 0.130. Move the default to NNPDF4.0 LO. The MC-generator variant NNPDF40MC_lo_as_01180 (lhaid 338500) is used rather than NNPDF40_lo_as_01180: it is a single member and ~0.7 MB against ~54 MB, which also keeps the CI grid cheap. It is 5-flavour; NNPDF4.0 has no 4-flavour LO counterpart, so this is used unconditionally and no scheme-dependent choice is made anywhere. tests/parallel_tests/madevent_comparator.py pins lhaid so the madevent side of the mg7-aligned comparison uses the identical set; it moves to 338500 and must stay in sync with the banner.py default. Three acceptance references move. alpha_s(M_Z) drops 9%, and the NNPDF4.0 quark and gluon luminosities are smaller than NNPDF2.3's at the x these processes probe, so every one drops. Each was checked with a same-code A/B (one output directory, one run per PDF set) so the shift is attributable to the PDF alone and not to a regression: u u > u u 1.277e+06 -> 7.76e+05 -39% A/B 1.272e+06 -> 7.79e+05 u q > u q QCD=0 4428 -> 3730 -16% A/B 4380 -> 3688 g g > b b~ HEFT 3.754e+08 -> 1.820e+08 -51% A/B 3.708e+08 -> 1.820e+08 u q > u q has QCD=0, so its -16% is pure PDF. u u > u u carries alpha_s^2 (-18%) on top of the smaller u density. The HEFT process is gg-initiated at a ~20 GeV dynamical scale, where the gluon luminosity is down ~25% and alpha_s^2 down ~22%. test_generation_from_file_1_mg7 (p p > go go) is NOT re-referenced: with this set the integration returns nan from the first survey iteration and never terminates. mg7 divides by the per-flavour parton density when it weights the initial-state flavour channels, and NNPDF40MC returns exactly 0.0 for c/b below their thresholds and at large x, giving 0/0; NNPDF23's floor there is ~6.5e-15, which is why this never showed. Bisected on the grid file: replacing every literal 0.0 with 1e-30 makes the process integrate cleanly. The reference is left at the old value with the diagnosis recorded next to it, so the failure stays visible. The mg7 tests guard on the default set being installed, and the heptools cache predates this change, so restore_heptools_lhapdf now fetches the set when it is absent (~0.7 MB, non-fatal, a no-op once cached); warm_cache.yml bakes it in alongside NNPDF23, which check_xsec_processes_mg7 still pins deliberately. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
p p > go go returned integral: nan(nan) from the first survey iteration and
then never terminated, killing test_generation_from_file_1_mg7. The trigger was
the switch of the mg7 default PDF to NNPDF40MC_lo_as_01180, which returns
exactly 0.0 for c/b below their thresholds and at large x (b from x >= 0.933 at
Q = 600 GeV) where NNPDF2.3 still had a ~1e-15 floor. The ~600 GeV gluino pair
forces large x, so the process hits it immediately.
kernel_sample_discrete_probs weights the initial-state flavour channels by the
parton densities and divides by them twice, both unguarded:
(A) probs[i] / prob_norm, which is 0/0 -> NaN when every density is zero;
(B) det = 1 / prob_out, which is 1/0 -> inf when the selected option has
probability zero. Reachable even when other options are non-zero, since
cum_prob accumulates in floating point and can end just below r, which
selects a trailing zero-probability option.
A channel whose density is exactly zero cannot contribute, so its correct
weight is zero, not inf and not NaN. Both divisions now fall back to a unit
divisor and the result is forced to zero, which is what the integrator needs
since these dets are multiplied into the event weight. The same case-(A) guard
is applied to kernel_sample_discrete_probs_inverse (whose det is the
probability itself, so zero already means zero weight) and to the matching
backward kernel, whose gradient is zero where the forward det is identically
zero.
The guards test "!= 0" rather than "> 0" so that every non-zero divisor,
including a negative one, divides exactly as before and keeps the sign of its
weight; they are inert unless the divisor is literally zero. Nothing is done to
the input densities: flooring those would alter the physics everywhere else.
A "> 0" variant was tried first and changed the NNPDF2.3 result badly enough to
abort the run.
Everything is branch-free and type-generic, so it holds for the scalar CPU,
SIMD, CUDA and HIP instantiations alike.
Re-references test_generation_from_file_1_mg7 to the default PDF. Measured over
6 runs: 3.7864 +- 0.0011, single-run error ~0.003, so the 1% tolerance is a ~12
sigma check; a 1e-30-patched grid independently gives 3.7887. The old 5.024 was
the NNPDF2.3 madevent value, and pinning NNPDF2.3 in the run_card still gives
5.0235 with this fix in place, i.e. 0.003 sigma from the same result before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
@theoheimel @Qubitol The two things, I did not do is
|
Member
|
I guess the dlopen and backends is related to renaming the devices clearly:
Or any other reasonable name. |
The CPU/SIMD backend names of the madmatrix (cudacpp) backend are renamed:
cppauto -> cpu
cppnone -> cpu_scalar
cppsse4 -> cpu_128b
cppavx2 -> cpu_256b
cpp512y -> cpu_512b_y
cpp512z -> cpu_512b
cuda and hip are unchanged. The bare 'cpp' token stays as it was: an alias
for the auto backend accepted by the Python option lists (launch_plugin,
trex) and resolved to BACKEND=cpu before make is invoked. It is still not a
member of the makefile's SUPPORTED_BACKENDS, exactly as before.
Structural changes beyond the literal rename, in madmatrix.mk and in the
MadtRex driver/runner makefiles:
- DIRTAG and TAG stripped the literal 'cpp' prefix of the backend to build
the build-lockfile tag ('cppavx2' -> 'avx2'). They now strip 'cpu_'
instead ('cpu_256b' -> '256b', 'cpu_scalar' -> 'scalar'), which keeps
the previous meaning: the tag is the short backend name, with cuda and
hip left unchanged. Without this the patsubst would silently stop
stripping and the tag would change meaning. DIRTAG has no consumer in
this repository (the MadtRex makefiles only mention it in comments and
derive their build directory from CUDACPP_BUILDDIR); TAG is the one that
actually names .build.<TAG> lockfiles and the all.<TAG> default goal.
- The "sanity check" that a non-GPU backend really is a CPU backend used
$(findstring cpp,$(BACKEND)); it now uses $(findstring cpu,$(BACKEND)),
otherwise every C++ build would hit the INTERNAL ERROR branch.
Build directories become build.cpu_scalar, build.cpu_128b, ... (they are
derived from $(BACKEND)); nothing in this repository greps for the old
build.cpp* names. The make target aliases (bldnone, bldsse4, bld512y, ...)
and the $(PROG)_cpp program suffix are deliberately left untouched: they are
not backend names.
The mg7 run_card 'devices' default changes from ["cppnone"] to
["cpu_scalar"]. This is a user-facing default change: existing generated
directories whose run_card.toml still names an old backend will now fail at
build time with the makefile's supported-backends error. No deprecation
alias layer is added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Qubitol
requested changes
Aug 14, 2026
Qubitol
left a comment
Qubitol
left a comment
Member
There was a problem hiding this comment.
I think we should completely remove cpp as a backend, since it might be confusing.
I marked the various lines in which it is present, but it may appear elsewhere, especially because the commit message explicitly states that cpp was kept and it is translated to cpu silently at some point.
The previous rename kept `cpp` as a Python-only alias resolving to `cpu`.
It was never a makefile backend (`make BACKEND=cpp` has always failed), so
it only ever existed as a pseudo-value in the Python layer, and silently
translating it is exactly the confusion the rename was meant to remove.
Drop it everywhere it appeared as a backend value:
- the `devices` option comment in the mg7 run_card.toml template, in
RunCardMG7.default_setup and in the tests/input_files copy
- the `cudacpp_supported_backends` lists and the run_card comment line in
madmatrix/launch_plugin.py, now a single CUDACPP_SUPPORTED_BACKENDS
constant
- the tRex `do_change` backend list in madmatrix/trex.py
The default value of madevent's `cudacpp_backend` was `cpp`; it becomes
`cpu`, which is a real makefile backend with the same auto-detecting
meaning. Existing process directories still carry `cudacpp_backend = cpp`
in their run card, and the generic ConfigFile `allowed=` handling reacts to
an out-of-range value by warning and silently keeping the previous value --
i.e. such a run would quietly build on a different backend. CPPRunCard now
overrides __setitem__ for this one parameter and raises InvalidRunCard
instead, naming the supported backends and the old->new mapping.
C++ exporter names, the `cpp` compiler key, and cudacpp*/CPPProcess/.cpp
identifiers are unrelated to the backend value and are left untouched, as
is madmatrix/contrib (vendored upstream harnesses targeting an external
MG5aMC checkout).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
main brought 21 commits, mostly the FD gauge work in aloha/madmatrix. Only
tests/acceptance_tests/test_cmd.py conflicted, and not over a disagreement:
main added two new tests (test_standalone_mg7_fd_vs_fortran and
test_standalone_mg7_fd_simd_lanes) immediately above the test this branch had
renamed. Both sides are kept.
main is written in the vocabulary this branch renames, so the incoming tests
were translated: test_standalone_mg7_fd_* -> test_madmatrix_fd_*,
output standalone_mg7 -> output standalone, and the backends cppnone/cppsse4
-> cpu_scalar/cpu_128b (cppauto -> cpu in the prose).
Three stale references lived in files that merged cleanly, so git never
flagged them: testIO_FDgauge_madmatrix called generate_fd('standalone_mg7'),
a docstring named both new tests by their old names, and
acceptancetest_mg7.yml invoked them by their old names. All repointed. The
workflow one mattered most: a step naming a test that no longer exists is a
job that passes while running nothing.
One translation trap, caught by running the tests rather than by review: on
main, `standalone` is the Fortran exporter, while on this branch it is
madmatrix and Fortran is `standalone_fortran`. So the incoming test's
pre-existing `standalone` references meant the opposite of what they now say.
Its Fortran reference arm (./check plus a Source build) and the make-check
dispatch keyed on the format name are now standalone_fortran; the arms that
were standalone_mg7 stay as standalone. Without this the test looked for a
Source directory inside a madmatrix tree and died on FileNotFoundError.
Verified: test_madmatrix_fd_vs_fortran, test_madmatrix_fd_simd_lanes and
test_madmatrix_vs_fortran all pass, and the unit suite is 892 tests with only
the two known pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
@Qubitol Thanks — Branch is now at
|
misc.sprint tags its output with [file at line N], so the statements that actually add noise were collected by running the operations rather than by grepping: output madevent / standalone / mg7 / standalone_fortran, widened to a decay chain, a merged-flavor process and MSSM. The firing set was stable at fifteen sites, all in the output phase. Fourteen are removed here. None sits before a raise, and none is in the 3.7.2 branch: madmatrix/output.py and madmatrix/model_handling.py do not exist there, 3.7.2 has no sprint at all in export_cpp.py, and the two in its helas_objects.py are both commented out. What goes is the "Entering ProcessExporterMadMatrix.*" traces and their type(...) dumps, an argname trace, two config-map dumps, the "Exporting model to C++ standalone format" line, raw coups_dep and type_param_dict dumps, and a merged-particle PDG trace. The fifteenth, misc.py sprint(error) in the desktop-notification helper, is kept: it is in 3.7.2 as well, and it is the only trace of an exception that is otherwise swallowed by the except/pass around it. Launch was already quiet -- bin/generate_events for both madevent and mg7 produced events with no sprint output at all -- so nothing there needed touching. Each removed line was checked to be a complete statement and not the sole body of its block. After the change the same four outputs still build and only the misc.py sprint remains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ring When the same lorentz name is defined twice, import_ufo compares the two structure strings and warns when they differ. The comparison sits right after a block that renumbers the indices of the vertex, and that renumbering can reorder the arguments of a symmetric function: importing the sm model reports Metric(3,2) against Metric(2,3) and Metric(4,2) against Metric(2,4), which are the same object written two ways. The warning therefore fired on every check run while telling the user nothing. Add models/lorentz_symmetric_structures.txt, listing the structure functions whose arguments commute, and canonicalize both sides by sorting the arguments of those functions before comparing. The file stores the rule rather than the individual pairs: the indices come out of the renumbering, so Metric(1,3), Metric(-1,2) and so on are all equally possible, and one line covers them all. The canonicalisation is deliberately narrow. It rewrites only calls whose arguments are all plain (possibly negative) integers, so a nested expression is never touched, and it applies only to the functions named in the file, which carries a warning that listing a non-symmetric one would silence a real redefinition. Kept as misc.sprint rather than promoted to a logger warning, so the line stays identical to the one in 3.7.2; what changes is only how often it has something to say. The new test class covers both directions, since suppressing the noise is only half of it: the Metric reorderings compare equal, including with summed (negative) indices which have to sort numerically rather than as text, while a different index, a reordered Gamma or ProjP, and a mixed case where only the symmetric half matches are all still reported. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replace models/lorentz_symmetric_structures.txt with an is_symmetric attribute
on the aloha objects: False on FactoryLorentz, True on Metric.
canonicalize_lorentz_structure now scans the expression for calls and asks the
structure whether its arguments commute, instead of consulting a list kept in a
separate file.
The object already knew: Metric.get_unique_name sorts its two indices, which is
exactly the statement that g^{mu nu} is symmetric, only expressed inside a
method. The attribute states it once, next to that method, and any other
consumer can now read it rather than rediscover it.
An unknown name answers False, so a structure aloha does not define is never
taken for symmetric.
Behaviour is unchanged: the two spellings of one Metric still compare equal,
including with summed (negative) indices, while a different index, a reordered
Gamma or ProjP, and a mixed case where only the symmetric half matches are all
still reported.
The aloha parallel suite has 9 pre-existing errors (NameError: TMP0, from tests
that eval generated expression strings). Checked against a detached worktree at
the previous commit: 124 tests and the same 9 error names on both sides, so
they are untouched by this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
oliviermattelaer
force-pushed
the
feat-big-renaming
branch
from
August 17, 2026 05:38
d2e4fe0 to
ba9b403
Compare
Member
15 tasks
Raise the minimum interpreter from 3.7 to 3.12 in every gate that guards an
entry point: bin/madgraph and bin/create_release.py, the MadSpin launcher, the
LO and NLO Template launchers (madevent, generate_events, aMCatNLO,
calculate_xsect, shower, the gridpack runner), and the two in-process checks in
madevent_interface and amcatnlo_run_interface. INSTALL is updated to match.
madmatrix carried its own, looser gate (minpython = (3,8), from the version
that made {} preserve insertion order). It is raised to the same 3.12 so that
the plugin can never be the reason a supported interpreter is refused, and its
comment now points at bin/madgraph rather than restating a rule of its own.
bin/madgraph also warned separately when version_info[1] < 10 that support for
3.9 and below had ended. That is unreachable once the hard gate is 3.12, and it
was testing only the minor number, so it would have fired on a hypothetical
4.0. Removed.
Left alone: madspace/pyproject.toml keeps requires-python = ">=3.11". It is a
separately packaged library and the two constraints do not conflict, MadGraph
simply being the stricter of the pair.
The template launchers ship into generated process directories, so new outputs
carry the new gate; directories generated earlier keep the one they were
written with. Checked on a fresh output that bin/madevent has it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Qubitol
reviewed
Aug 17, 2026
| if path: | ||
| type = self.cmd.find_output_type(self, path) | ||
| if type in ['standalone', 'standalone_cpp', 'pythia8', 'madevent']: | ||
| if type in ['standalone_fortran', 'standalone_cpp_family', |
Member
There was a problem hiding this comment.
Are we keeping old standalone_cpp (renamed as standalone_cpp_family) at the end?
Contributor
Author
There was a problem hiding this comment.
not as a user backend... but since all cpp output are based on that one (pythia8. matchbox and madmatrix) we can not fully remove it. So the CI can still use it in practise.
Qubitol
reviewed
Aug 17, 2026
Member
There was a problem hiding this comment.
Is this change related to renaming?
Contributor
Author
There was a problem hiding this comment.
Indirectly, yes without it a lot of CI are failing due to the new PDF set having some zero probability...
But I dought that I did a dedicated PR for that, (which will need to be merged first then) will double check that point before merging.
Contributor
Author
There was a problem hiding this comment.
So indeed this will be superseeded by #66
When that one will be merged, we will need to drop the change on this side for their fix but we need to keep the change of the reference value for the test which is related to the change of PDF.
Qubitol
reviewed
Aug 17, 2026
Member
There was a problem hiding this comment.
Python version has been bumped to 3.12. I'm ok with that, but are we all onboard?
Contributor
Author
There was a problem hiding this comment.
I think 3.11 is the minimum for madspace in any case ...
Now we can decide to the version that we want
Qubitol
reviewed
Aug 17, 2026
| # options: cuda, hip, cpp, cppnone, cppsse4, cppavx2, cpp512y, cpp512z, cppauto | ||
| devices = ["cppnone"] | ||
| # options: cuda, hip, cpu_scalar, cpu_128b, cpu_256b, cpu_512b_y, cpu_512b, cpu | ||
| devices = ["cpu_scalar"] |
Member
There was a problem hiding this comment.
Shall we make cpp the default instead?
Read it like we automatically pick the best vectorisation option for you.
Qubitol
requested changes
Aug 17, 2026
Qubitol
left a comment
Qubitol
left a comment
Member
There was a problem hiding this comment.
I think it's really nice, I like the new ./bin/madgraph.
I'd just propose one change: to make cpp the new default, instead of cpp_scalar.
Then, few more comments on other things I found which I want to make sure everyone is aware.
I found out there are a lot of files where a bunch of sprints log messages have been removed, the Python version has been bumped to 3.12, and there are a lot of changes in tests and other parts of the code which are doing much more than a renaming.
Just wanted to point it out to be sure we are on the same page on everything is going to be merged.
Contributor
Author
This was on the list for the alpha release (but indeed not in the issue for the renaming)
That was also in the list of the alpha release.
Some tests had to be refactorised/updated/... to match/correct the change of the renaming. The PDF bug related to the new PDF set could be shipped via his own PR indeed. |
The mg7 run_card shipped devices = ["cpu_scalar"], so a default run used the scalar code path and left every SIMD backend unused unless the user knew to ask for one. Default to "cpu" instead, which resolves to the widest vectorisation the host supports. This works because launch.py already resolves it: a devices entry becomes a library file name (lib/libmadmatrix_<proc>_<device>.so) and no ..._cpu.so is ever built, so the loader runs "make -n BACKEND=cpu detect-backend" and uses the resolved name for the path, while the compile itself is driven with BACKEND=cpu. Without that step defaulting to "cpu" would have looked right and then failed to find its library. The options comment now lists cpu first and says it auto-detects, in the param and in the run_card template: the name alone does not convey that it picks a SIMD width, and it is what a user gets without asking. Only this one place sets the default. The other cpu_scalar mentions are tests that pass an explicit backend, the check-language backend loop and make targets, none of which should follow it. Checked on a real run: the generated card carries devices = ["cpu"], the run resolves to cpu_128b on this host and builds libmadmatrix_P1_epem_mupmum_cpu_128b.so. Forcing cpu_scalar in the same directory gives 0.1042841 +- 0.0006047 against 0.1041746 +- 0.0006001 for the auto backend, a 0.13 sigma difference, with both libraries present so the switch is real. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Default value changed to cpu Would need to land first PR #66 before merging this one |
Qubitol
marked this pull request as ready for review
August 18, 2026 08:58
Qubitol
self-requested a review
August 18, 2026 08:58
Qubitol
approved these changes
Aug 18, 2026
Qubitol
left a comment
Qubitol
left a comment
Member
There was a problem hiding this comment.
Good for me to merge.
We would need to do some conflict resolution, but other than that the changes are good.
main brought 8 commits: the SubProcesses parallel build (#67), the dead CI test names and the test_manager change that makes a name matching nothing an error (#70), the aloha TMP eval namespace (#69), a more robust cppauto resolution and a compile log for the mg7 SubProcesses build. Four conflicts, all resolved towards main where it restructured something and towards this branch where it only renamed: - mg7/launch.py: main moved the per-device resolution out of the loop into resolve_cppauto_backend(), with real error handling, and builds the libraries up front. Its version is taken whole; the inline quick fix this branch had renamed no longer exists. - aloha.yml: this branch renamed the launcher, main fixed the test selector (dropping a testIO_aloha.* that never existed and a missing space that made the -e exclusion inert). Both kept. - acceptancetest.yml: main deletes three jobs naming tests that no longer exist; this branch had only renamed the launcher inside them. Deletion kept, which matters now that a name matching nothing is an error. - madgraph_interface.py: main changed the detection file from madmatrix.mk to madmatrix_standalone.mk, because after its restructure both exporters write madmatrix.mk and only the standalone one writes the other. Its file name with this branch's format name, otherwise a plain mg7 output is detected as standalone. main is written in the backend vocabulary this branch renames, and two of those files merged without a conflict: - launch.py arrived with seventeen cppauto references (the function and its name, the make invocation, the regex, the messages), translated to cpu. - madmatrix.mk gained BLDAVXS = cppnone cppsse4 ... from the parallel build. The bldavxs target happened to survive, since $(subst cpp,,cppnone) still gives bldnone, but BLDBACKENDS feeds commonlib.% which runs BACKEND=$* directly, so bldcommonlib died with Invalid backend BACKEND='cppnone': supported backends are ... 'cpu' that is, the dispatcher the parallel build routes through. BLDAVXS now holds the new names and the bld* targets are mapped explicitly, because they keep their historical short names (bldnone, not bldcpu_scalar) and three tests invoke make bldnone directly. Checked: bldall and bldcommonlib resolve to cpu_scalar and cpu_128b on this host with no invalid backend, and a full mg7 run goes from devices = ["cpu"] through "Device 'cpu' resolved as 'cpu_128b'" to events, with main's new compile_subprocesses.log in place. Unit suite 901 tests with only the two known pre-existing failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
main brought 120 commits: the madspace codegen work (#77), the gluon amplitude optimisation (#57), the numerator atomicadd (#59), the madnis paper branch (#68) and the zero-density flavour fix (#66). One conflict, madspace/src/kernels/discrete.hpp, and it was expected: this branch guarded the divisions there so a zero parton density gave det = 0, and #66 replaced that with a mask which never samples a zero-probability entry in the first place. main's version is taken whole. It is the better fix -- it also handles negative pdf values, and it avoids the +inf that det = 0 sent into the adaptive_prob column. The p p > go go reference stays at 3.786, which belongs to the NNPDF40MC default this branch introduces and has no counterpart on main; test_generation_from_file_1_mg7 passes, so main's kernel reaches that value by its own route. One regression against the renaming, in code that merged without a conflict: madgraph_interface.py arrived from the gluon work with _gpu_me_formats = ['mg7', 'mg7_v5', 'standalone_mg7'] and standalone_mg7 is the name this branch removed; that exporter is now 'standalone'. The list decides whether merge_quartic_vertices='auto' resolves to the gpu 'slots' layout or the cpu 'speed' one, and the fallback beside it cannot cover the gap: all three formats declare exporter 'cpp', so the == 'gpu' test is false for every one of them and the name list is the only way in. Left alone, output standalone would have been generated with the cpu layout. Renamed to 'standalone'. docs/gluon-quartic-plan.md came with the old vocabulary too: cppsse4, a pure rename to cpu_128b, and a session written standalone -> standalone_mg7 -> standalone, which needs the swap applied rather than a substitution, since 'standalone' means the fortran exporter on main and the madmatrix one here. Checked for the rest: no other old backend name, no PROCMG7_, no bin/mg5_aMC, and none of main's python, yml, mk or sh additions use 'standalone' in the fortran sense. Unit suite 927 tests with only the two known pre-existing failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is associated to issue #14
current status is
Rename the 🚀 to the prompt name we will decide to use(done in another branch)~/.config/mg5_history -> ~/.config/mg7_history
change default value for run_card