Recent grammar addition for multi-copying

[willend]

Hi @farhi,

As you know, @g5t is providing us with the potential next-generation code-generator mccode-antlr, and is in the process of trying to port the newly added SECTION COPY(part_one) COPY(part_two) grammar. (It was/is actually feature complete at least on the McStas side, see e.g. https://new-nightly.mcstas.org/2025-05-21_output.html)

Looking at what was implemented on the McStas/McXtrace side, not all aspects of the new syntax seems clear - especially in regard to what aspects are component-specific vs. instrument-specific, and the order of application of e.g. EXTEND in such a "multi-layer sandwich".

I am starting this discussion item to allow more direct and open communication on the found issues - in honesty also to avoid becoming "man in the middle". ;-)

(Also, I think once this stuff is stabilised we need to define a 'policy' on cogen changes going forward - i.e. think, exchange and design together before we start hacking away. ;-) )

[g5t]

In #2012 (comment) @farhi indicated that multiple %{ %}, COPY (identifier), EXTEND %{ %} statements would be allowed on SHARE, DECLARE, INIT, TRACE, FINALLY, and DISPLAY sections.

Investigating grammar changes:

Lexer entries renamed

Some identifiers in the lexer have been changed to include comp_, but this does not indicate that those identifiers are only available in component definition files.

	old instrument.y	new instrument.y	'copyable' in *.comp	'copyable' in *.instr
%{ %}	codeblock	codeblock	no	no
SHARE	share	comp_share	yes	no
USERVARS	uservars	comp_uservars	no	no
DECLARE	declare	comp_declare	yes	yes
INIT	initialize	comp_initialize	yes	yes
TRACE	trace	comp_trace	yes	no
FINALLY	finaly	comp_finally	yes	yes
DISPLAY	display	comp_display	yes	no

Repeated copy grammar rules

Each of the 'copyable' lexer entries now has a {entry} codeblock {entry}_copy_extend and a {entry} {entry}_copy_extend rule
where {entry} is any of ("SHARE", "DECLARE", "INIT", "TRACE", "FINALLY", "DISPLAY") or its lowercase equivalent.

Each of the {entry}_copy_extend rules can match an empty string, "COPY" TOK_ID {entry}_copy_extend, or "EXTEND" codeblock {entry}_copy_extend.

If we refer to "COPY" TOK_ID as just copy and "EXTEND" codeblock as just extend, I think that these rules boil down to

{entry} codeblock? (copy|extend)*

If my understanding is correct, that means that statements like the following are now valid:

INIT
%{
int x=1;
%}
EXTEND
%{
x=2;
%}
EXTEND
%{
x=3;
%}
COPY(a_component_that_does_something_else_to_x_on_initialize)
COPY(something_else)
EXTEND
%{
x=4;
%}

Copied contents

Each of the {entry}_copy_extend rules uses read_component to get the code block contents to copy. The implementation of read_component at least claims to only read "*.com" [sic] files, and returns a struct comp_def pointer.

Questions

1. If {entry} codeblock? (copy|extend)* is actually valid and intended; is the EXTEND keyword still necessary?
2. If it is, should COPY be more explicit too, e.g., COPY(X) EXTEND COPY(Y)?
3. Is it intended/useful that, e.g., DECLARE COPY(component_name) should be possible from inside of an instrument file?

Proposal

• Drop EXTEND (it can be kept as a reserved word but ignored) and simplify the new copy rules to the equivalent of {entry} (copy|codeblock)+

[g5t]

Before releasing a mccode-antlr that includes these changes I'd like to hear from @farhi whether

• I correctly identified the implemented/intended new syntax of {entry} codeblock? (copy|extend)*,
• copying via read_component really limits copying to be from component files,
• and if copying in an instr from component file sections is intended

If I've understood the new syntax correctly, I think that the following would also be valid (by matching "INIT" init_copy_extend)

INIT
EXTEND
%{
int x=1;
%}

further highlighting that EXTEND is now redundant in this context.

[g5t]

Also worth noting: contrary to the PR description,

SHARE COPY(Powder) COPY(Fluorescence)

is not valid syntax for the comp_share rule but almost matches "SHARE" comp_share_copy_extend and the comp_share_copy_extend rule "COPY" TOK_ID comp_share_copy_extend.
That is, the new syntax does not include parentheses.

SHARE COPY Powder COPY Fluorescence

@willend @farhi Should {entry} COPY {identifier} or {entry} COPY({identifier}) be valid?

[farhi]

You are right, a syntax with (...)is probably more explicit, and matches as well the inner use in TRACE.

[willend]

@farhi @g5t I didn't look in the details of what was implemented on McCode/main up to this point.

Could you possibly (soonish? :-) ) add / correct what is missing to achieve the (...) syntax? (Then I could launch 3.5.30 after.)

My understanding is also that some more discussion on the questions raised by @g5t above would be welcome - do we need a zoom?

[farhi]

We converge to:

• replace "COPY" by "INHERIT" for all component definition stuff in instrument.y
• also apply to "USERVARS"

[willend]

https://github.com/mccode-dev/McCode/tree/COPY_to_INHERIT_in_comps is the place. I will start on the work.