Skip to content

Multi-plants - #186

Merged
VEZY merged 236 commits into
mainfrom
multi-plants
Sep 6, 2026
Merged

Multi-plants#186
VEZY merged 236 commits into
mainfrom
multi-plants

Conversation

@VEZY

@VEZY VEZY commented Mar 18, 2026

Copy link
Copy Markdown
Member

No description provided.

@github-actions

github-actions Bot commented May 3, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main 07e958a... main / 07e958a...
bench_linux/PBP 18.4 ± 2.6 ms 28.2 ± 4.8 ms 0.654 ± 0.15
bench_linux/PBP_multiple_timesteps_MT 0.266 ± 0.066 s 0.277 ± 0.041 s 0.962 ± 0.28
bench_linux/PBP_multiple_timesteps_ST 0.269 ± 0.023 s 0.248 ± 0.022 s 1.08 ± 0.13
bench_linux/PSE 5.4 s 0.0605 h 0.0248
bench_linux/PSE_multirate_output_request_run 8.41 s 6.04 s 1.39
bench_linux/PSE_multirate_status_tracked_run 4.66 ± 0.49 s 4.2 ± 0.46 s 1.11 ± 0.17
bench_linux/XPalm_convert_outputs 1.3 ± 0.17 s 1.28 ± 0.17 s 1.01 ± 0.19
bench_linux/XPalm_run 12.5 s 18.3 s 0.682
bench_linux/XPalm_setup 1.04 ± 0.12 s 0.997 ± 0.13 s 1.04 ± 0.18
time_to_load 2.54 ± 0.05 s 2.49 ± 0.0034 s 1.02 ± 0.02
Memory benchmarks
main 07e958a... main / 07e958a...
bench_linux/PBP 0.131 M allocs: 8.11 MB 0.196 M allocs: 12.5 MB 0.65
bench_linux/PBP_multiple_timesteps_MT 3.73 M allocs: 0.237 GB 3.75 M allocs: 0.238 GB 0.995
bench_linux/PBP_multiple_timesteps_ST 3.58 M allocs: 0.213 GB 3.6 M allocs: 0.214 GB 0.994
bench_linux/PSE 0.0499 G allocs: 3.7 GB 3.06 G allocs: 64.9 GB 0.0569
bench_linux/PSE_multirate_output_request_run 0.0851 G allocs: 3.5 GB 0.0741 G allocs: 2.45 GB 1.43
bench_linux/PSE_multirate_status_tracked_run 0.0607 G allocs: 1.93 GB 0.0617 G allocs: 1.94 GB 0.991
bench_linux/XPalm_convert_outputs 6.62 M allocs: 0.436 GB 6.62 M allocs: 0.436 GB 1
bench_linux/XPalm_run 0.104 G allocs: 6.71 GB 0.143 G allocs: 8.4 GB 0.798
bench_linux/XPalm_setup 1.26 M allocs: 0.0375 GB 1.4 M allocs: 0.044 GB 0.854
time_to_load 0.145 k allocs: 11 kB 0.145 k allocs: 11 kB 1

@VEZY
VEZY marked this pull request as draft May 4, 2026 12:59
VEZY added 26 commits June 3, 2026 17:25
- Added `Domain`, `SimulationMapping`, and `DomainSimulation` for composing plant, soil, scene, and future environment domains.
- Added cross-domain stream/value dependencies with `AllDomains(...)` and explicit `Route(...)` materialization.
- Added hard-domain dependencies with `HardDomains(...)`, `dependency_targets(...)`, `model_target(...)`, and `run_target!(...)` so scene models can manually run plant/soil targets, including iterative workflows.
- Added MTG-backed domain support with selectors, multi-plant domains, graph-domain output publication, dynamic topology registration, organ removal, and reparenting.
- Added `Updates(:var; after=...)` for intentional same-scale variable updates by later models.
- Added environment/microclimate backend protocol with `AbstractEnvironmentBackend`, `GlobalConstant`, `meteo_inputs_`, `meteo_outputs_`, sampling, scattering, and validation.
- Extended multirate support with `Dates`-based timesteps, inferred/explicit bindings, temporal output policies, scoped streams, meteo aggregation, and requested output export.
- Added MAESPA-style example in [examples/maespa_domain_example.jl](/Users/rvezy/Documents/dev/PlantSimEngine/examples/maespa_domain_example.jl) with two plant species, shared soil, scene-scale iterative energy balance, and hard-domain targets.
- Added docs for domain simulation, model traits, hard-domain target design, and implementation plan/handoff notes.
- Added focused tests for domains, environment backends, meteo traits, updates, and the MAESPA example.

Verification already run:
- `julia --project=test test/runtests.jl` passed: `1844 / 1844`
- `julia --project=docs docs/make.jl` passed
- `git diff --check` passed

Files to remember are untracked too, especially:
- `src/domains/domain_simulation.jl`
- `src/time/runtime/environment_backends.jl`
- `src/dependencies/update_dependencies.jl`
- `examples/maespa_domain_example.jl`
- new docs under `docs/src/dev/`
- new tests under `test/test-domain-simulation.jl`, `test/test-environment-backends.jl`, `test/test-maespa-domain-example.jl`, `test/test-meteo-traits.jl`, `test/test-updates.jl`
- remove ModelList compatibility
- replace Symbol("") mapping sentinels
- refactor repeated multirate input-resolution flow
- split the large domain runner into scheduler/routes/environment/graph/publication pieces
- replace assertions by more standard errors
Shared Status / StatusView interface helpers.
Extracted repeated test graph comparison runner.
Extracted toy tutorial MTG helpers.
Typed export plans, reverse mappings, temporal stream aliases.
Preserved renamed producer source variables in binding inference.
Added explicit domain DAG run ordering.
Added topology reindexing for RefVectors after add/remove/reparent.
Split MAESPA scene solver math from side effects and added validation tests.
Fix recursive call leading to stack overflow
compile_scene (line 638) now validates required inputs_(model) variables.
A required input is valid if it has a compiled Inputs(...)/inferred binding or already exists on the target object Status.
Missing inputs now error with application id, object id, and input name.
Added tests for missing inputs, ambiguous same-object producers, and status-provided inputs.
…cies.

Unscoped Inputs(...) / Calls(...) dependency selectors now infer scope from the consumer object:
scene consumers default to SceneScope()
non-scene consumers default to Self()
Direct public resolve_object_ids(scene, Many(...)) remains scene-wide unless within=... is explicit.
Shared cross-scope dependencies from organs now need explicit within=SceneScope(), as in the updated shared-soil test.
Added tests showing plant-level unscoped leaf inputs are plant-local, while scene-level unscoped leaf inputs are scene-wide.
Exceptional-organ Override(...).
Instance and object-level model replacement validation.
Concrete dispatch preserved for same-type parameter overrides.
explain_instances(scene) and instance membership in explain_objects.
New organs inherit instance kind and species.
Hard-called overridden models use the correct per-object implementation.
… their helpers, runtime, tests, and documentation.

This API was a failed tentative
- Overlap-weighted temporal windows, including pre-window held samples and output resampling.
- timestep_hint/meteo_hint delegation for object overrides.
- Atomic rejection of instance-root or ancestor removal/reparenting.
Dynamic organ creation repeatedly rebuilt scene bindings, selector matches, MTG IDs, and attribute/output structures—effectively rescanning the whole plant per organ. XPalm also had an ambiguous fruit-count producer binding. Incremental updates, caching, and explicit binding fixed it.
VEZY added 2 commits August 28, 2026 17:44
up MTG and PlantMeteo to latest releases (needed for this version, we previously had to use them as dev versions)
@VEZY

VEZY commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

Can we get back to release performance before merging?
-> PlantSimEngine 0.14.1, median reference (4 160 time-steps): 4,73s, 6,99 Go

@VEZY

VEZY commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Proposed v0.15.0 changelog and migration scope

The detailed v0.15 changelog prepared for this PR is organized around one central change: PlantSimEngine now uses a single compiled CompositeModel / Object runtime instead of the historical ModelMapping / MultiScaleModel execution stack.

Why this redesign

The same scenario representation can now cover one object, multiscale plants, several plants or species, shared resources, dynamic topology, multirate execution, global or spatial environments, and retained output streams. Coupling, schedules, environment bindings, writers, and lifecycle changes are compiled and can be inspected through structured diagnostics.

Main breaking changes

  • ModelList, ModelMapping, GraphSimulation, MultiScaleModel, their mapping run! methods, and mapping-era wrappers such as TimeStepModel, InputBindings, MeteoBindings, ScopeModel, and OutputRouting were removed without a compatibility layer.
  • Model packages must adopt PlantSimEngine.run!(model, status, environment, constants, context), read parameters from model, and declare inputs with Required(T) or Default(value).
  • Scenarios now use named ModelSpec applications, explicit selectors, application identities, outputs=..., and continue! / step! for timeline continuation.
  • Hard calls return CallTargets and default to publish=false; iterative controllers must publish the accepted state explicitly.
  • Environment providers use compiled bindings, trial state is passed through run_call!(...; environment=...), and accepted mutable state is committed explicitly.
  • Compiler internals and focused APIs are now separated into the Advanced, Authoring, Diagnostics, GraphEditor, EnvironmentAPI, and Evaluation namespaces.

Main additions

The release adds the unified object registry; One / OptionalOne / Many selectors; templates, instances, and overrides; compiled value and distributed-output coupling; multirate temporal policies; MTG adaptation and dynamic lifecycle operations; typed output streams and explicit retention; structured diagnostics; the serializable PlantSimEngine.Authoring API; and the package-owned agent skill.

Scientific equations can usually remain unchanged, but both model interfaces and scenario assembly must be migrated. Concrete translations for scenario structure, cross-object inputs, hard calls, multiple plants, multirate coupling, environments, dynamic organs, and output collection are documented in the CompositeModel/Object migration guide, including its compact migration table.

Historical benchmark figures are intentionally not presented as validation of the current PR head; final performance evidence should be reported separately against the eventual merge SHA.

@VEZY

VEZY commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

Can't make the benchmark pass because there is so much difference in the API between release and this PR. I did some benchmarking locally instead:

Benchmark Release PSE 0.14.1 PR PSE 0.15 PR / Release Validation / observation
Construction of 1,024 Status objects 37.209 µs 39.938 µs 1.073× Slightly slower
Read/write of 1,024 Status objects 2.125 µs 2.125 µs 1.000× Identical
Construction of a 1,024-element RefVector 171.354 µs 122.292 µs 0.714× 29% faster
Sum of a 100,000-element RefVector 356.646 µs 375.688 µs 1.053× 5% slower
Write to a 100,000-element RefVector 395.354 µs 433.208 µs 1.096× 10% slower
PSE single-rate runtime: 2,000 leaves × 720 steps, retained outputs 785.450 ms 44.758 ms 0.057× 17.5× faster, identical results
PlantBiophysics: 8,760 steps, retained outputs 83.662 ms 13.333 ms 0.159× 6.3× faster; 113,880 values strictly identical
PlantBiophysics: 8,760 steps, no outputs 12.288 ms Engine-only measurement
XPalm: complete 4,160-day cycle 3.580 s 4.259 s 1.190× 19% slower, but ~60% less memory and ~61% fewer allocations; each stack matches its scientific oracle

So overall PSE is now faster, except for XPalm where we observe a bit of a slowdown, but this is the price of hardening the computation -> better handling of new and deleted organs, and most importantly new API for multi-plants, which adds a lot more to handle because it's done with genericity in mind. So not a bad result at all!

VEZY added 3 commits September 5, 2026 21:21
Snapshot published outputs, temporal inputs, and final state so in-place
updates cannot corrupt historical values or producer state.

Support fixed subsecond periods, reject incompatible cadence grids and
nonfinite durations, and tolerate equivalent floating representations.
Preserve explicitly declared or remapped environment duration fields.

Validation: output, timing, environment, previous-step, multirate, and
lifecycle regressions passed, including existing scalar allocation checks.
Check whether run! methods accept the five-argument kernel contract,
including valid variadic implementations. Compare named output policies
independently of declaration order, matching runtime interface semantics.

Remove unused helpers from the retired status-vector execution path and
correct hard-dependency syntax in the generated model tutorial.

Validation: 141 authoring assertions and 35 Status/process tutorial
assertions passed.
Show the canonical model source in the first authoring tutorial. Correct
rate-integration examples with duration-aware reducers and explain cadence
grids, independent snapshots, and exact scientific contract matching.

Make the packaged AI skill and evaluation harness portable across Julia
hosts while respecting repository execution requirements. Remove the
requirement to implement an additional hypothesis for every authoring task.
Record the release corrections in the changelog.

Validation: documentation build and rendered examples passed; the corrected
packaged skill suite passed 112 assertions with all 15 oracles passing.
@VEZY
VEZY marked this pull request as ready for review September 6, 2026 05:44
@VEZY
VEZY merged commit 892a778 into main Sep 6, 2026
11 of 13 checks passed
@VEZY
VEZY deleted the multi-plants branch September 6, 2026 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant