Skip to content

Refactor regional inputs and add standalone AvaDirectory migration workflow#43

Open
chhesselbach wants to merge 5 commits into
masterfrom
CH_dev
Open

Refactor regional inputs and add standalone AvaDirectory migration workflow#43
chhesselbach wants to merge 5 commits into
masterfrom
CH_dev

Conversation

@chhesselbach

Copy link
Copy Markdown
Collaborator

Commit overview

refactor: use consistent dataUtils imports

  • Replaced redundant direct helper imports with consistent dataUtils.* calls.
  • Applied the same import style across PRA and AvaDirectory modules.
  • Reduced duplicate imports without changing workflow behavior.

refactor: simplify regional inputs and fix PRA output handling

  • Made processing boundaries and regional metadata optional and modular.
  • Added automatic DEM-outline generation when no boundary is provided.
  • Removed the requirement for fixed administrative column names.
  • Made the forest raster optional for PRA delineation.
  • Fixed Step 07 GeoJSON filenames and internal layer names.
  • Fixed size-specific PRA filtering.
  • Fixed duplicated praIDpraID... raster names.
  • Fixed missing pathInputpra values.
  • Prevented praID.tif from being copied into RELArea.

rm tests

  • Removed local development tests from version control.
  • Local tests remain excluded through .gitignore.
  • This does not affect the model-chain runtime.

feat: add standalone AvaDirectory migration runner

  • Added runAvaDirectory.py for running AvaDirectory Steps 13–15 independently.
  • Added a dedicated default INI and local override support.
  • Uses a single projectRoot setting.
  • Automatically discovers current flat and old nested FlowPy structures.
  • Normalizes outputs into flat 11_avaDirectoryData and 12_avaDirectory directories.
  • Reuses completed scenario folders and processes only missing or incomplete scenarios.
  • Normalizes legacy praIDpraID... filenames during collection.
  • Added output validation so empty steps no longer report successful completion.
  • Added the pixi run avadirectory task.

@chhesselbach

Copy link
Copy Markdown
Collaborator Author

@PaulaSp3 please confirm that all is running as it should. thanks

Comment thread ati/mod0Helper/avaDirectory/avaDirBuildFromFlowPy.py Outdated
Comment thread ati/mod0Helper/avaDirectory/avaDirBuildFromFlowPy.py
Comment thread ati/mod0Helper/avaDirectory/avaDirBuildFromFlowPy.py Outdated
Comment thread ati/mod0Helper/avaDirectory/avaDirectoryWorkflow.py Outdated
Comment thread ati/mod1Release/praDelineationVeitinger.py Outdated
Comment thread ati/mod1Release/praDelineationVeitinger.py Outdated
Comment thread ati/mod0Helper/dataUtils.py
Comment thread ati/mod0Helper/dataUtils.py Outdated
@PaulaSp3

Copy link
Copy Markdown
Contributor

dynamic parameterization fails! Please check the minor modifications!

@chhesselbach

Copy link
Copy Markdown
Collaborator Author

dynamic parameterization fails! Please check the minor modifications!

Okay, I understand why the error occurs now. While testing the new structure from ATI PR #32 and the addition of runAutoAtesModelChain.py, everything was running, but I did not recognize that the intended separation of size classes in AvaScenario was not working correctly. Every input PRA layer contained all size classes because gSel = gBand.copy() overwrote the preceding size filter.

For runAutoAtesModelChain.py and dynamic parameterisation, keeping all PRAs together in one raster is required. The existing gSel = gBand.copy() provided this behavior, although the combined raster was misleadingly named as a size-5 output.

runAvaScenModelChain.py, however, requires real size separation: PRAs must be filtered according to their assigned size classes and processed separately.

When I removed the unconditional gSel = gBand.copy(), AvaScenario started separating the size classes correctly, but AutoATES/dynamic parameterisation lost the combined raster they require. I did not recognize this dependency at first. sorry!

Your proposed condition is a suitable quick fix:

  • If assignElevSize = True, filter by the assigned size.
  • If assignElevSize = False, keep all PRAs together.

This restores both workflows, although AutoATES still creates repeated or misleading size-labelled outputs.

As a cleaner long-term solution, I suggest introducing two explicit modes:

  • classified for AvaScenario: split PRAs by elevation band and size.
  • allSizes for AutoATES/dynamic parameterisation: create one combined PRA-area raster and one combined PRA-ID raster.

Example AvaScenario inputs:

pra030secE-1800-2000-size3-praAreaM.tif
pra030secE-1800-2000-size5-praAreaM.tif

Example AutoATES/dynamic inputs:

pra030secE-allPRA-maxSize5-praAreaM.tif
pra030secE-allPRA-maxSize3-praAreaM.tif

The existing sizeMax setting remains responsible for capping the size-dependent dynamic parameterisation, so no additional size-cap setting is needed. Including maxSize in the filename would also make the applied parameterisation limit explicit.

How do you like the idea? should i give it a try?

@chhesselbach
chhesselbach requested a review from PaulaSp3 July 23, 2026 08:26
@chhesselbach

Copy link
Copy Markdown
Collaborator Author

@PaulaSp3 pls check again.

  • added NumPy-style docstrings to the four new helper functions;
  • consolidated the optional-forest handling for both path modes;
  • restored the combined-PRA behavior for AutoATES/dynamic parameterisation while preserving size filtering for AvaScenario (for now easy fix! ;))

I also tested runAutoAtesModelChain.py: Step 07 now completes without the previous zero-feature error. Could you please review the latest changes again?

@PaulaSp3

Copy link
Copy Markdown
Contributor

I've created an issue (#46) for the ideas you had to solve if we write different files for different classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants