Running locally

On a workstation with enough cores and memory, the pipeline can be reproduced in parallel with:

pixi run repro-local

This is the local counterpart of repro.sh. It runs validate_lhm_coupled and export_webmap and everything upstream of them, except frozen stages: the stages that repro.sh runs, up to koppelen, and also run_lhm_coupled, validate_lhm_coupled and export_webmap. The stages and their commands are taken from dvc.yaml. It:

  1. Removes the outputs of all stages that are run, like dvc repro does, and runs dvc pull --force for everything upstream of them; no manual pull is needed.
  2. Runs each stage as soon as the stages it depends on have succeeded, so the water authorities run in parallel. If a stage fails, the stages downstream of it are skipped, and the others continue.
  3. Runs dvc commit for the stages that succeeded. If all succeeded, it runs pixi run check, a second time if hooks fixed files such as dvc.lock, and with --push also dvc push.
  4. If export_webmap ran, publishes the model to the model viewer as a preview of the pull request of the current branch, compared with main. This needs a pull request for the branch, MinIO credentials with write access in .env, and a logged in gh.

Like repro.sh, it runs the stage commands directly, since parallel dvc repro processes would contend for the DVC lock. The output of each stage is written to logs/repro/<timestamp>/<stage>.log, and a summary with the status and duration of each stage is printed at the end.

Options

  • Targets: stages to reproduce together with their upstream stages, validate_lhm_coupled and export_webmap by default. For example, pixi run repro-local koppelen only builds the coupled model, like repro.sh.
  • --jobs N: the maximum number of stages that run at once, 12 by default. Each stage is a Python process that may also run Ribasim, so watch the memory use when increasing it.
  • --stagger SECONDS: the minimum time between stage starts, 30 by default, to not overload the GoodCloud server that some stages still download from.
  • --incremental: only run the stages that DVC reports as changed, and everything downstream of them, rather than all stages. Missing outputs count as changed, so this also reruns stages that failed before. Run pixi run dvc pull first if the workspace may be behind dvc.lock.
  • --push: run dvc push for the reproduced stages if they all succeeded.
  • --no-publish: do not publish the model to the model viewer.

To keep the run going after closing the terminal, start it in for instance tmux or with nohup.