Running on SLURM

The full pipeline can be reproduced on an HPC cluster using two scripts:

repro.sh — submits DVC pipeline stages as SLURM jobs with correct dependencies
run.sh — submits isolated Ribasim simulation runs

repro.sh: pipeline reproduction

Submits the DVC pipeline as individual SLURM jobs. Each foreach stage gets its own job, and SLURM --dependency=afterok ensures correct ordering.

./repro.sh

This submits jobs in three waves:

rwzi (shared dependency)
22 parallel jobs: 11 dynamic@*, 10 forcing@*, 1 hws_transient
samenvoegen → koppelen

On completion, job IDs are written to repro_jobs.txt for use with run.sh.

run.sh: isolated simulation runs

Copies a model to runs/<name>/ and submits a SLURM job using the named Ribasim core configured in scripts/install_ribasim_core.py.

As an immediate preparation step, run.sh runs the archive-core Pixi task. This task depends on install-core, then archives bin/ribasim at /p/ribasim-nl/bin/<NAME>. Pixi skips the copy when the input directory and installer configuration are unchanged.

Named archives are immutable: archive-core does not replace an existing /p/ribasim-nl/bin/<NAME> directory. Change NAME when archiving a different core.

The submitted SLURM job runs this archived executable, not bin/ribasim in the repository. The repository and its installed core can therefore change while simulations are queued or running without changing the executable used by those simulations. The exact Ribasim version is written to each SLURM log before the model starts.

./run.sh <name> <model_dir> [--after=<jobid>] [key=value ...]

TOML overrides use dot notation for nested keys.

Examples

Run lhm_coupled after koppelen finishes:

./run.sh lhm_coupled data/Rijkswaterstaat/modellen/lhm_coupled \
  --after=$(grep koppelen repro_jobs.txt | cut -f2)

Run lhm_coupled with a shorter simulation period:

./run.sh lhm_coupled_1yr data/Rijkswaterstaat/modellen/lhm_coupled \
  --after=$(grep koppelen repro_jobs.txt | cut -f2) \
  endtime="2018-01-01 00:00:00"

Compare solver settings:

./run.sh lhm_coupled_tight data/Rijkswaterstaat/modellen/lhm_coupled \
  --after=$(grep koppelen repro_jobs.txt | cut -f2) \
  solver.abstol=1e-6

./run.sh lhm_coupled_qndf data/Rijkswaterstaat/modellen/lhm_coupled \
  --after=$(grep koppelen repro_jobs.txt | cut -f2) \
  solver.algorithm=QNDF solver.abstol=1e-5

All submissions are logged to runs/jobs.txt. Run directories and job logs are gitignored.