Testing

Run the repository checks before submitting changes:

pixi run check

This also runs on CI.

Model input validation

Model.validate_ribasim_nl() runs extensible checks on model input and supplements Ribasim schema validation. Add project-wide input rules to ribasim_nl.validation.validate_model.

Result regression testing

The dry and wet steady-state scenarios guard against a Basin losing its ability to maintain its water level. During the existing bergend_* and forcing_* DVC stages, and in repro.sh, the dry and wet models are run and checked. A Basin is steady-state when its final water level differs by at most 0.01 m from the mean of the final 24 hours and its storage remains positive throughout the simulation.

Known level-instability exceptions are kept as explicit scenario-specific Basin node IDs in ribasim_nl.steady_state_regression. Exceptions do not bypass the positive-storage check, and an exception for a missing Basin ID fails the run.

Scenario results are available for inspection in each model’s results directory, including output_controle.gpkg and output_controle.qlr.

Coupled model result checks

The validate_lhm_coupled DVC stage checks the results of run_lhm_coupled for large regressions, with ribasim_nl.result_validation. It takes a few seconds, so after pulling the results it can be run with:

pixi run dvc repro --single-item validate_lhm_coupled

It summarizes the run in metrics, and checks them against the loose limits in LIMITS:

  • solver: the number of right-hand side calls, the mean timestep and the 5th percentile of the mean timestep per day, and the fraction of rejected timesteps.
  • observations: the number of Observation nodes that are compared with the simulated flow rate of the node they observe, and their median Nash-Sutcliffe (NSE) and Kling-Gupta efficiencies (KGE).
  • discharge_kge: the KGE of the discharge at key points of the main water system, such as the Rhine branches and the sluices of the Afsluitdijk.
  • level_p01 and level_p99: the 1st and 99th percentile of the levels of key Basins, such as the IJsselmeer and the Noordzeekanaal.

The metrics are written to data/Rijkswaterstaat/metrics/lhm_coupled.json, which is tracked by git, so a pull request shows how they changed, and the comparison per Observation node to lhm_coupled_observations.csv next to it. Compare the metrics with those of main with:

pixi run dvc metrics diff main

The metrics also include the computation time in hours, but it is not checked, since it depends on the machine and its load. The other solver metrics hardly do, so they catch slowdowns more reliably. If a change of the results is intended, change LIMITS in the same pull request and explain why.

Completed-model result checks

Control.run() exports selected result metrics to results/output_controle.gpkg. It defaults to DYNAMIC_FORCING_METRICS; use STATIC_FORCING_METRICS or AFVOER_METRICS for the other established workflows, or pass a custom metric set. Call Control.validate_result_conditions() after export to enforce conditions. It logs every violation with its Basin node ID before raising one error, so all failures can be addressed together.

New completed-model checks belong on Control and can be called after future dynamic model exports without changing the dry/wet scenario runner.

Allowing a known exception

Only after reviewing an intentional model change, add the Basin node ID to ALLOWED_NON_STEADY_STATE_NODE_IDS for its authority and scenario. Remove an exception once the Basin becomes steady-state again.