Skip to main content

Cloud storage

The project data and models are versioned with DVC on MinIO. The project used to store them on The Good Cloud, and the class ribasim_nl.CloudStorage still has that name, see the API reference.

Paths

Most code uses CloudStorage to find local paths under RIBASIM_NL_DATA_DIR:

from ribasim_nl import CloudStorage

cloud = CloudStorage()

verwerkt = cloud.joinpath("Delfland", "verwerkt")
model_dir = cloud.model_dir("Delfland", "forcing")

The Good Cloud

Warning

The rest of this page is about the legacy storage. It is not the main storage method for current workflows: use MinIO through DVC instead, with the credentials of the developer setup.

The Good Cloud needs the password RIBASIM_NL_CLOUD_PASS in .env, which is only shared among project collaborators, since not all input data is public.

The pipeline still has one use for it: if data/Basisgegevens/GKW is not local, which is not versioned with DVC, the parameterized stages of 9 water authorities download it from there, see ribasim_nl.gkw.

# what is available
cloud.source_data
cloud.water_authorities

# download national data
cloud.download_basisgegevens(bronnen=["PBL_watertypenkaart"])

# download the data of a water authority: delivered, processed, or both
cloud.download_aangeleverd("Rijkswaterstaat")
cloud.download_verwerkt("Rijkswaterstaat")
cloud.download_all("Rijkswaterstaat")

# upload processed data
cloud.upload_verwerkt("Rijkswaterstaat", overwrite=True)

# see which models have been uploaded, and upload a new version of the `ijsselmeer` model
cloud.uploaded_models("Rijkswaterstaat")
cloud.upload_model("Rijkswaterstaat", model="ijsselmeer")