Skip to content

Data and Simulation Pipeline

This page gives a high-level map of the RenCal workflow from external data sources to simulation outputs.

External APIs
-> DataDownloader (writes local artifacts)
-> DataLoader (reads and normalizes artifacts)
-> Dataset Models (validated data contracts)
-> Calibration (fit technology-specific relationships)
-> Simulation (sample historical weather-conditioned behavior)
-> Forecast-ready streams and metrics

RenCal downloads source data from CDS, LCCC, and Elexon, then stores normalized artifacts under the project data directory.

  • ERA5 weather files (.nc)
  • plant + BMU mappings (.csv)
  • generation time series (.parquet)

See Data Downloader Architecture.

LocalDataLoader reads local artifacts and applies ingestion-time normalization (for example plant ID alignment and ERA5 variable selection).

See Data Loader Architecture.

Loader methods return typed model objects:

These models enforce schema and type checks (via Pydantic + model validators), creating a fail-fast boundary before calibration and simulation.

See Dataset Models.

Calibration modules fit plant/technology relationships between weather drivers and generation behavior (for wind, logistic + Weibull components).

Start here:

Simulation uses calibrated/historical artifacts to sample future periods while preserving seasonal structure through bucketing.

In code terms, this stage combines:

Start with Weather Data Sampling. For bucket design details, see Bucketing Strategy.