Skip to content

rencal.core.data_downloader

rencal.core.data_downloader defines downloaders for CfD register/mapping, generation data, and ERA5 weather data, plus a coordinating download manager.

The data downloaders handle:

  • Directory setup for persisted data downloads
  • ERA5 retrieval and metadata verification
  • CfD register and BMU mapping enrichment
  • Generation data aggregation to hourly UTC
  • A unified DownloadManager API for running all steps
  • Copernicus CDS account and API key for ERA5 downloads
  • Network access to CDS, LCCC, and Elexon endpoints
  • Python dependencies: cdsapi, xarray, pandas, requests, numpy, tqdm, certifi
  • Sufficient disk space for multi-year ERA5 NetCDF files

ERA5 downloads require a CDS API key. You can provide it in either of the following ways:

  • Environment variable: Set CDS_API_KEY in your environment, and optionally CDS_API_URL to override the default.
  • Constructor argument: Pass api_key (and optionally api_url) directly to ERA5DataDownloader or DownloadManager.
import os
from rencal.core.data_downloader import DownloadManager
os.environ["CDS_API_KEY"] = "uid:token"
downloader = DownloadManager()
downloader.download_era5()
class DataDownloader(ABC): DataDownloader

Abstract base downloader with shared output directory and logger behavior.

Returns DataDownloader

class ERA5DataDownloader(api_key: str | None = None, api_url: str | None = None): ERA5DataDownloader

Downloader for ERA5 data from the Copernicus CDS API.

  • api_key str | None

    Copernicus CDS API key. If not provided, the downloader reads CDS_API_KEY.

  • api_url str | None

    Optional CDS API URL override.

Returns ERA5DataDownloader

class CfDDataDownloader(): CfDDataDownloader

Downloader for CfD register and BMU mapping datasets.

This constructor does not take any parameters.

Returns CfDDataDownloader

class GenerationDataDownloader(): GenerationDataDownloader

Downloader for Elexon settled generation and UTC aggregation flow.

This constructor does not take any parameters.

Returns GenerationDataDownloader

class DownloadManager(cds_api_key: str | None = None, cds_api_url: str | None = None): DownloadManager

Facade class orchestrating all download flows.

Returns DownloadManager

download(*args: Any, **kwargs: Any) -> None

Abstract method implemented by concrete downloaders.

Returns None

download() -> None

Downloads ERA5 yearly NetCDF artifacts for the calibration horizon and verifies metadata.

Returns None

download() -> None

Downloads CfD register and BMU mapping, merges, and writes output CSV.

Returns None

download() -> None

Downloads settled generation data and writes hourly UTC aggregated parquet output.

Returns None

download_cfd() -> None

Runs CfD/BMU downloader.

Returns None

download_generation_data() -> None

Runs generation downloader.

Returns None

download_era5() -> None

Runs ERA5 downloader.

Returns None

download_all() -> None

Runs all download flows in sequence.

Returns None

Base output directory for downloaded files. Concrete downloaders write to subfolders such as era5/, plant/, and generation/.

Type pathlib.Path

Lazy-initialized CDS API client.

Type cdsapi.Client