rencal.core.data_downloader
rencal.core.data_downloader defines downloaders for CfD register/mapping, generation data, and ERA5 weather data, plus a coordinating download manager.
Implementation Overview
Section titled “Implementation Overview”The data downloaders handle:
- Directory setup for persisted data downloads
- ERA5 retrieval and metadata verification
- CfD register and BMU mapping enrichment
- Generation data aggregation to hourly UTC
- A unified
DownloadManagerAPI for running all steps
Prerequisites
Section titled “Prerequisites”- Copernicus CDS account and API key for ERA5 downloads
- Network access to CDS, LCCC, and Elexon endpoints
- Python dependencies:
cdsapi,xarray,pandas,requests,numpy,tqdm,certifi - Sufficient disk space for multi-year ERA5 NetCDF files
CDS API Key Usage
Section titled “CDS API Key Usage”ERA5 downloads require a CDS API key. You can provide it in either of the following ways:
- Environment variable: Set
CDS_API_KEYin your environment, and optionallyCDS_API_URLto override the default. - Constructor argument: Pass
api_key(and optionallyapi_url) directly toERA5DataDownloaderorDownloadManager.
Example Usage
Section titled “Example Usage”import osfrom rencal.core.data_downloader import DownloadManager
os.environ["CDS_API_KEY"] = "uid:token"
downloader = DownloadManager()downloader.download_era5()Constructor
Section titled “Constructor”DataDownloader
Section titled “DataDownloader”class DataDownloader(ABC): DataDownloaderAbstract base downloader with shared output directory and logger behavior.
Returns DataDownloader
ERA5DataDownloader
Section titled “ERA5DataDownloader”class ERA5DataDownloader(api_key: str | None = None, api_url: str | None = None): ERA5DataDownloaderDownloader for ERA5 data from the Copernicus CDS API.
Parameters
Section titled “Parameters”-
Copernicus CDS API key. If not provided, the downloader readsapi_keystr|NoneCDS_API_KEY. -
Optional CDS API URL override.api_urlstr|None
Returns ERA5DataDownloader
CfDDataDownloader
Section titled “CfDDataDownloader”class CfDDataDownloader(): CfDDataDownloaderDownloader for CfD register and BMU mapping datasets.
Parameters
Section titled “Parameters”This constructor does not take any parameters.
Returns CfDDataDownloader
GenerationDataDownloader
Section titled “GenerationDataDownloader”class GenerationDataDownloader(): GenerationDataDownloaderDownloader for Elexon settled generation and UTC aggregation flow.
Parameters
Section titled “Parameters”This constructor does not take any parameters.
Returns GenerationDataDownloader
DownloadManager
Section titled “DownloadManager”class DownloadManager(cds_api_key: str | None = None, cds_api_url: str | None = None): DownloadManagerFacade class orchestrating all download flows.
Parameters
Section titled “Parameters”-
Optional CDS API key forwarded tocds_api_keystr|NoneERA5DataDownloader. -
Optional CDS API URL override forwarded tocds_api_urlstr|NoneERA5DataDownloader.
Returns DownloadManager
Methods
Section titled “Methods”DataDownloader.download
Section titled “DataDownloader.download”download(*args: Any, **kwargs: Any) -> NoneAbstract method implemented by concrete downloaders.
Returns None
ERA5DataDownloader.download
Section titled “ERA5DataDownloader.download”download() -> NoneDownloads ERA5 yearly NetCDF artifacts for the calibration horizon and verifies metadata.
Returns None
CfDDataDownloader.download
Section titled “CfDDataDownloader.download”download() -> NoneDownloads CfD register and BMU mapping, merges, and writes output CSV.
Returns None
GenerationDataDownloader.download
Section titled “GenerationDataDownloader.download”download() -> NoneDownloads settled generation data and writes hourly UTC aggregated parquet output.
Returns None
DownloadManager.download_cfd
Section titled “DownloadManager.download_cfd”download_cfd() -> NoneRuns CfD/BMU downloader.
Returns None
DownloadManager.download_generation_data
Section titled “DownloadManager.download_generation_data”download_generation_data() -> NoneRuns generation downloader.
Returns None
DownloadManager.download_era5
Section titled “DownloadManager.download_era5”download_era5() -> NoneRuns ERA5 downloader.
Returns None
DownloadManager.download_all
Section titled “DownloadManager.download_all”download_all() -> NoneRuns all download flows in sequence.
Returns None
Properties
Section titled “Properties”DataDownloader.output_dir
Section titled “DataDownloader.output_dir”Base output directory for downloaded files. Concrete downloaders write to subfolders such as era5/, plant/, and generation/.
Type pathlib.Path
ERA5DataDownloader.client
Section titled “ERA5DataDownloader.client”Lazy-initialized CDS API client.
Type cdsapi.Client

