rencal.core.data_loader
rencal.core.data_loader defines dataset loading interfaces and local implementations for plant, generation, ERA5, and calibrated weather artifacts.
For conceptual guidance, see:
Example
Section titled “Example”from rencal.core.data_loader import LocalDataLoader
loader = LocalDataLoader()plant = loader.load_plant_data()generation = loader.load_generation_data()era5 = loader.load_era5_data()Constructor
Section titled “Constructor”DataLoader
Section titled “DataLoader”class DataLoader(ABC): DataLoaderAbstract loading contract for plant, generation, and ERA5 datasets.
Returns DataLoader
LocalDataLoader
Section titled “LocalDataLoader”class LocalDataLoader(data_path: str = DOWNLOAD_DATA_DIR): LocalDataLoaderLocal filesystem-backed implementation.
Parameters
Section titled “Parameters”
Base path containingdata_pathstrplant/,generation/,era5/, andcalibrated/artifacts.
Returns LocalDataLoader
Methods
Section titled “Methods”DataLoader.load_plant_data
Section titled “DataLoader.load_plant_data”load_plant_data() -> PlantDatasetModelLoad CfD plant/location/capacity data.
Returns PlantDatasetModel
DataLoader.load_generation_data
Section titled “DataLoader.load_generation_data”load_generation_data() -> GenerationDatasetModelLoad generation/settlement time series.
Returns GenerationDatasetModel
DataLoader.load_era5_data
Section titled “DataLoader.load_era5_data”load_era5_data() -> ERA5DatasetModelLoad ERA5 weather data using standard variables.
Returns ERA5DatasetModel
LocalDataLoader.load_plant_data
Section titled “LocalDataLoader.load_plant_data”load_plant_data(id_column: str = PLANT_ID_COLUMN) -> PlantDatasetModelLoads plant data from the configured local plant file and normalizes the plant ID column.
Parameters
Section titled “Parameters”
Source column name containing plant identifiers.id_columnstr
Returns PlantDatasetModel
Raises
FileNotFoundError: If plant data file is missing.
LocalDataLoader.load_generation_data
Section titled “LocalDataLoader.load_generation_data”load_generation_data(id_column: str = PLANT_ID_COLUMN) -> GenerationDatasetModelLoads generation data from the configured local generation file and normalizes the plant ID column.
Parameters
Section titled “Parameters”
Source column name containing plant identifiers in the generation dataset.id_columnstr
Returns GenerationDatasetModel
Raises
FileNotFoundError: If generation data file is missing.
LocalDataLoader.load_era5_data
Section titled “LocalDataLoader.load_era5_data”load_era5_data() -> ERA5DatasetModelLoads ERA5 NetCDF files from the configured local ERA5 directory, combines them on the time dimension, and filters to standard wind/solar variables.
Returns ERA5DatasetModel
Raises
FileNotFoundError: If no ERA5 files are found.ValueError: If no files can be loaded successfully, no valid time dimension is present, concatenation fails, or no standard ERA5 variables are available.
LocalDataLoader.path_resolver_weather_data
Section titled “LocalDataLoader.path_resolver_weather_data”path_resolver_weather_data(basename: str) -> str | PathResolves calibrated weather artifact basename to local path.
Parameters
Section titled “Parameters”
Name of the calibrated weather artifact (for example, abasenamestr.npyfilename).
Returns str | Path
LocalDataLoader.check_historical_weather
Section titled “LocalDataLoader.check_historical_weather”check_historical_weather() -> dict[str, Any]Returns the manifest metadata for the calibrated historical wind NPY file and validates it against its manifest (SHA-256 hash, shape, and file size).
Returns dict[str, Any] — Manifest dictionary containing artifact metadata (hash, columns, size) plus a basename key for use in downstream executors.
Raises
FileNotFoundError: If the calibrated wind NPY artifact does not exist.ValueError: If checksum, shape, or file size validation fails.
LocalDataLoader.get_prefix_histograms
Section titled “LocalDataLoader.get_prefix_histograms”get_prefix_histograms() -> NDArray | NoneLoads prefix histogram artifact when available.
Loads calibrated prefix histograms for fast distribution lookups (memory-mapped read-only). Returns None when histogram artifacts are not available.
Returns NDArray | None
LocalDataLoader.get_historical_weather
Section titled “LocalDataLoader.get_historical_weather”get_historical_weather() -> NDArray | NoneLoads historical weather NPY artifact when available.
Loads calibrated historical wind load-factor data (memory-mapped read-only). Returns None when the artifact does not exist or cannot be loaded.
Returns NDArray | None
LocalDataLoader.verify_npy_against_manifest
Section titled “LocalDataLoader.verify_npy_against_manifest”@staticmethodverify_npy_against_manifest( npy_path: str | Path, manifest_path: str | Path,) -> dict[str, Any]Validates a NPY file against its JSON manifest by checking SHA-256 hash, column count (second dimension), and file size. Raises ValueError on any mismatch.
Parameters
Section titled “Parameters”-
Path to thenpy_pathstr|Path.npyfile to validate. -
Path to the correspondingmanifest_pathstr|Path.manifest.jsonfile.
Returns dict[str, Any] — The parsed manifest dictionary.
Raises ValueError on SHA-256 hash, column count, or file size mismatch.

