DataLoader

class Melodie.data_loader.DataLoader(manager, config: Config, scenario_cls: Type[Scenario], as_sub_worker=False)

Bases: object

DataLoader loads dataframes or matrices.

Simulator/Trainer/Calibrator will have reference to DataLoader to avoid defining tables multiple times.

Parameters:

  • manager – The Simulator/Trainer/Calibrator this dataloader belongs to.

  • config – A Melodie.Config instance, the configuration in current project.

  • scenario_cls – The class of scenario used in this project.

  • as_sub_worker – If True, DataLoader will be disabled to avoid unneed database operations.

register_dataframe(table_name: str, data_frame: pd.DataFrame, data_types: dict = None) → None

Register a pandas dataframe.

TODO! register data frame to database!!!

Parameters:

  • table_name – Name of dataframe

  • data_frame – A pandas dataframe

  • data_types – A dictionary describing data types.

Returns:

None

clear_cache()

Clear all caches under caching directory.

load_dataframe(df_info: DataFrameInfo) → None

Register static table. The static table will be copied into database.

The scenarios/agents parameter tables can also be registered by this method.

Returns:

None

load_matrix(matrix_info: MatrixInfo) → None

Register static matrix.

Returns:

None

dataframe_generator(df_info: DataFrameInfo, rows_in_scenario: int | Callable[[Scenario], int]) → DataFrameGenerator

Create a new generator for dataframes.

Parameters:

  • df_info – Dataframe info indicating the dataframe to be generated.

  • rows_in_scenario – How many rows will be generated for a specific scenario. This argument should be an integer as number of rows for each scenario, or a function with a parameter with type Scenario and return an integer for how many rows to generate for this scenario .

Returns:

A dataframe generator object

generate_scenarios_from_dataframe(df_name: str) → List[Scenario]

Generate scenario objects by the parameter from static table named df_name.

Parameters:

df_name – Name of static table.

Returns:

A list of scenario object.

generate_scenarios(manager_type: str) → List[Scenario]

Generate scenario objects by the parameter from static tables or scenarios_dataframe.

Parameters:

manager_type – The type of scenario manager, a str in “simulator”, “trainer” or “calibrator”.

Returns:

A list of scenarios.