ml_logger.parser.experiment package

Submodules

ml_logger.parser.experiment.experiment module

Container for the experiment data.

class ml_logger.parser.experiment.experiment.Experiment(configs: List[Dict[str, Any]], metrics: Dict[str, pandas.core.frame.DataFrame], info: Optional[Dict[Any, Any]] = None)

Bases: object

property config

Access the config property.

serialize(dir_path: str) → None

Serialize the experiment data and store at dir_path.

• configs are stored as jsonl (since there are only a few configs per experiment) in a file called config.jsonl.

• metrics are stored in [feather format](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_feather.html).

• info is stored in the gzip format.

class ml_logger.parser.experiment.experiment.ExperimentSequence(experiments: List[ml_logger.parser.experiment.experiment.Experiment])

Bases: collections.UserList

aggregate(aggregate_configs: Callable[[List[List[Dict[str, Any]]]], List[Dict[str, Any]]] = <function return_first_config>, aggregate_metrics: Callable[[List[Dict[str, pandas.core.frame.DataFrame]]], Dict[str, pandas.core.frame.DataFrame]] = <function concat_metrics>, aggregate_infos: Callable[[List[Dict[Any, Any]]], Dict[Any, Any]] = <function return_first_infos>) → ml_logger.parser.experiment.experiment.Experiment

Aggregate a sequence of experiments into a single experiment.

Parameters

• aggregate_configs (Callable[ [List[List[ConfigType]]], List[ConfigType] ], optional) – Function to aggregate the configs. Defaults to return_first_config.

• aggregate_metrics (Callable[ [List[ExperimentMetricType]], ExperimentMetricType ], optional) – Function to aggregate the metrics. Defaults to concat_metrics.

• aggregate_infos (Callable[ [List[ExperimentInfoType]], ExperimentInfoType ], optional) – Function to aggregate the information. Defaults to return_first_infos.

Returns

Aggregated Experiment.

Return type

Experiment

filter(filter_fn: Callable[[ml_logger.parser.experiment.experiment.Experiment], bool]) → ml_logger.parser.experiment.experiment.ExperimentSequence

Filter experiments in the sequence.

Parameters

filter_fn – Function to filter an experiment

Returns

A sequence of experiments for which the filter condition is true

Return type

ExperimentSequence

groupby(group_fn: Callable[[ml_logger.parser.experiment.experiment.Experiment], str]) → Dict[str, ml_logger.parser.experiment.experiment.ExperimentSequence]

Group experiments in the sequence.

Parameters

group_fn – Function to assign a string group id to the experiment

Returns

A dictionary mapping the sring group id to a sequence of experiments

Return type

Dict[str, ExperimentSequence]

ml_logger.parser.experiment.experiment.concat_metrics(metric_list: List[Dict[str, pandas.core.frame.DataFrame]]) → Dict[str, pandas.core.frame.DataFrame]

Concatenate the metrics.

Parameters

metric_list (List[ExperimentMetricType]) –

Returns

ExperimentMetricType

ml_logger.parser.experiment.experiment.deserialize(dir_path: str) → ml_logger.parser.experiment.experiment.Experiment

Deserialize the experiment data stored at dir_path and return an Experiment object.

ml_logger.parser.experiment.experiment.return_first_config(config_lists: List[List[Dict[str, Any]]]) → List[Dict[str, Any]]

Return the first config list, from a list of list of configs, else return empty list.

Parameters

config_lists (List[List[ConfigType]]) –

Returns

List[ConfigType]

ml_logger.parser.experiment.experiment.return_first_infos(info_list: List[Dict[Any, Any]]) → Dict[Any, Any]

Return the first info, from a list of infos. Otherwise return empty info.

Parameters

info_list (List[ExperimentInfoType]) –

Returns

ExperimentInfoType

ml_logger.parser.experiment.parser module

Implementation of Parser to parse experiment from the logs.

class ml_logger.parser.experiment.parser.Parser(parse_config_line: Callable[[str], Optional[Dict[str, Any]]] = <function parse_json_and_match_value>, parse_metric_line: Callable[[str], Optional[Dict[str, Any]]] = <function parse_json_and_match_value>, parse_info_line: Callable[[str], Optional[Dict[str, Any]]] = <function parse_json>)

Bases: ml_logger.parser.base.Parser

Class to parse an experiment from the log dir.

parse(filepath_pattern: Union[str, pathlib.Path]) → ml_logger.parser.experiment.experiment.Experiment

Load one experiment from the log dir.

Parameters

filepath_pattern (Union[str, Path]) – filepath pattern to glob or instance of Path (directory) object.

Returns

Experiment

Module contents

Module to interact with the experiment data.