xplogger.experiment_manager.record package

Submodules

xplogger.experiment_manager.record.base module

Record class.

class xplogger.experiment_manager.record.base.Record(record: dict[str, Any])

Bases: collections.UserDict

xplogger.experiment_manager.record.base.get_experiment_params(record: Record, viz_params: list[str]) → dict[str, Any]

Get experiment params.

xplogger.experiment_manager.record.base.get_nested_item(data: Record, keys: list[Any]) → Any

Get item from a nested dict / record.

Parameters

• data (Record) –

• keys (list[Any]) –

Returns

Return type

Any

xplogger.experiment_manager.record.base.load_all_experiments_from_dir(load_experiment_from_dir: Any, base_dir: str) → xplogger.parser.experiment.experiment.ExperimentSequence

Load all experiments in a directory.

xplogger.experiment_manager.record.base.load_experiment(record: xplogger.experiment_manager.record.base.Record, load_experiment_from_dir: Callable[[str], xplogger.parser.experiment.experiment.Experiment]) → xplogger.parser.experiment.experiment.Experiment

Load experiment given a record.

xplogger.experiment_manager.record.mongo module

Mongo Record class.

class xplogger.experiment_manager.record.mongo.Record(record: dict[str, Any])

Bases: xplogger.experiment_manager.record.base.Record

Wrappper over mongodb records.

property id

Get record id.

xplogger.experiment_manager.record.mongo.make_record(config: Dict[str, Any]) → xplogger.experiment_manager.record.mongo.Record

Make a mongo record.

xplogger.experiment_manager.record.omegaconf module

OC Record.

xplogger.experiment_manager.record.omegaconf.make_record(mongo_record: xplogger.experiment_manager.record.mongo.Record) → omegaconf.dictconfig.DictConfig

Make record.

xplogger.experiment_manager.record.record_list module

Record class.

class xplogger.experiment_manager.record.record_list.RecordList(records: list[base_record.Record])

Bases: collections.UserList

add_slurm_field(collection: pymongo.collection.Collection) → None

Add slurm field to records (in the db).

Parameters

collection (pymongo.collection.Collection) –

delete(collection: pymongo.collection.Collection, delete_from_filesystem: bool = False) → None

Delete jobs from the db and filesystem (optionally).

Parameters

• collection (pymongo.collection.Collection) –

• delete_from_filesystem (bool, optional) – Should delete the job from the filesystem. Defaults to False.

get_groups_and_hyperparams(viz_params: list[str]) → tuple[dict[Any, RecordList], dict[str, set[ValueType]]]

Group experiments.

get_unique(key_func: Callable[[xplogger.experiment_manager.record.base.Record], str]) → xplogger.experiment_manager.record.record_list.RecordList

Get unique records from the current record list.

Parameters

key_func (Callable[[base_record.Record], str]) – This function computes the key (or hash) unsed to identify a record.

Returns

List of unique records.

Return type

RecordList

get_unique_issues() → collections.Counter[str]

Get unique issues from the record list.

get_viz_params() → set[str]

Get params for vizualization.

load_experiments(load_experiment_from_dir: Callable[[str], xplogger.parser.experiment.experiment.Experiment]) → xplogger.parser.experiment.experiment.ExperimentSequence

Load experiments.

make_experiment_sequence_dict_groups_and_hyperparams(viz_params: list[str], load_experiment_from_dir: LoadExperientFromDirType) → tuple[ExperimentSequenceDict, dict[Any, RecordList], dict[str, set[ValueType]]]

Make experiment groups.

make_oc_records() → xplogger.experiment_manager.record.record_list.RecordList

Make OC records.

map_to_slurm_id() → dict[str, RecordList]

Map the record list to a list of slurm ids.

Returns

dictionary where the key is the slurm id

and value is the list of records. We return a list of records as sometimes the records are duplicated.

Return type

dict[str, RecordList]

mark_analyzed(collection: pymongo.collection.Collection) → None

Mark records as analyzed (in the db).

Parameters

collection (pymongo.collection.Collection) –

ray_make_experiment_sequence_dict_groups_and_hyperparams(viz_params: list[str], load_experiment_from_dir: LoadExperientFromDirType) → tuple[ExperimentSequenceDict, dict[Any, RecordList], dict[str, set[ValueType]]]

Make experiment groups.

ray_make_oc_records() → xplogger.experiment_manager.record.record_list.RecordList

Make OC records using ray.

update_status(collection: pymongo.collection.Collection, new_status: str) → None

Update the status of the records(in the db).

Parameters

collection (pymongo.collection.Collection) –

Module contents