edflow.util module

Some Utility functions, that make yur life easier but don’t fit in any better catorgory than util.

Summary

Exceptions:

KeyNotFoundError

Classes:

NoModel
PRNGMixin	Adds a prng property which is a numpy RandomState which gets reinitialized whenever the pid changes to avoid synchronized sampling behavior when used in conjunction with multiprocessing.
Printer	For usage with walk: collects strings for printing
TablePrinter	For usage with walk: Collects string to put in a table.

Functions:

cached_function	a very rough cache for function calls.
contains_key	Tests if the path like key can find an object in the nested_thing.
edprint	Prints every leaf variable in nested_thing in the form of a table.
get_leaf_names
get_obj_from_str
get_str_from_obj
get_value_from_key	Get value from collection given key
linear_var	Linear from \((a, \alpha)\) to \((b, \beta)\), i.e.
pop_keypath	Given a nested list or dict structure, pop the desired value at key expanding callable nodes if necessary and expand is True.
pop_value_from_key	Pop item from collection given key
pp2mkdtable	Turns a formatted string into a markdown table.
pprint	Prints nested objects and tries to give relevant information.
pprint_str	Formats nested objects as string and tries to give relevant information.
retrieve	Given a nested list or dict return the desired value at key expanding callable nodes if necessary and expand is True.
set_default	Combines retrieve() and set_value() to create the behaviour of pythons dict.setdefault: If key is found in list_or_dict, return its value, otherwise return default and add it to list_or_dict at key.
set_value	Sets a value in a possibly nested list or dict object.
strenumerate	Works just as enumerate, but the returned index is a string.
update
walk	Walk a nested list and/or dict recursively and call fn on all non list or dict objects.

Reference

edflow.util.get_str_from_obj(obj)

edflow.util.get_obj_from_str(string)

edflow.util.linear_var(step, start, end, start_value, end_value, clip_min=0.0, clip_max=1.0)

Linear from \((a, \alpha)\) to \((b, \beta)\), i.e. \(y = (\beta - \alpha)/(b - a) * (x - a) + \alpha\)

Parameters

• step (int) – \(x\)

• start (float) – \(a\)

• end (float) – \(b\)

• start_value (float) – \(\alpha\)

• end_value (float) – \(\beta\)

• clip_min (float) – Minimal value returned.

• clip_max (float) – Maximum value returned.

Returns

:math:`y`

Return type

float

edflow.util.walk(dict_or_list, fn, inplace=False, pass_key=False, prev_key='', splitval='/', walk_np_arrays=False)

Walk a nested list and/or dict recursively and call fn on all non list or dict objects.

Example

dol = {'a': [1, 2], 'b': {'c': 3, 'd': 4}}

def fn(val):
    return val**2

result = walk(dol, fn)
print(result)  # {'a': [1, 4], 'b': {'c': 9, 'd': 16}}
print(dol)  # {'a': [1, 2], 'b': {'c': 3, 'd': 4}}

result = walk(dol, fn, inplace=True)
print(result)  # {'a': [1, 4], 'b': {'c': 9, 'd': 16}}
print(dol)  # {'a': [1, 4], 'b': {'c': 9, 'd': 16}}

Parameters

• dict_or_list (dict or list) – Possibly nested list or dictionary.

• fn (Callable) – Applied to each leave of the nested list_dict-object.

• inplace (bool) – If False, a new object with the same structure and the results of fn at the leaves is created. If True the leaves are replaced by the results of fn.

• pass_key (bool) – Also passes the key or index of the leave element to fn.

• prev_key (str) – If pass_key == True keys of parent nodes are passed to calls of walk on child nodes to accumulate the keys.

• splitval (str) – String used to join keys if pass_key is True.

• walk_np_arrays (bool) – If True, numpy arrays are intepreted as list, ie not as leaves.

Returns

• The resulting nested list-dict-object with the results of

• fn at its leaves. (dict or list)

exception edflow.util.KeyNotFoundError(cause, keys=None, visited=None)

Bases: Exception

__init__(cause, keys=None, visited=None)

Initialize self. See help(type(self)) for accurate signature.

edflow.util.retrieve(list_or_dict, key, splitval='/', default=None, expand=True, pass_success=False)

Given a nested list or dict return the desired value at key expanding callable nodes if necessary and expand is True. The expansion is done in-place.

Parameters

• list_or_dict (list or dict) – Possibly nested list or dictionary.

• key (str) – key/to/value, path like string describing all keys necessary to consider to get to the desired value. List indices can also be passed here.

• splitval (str) – String that defines the delimiter between keys of the different depth levels in key.

• default (obj) – Value returned if key is not found.

• expand (bool) – Whether to expand callable nodes on the path or not.

Returns

• The desired value or if default is not None and the

• key is not found returns default.

:raises Exception if key not in list_or_dict and default is: :raises None.:

edflow.util.pop_keypath(current_item: Union[callable, list, dict], key: str, splitval: str = '/', default: object = None, expand: bool = True, pass_success: bool = False)

Given a nested list or dict structure, pop the desired value at key expanding callable nodes if necessary and expand is True. The expansion is done in-place.

Parameters

• current_item (list or dict) – Possibly nested list or dictionary.

• key (str) – key/to/value, path like string describing all keys necessary to consider to get to the desired value. List indices can also be passed here.

• splitval (str) – String that defines the delimiter between keys of the different depth levels in key.

• default (obj) – Value returned if key is not found.

• expand (bool) – Whether to expand callable nodes on the path or not.

Returns

• The desired value or if default is not None and the

• key is not found returns default.

:raises Exception if key not in list_or_dict and default is: :raises None.:

edflow.util.get_value_from_key(collection: Union[list, dict], key: str)

Get value from collection given key

edflow.util.pop_value_from_key(collection: Union[list, dict], key: str)

Pop item from collection given key

Parameters

• collection (Union[list, dict]) –

• key –

edflow.util.set_default(list_or_dict, key, default, splitval='/')

Combines retrieve() and set_value() to create the behaviour of pythons dict.setdefault: If key is found in list_or_dict, return its value, otherwise return default and add it to list_or_dict at key.

Parameters

• list_or_dict (list or dict) – Possibly nested list or dictionary. splitval (str): String that defines the delimiter between keys of the different depth levels in key.

• key (str) – key/to/value, path like string describing all keys necessary to consider to get to the desired value. List indices can also be passed here.

• default (object) – Value to be returned if key not in list_or_dict and set to be at key in this case.

• splitval (str) – String that defines the delimiter between keys of the different depth levels in key.

Returns

• The retrieved value or if the key is not found returns

• default.

edflow.util.set_value(list_or_dict, key, val, splitval='/')

Sets a value in a possibly nested list or dict object.

Parameters

• key (str) – key/to/value, path like string describing all keys necessary to consider to get to the desired value. List indices can also be passed here.

• value (object) – Anything you want to put behind key

• list_or_dict (list or dict) – Possibly nested list or dictionary.

• splitval (str) – String that defines the delimiter between keys of the different depth levels in key.

Examples

dol = {"a": [1, 2], "b": {"c": {"d": 1}, "e": 2}}

# Change existing entry
set_value(dol, "a/0", 3)
# {'a': [3, 2], 'b': {'c': {'d': 1}, 'e': 2}}}

set_value(dol, "b/e", 3)
# {"a": [3, 2], "b": {"c": {"d": 1}, "e": 3}}

set_value(dol, "a/1/f", 3)
# {"a": [3, {"f": 3}], "b": {"c": {"d": 1}, "e": 3}}

# Append to list
dol = {"a": [1, 2], "b": {"c": {"d": 1}, "e": 2}}

set_value(dol, "a/2", 3)
# {"a": [1, 2, 3], "b": {"c": {"d": 1}, "e": 2}}

set_value(dol, "a/5", 6)
# {"a": [1, 2, 3, None, None, 6], "b": {"c": {"d": 1}, "e": 2}}

# Add key
dol = {"a": [1, 2], "b": {"c": {"d": 1}, "e": 2}}
set_value(dol, "f", 3)
# {"a": [1, 2], "b": {"c": {"d": 1}, "e": 2}, "f": 3}

set_value(dol, "b/1", 3)
# {"a": [1, 2], "b": {"c": {"d": 1}, "e": 2, 1: 3}, "f": 3}

# Raises Error:
# Appending key to list
# set_value(dol, 'a/g', 3)  # should raise

# Fancy Overwriting
dol = {"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2}

set_value(dol, "e/f", 3)
# {"a": [1, 2], "b": {"c": {"d": 1}}, "e": {"f": 3}}

set_value(dol, "e/f/1/g", 3)
# {"a": [1, 2], "b": {"c": {"d": 1}}, "e": {"f": [None, {"g": 3}]}}

# Toplevel new key
dol = {"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2}
set_value(dol, "h", 4)
# {"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2, "h": 4}

set_value(dol, "i/j/k", 4)
# {"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2, "h": 4, "i": {"j": {"k": 4}}}

set_value(dol, "j/0/k", 4)
# {"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2, "h": 4, "i": {"j": {"k": 4}}, "j": [{"k": 4}], }

# Toplevel is list new key
dol = [{"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2}, 2, 3]

set_value(dol, "0/k", 4)
# [{"a": [1, 2], "b": {"c": {"d": 1}}, "e": 2, "k": 4}, 2, 3]

set_value(dol, "0", 1)
# [1, 2, 3]

edflow.util.contains_key(nested_thing, key, splitval='/', expand=True)

Tests if the path like key can find an object in the nested_thing.

edflow.util.update(to_update, to_update_with, splitval='/', expand=True)

edflow.util.get_leaf_names(nested_thing)

edflow.util.strenumerate(iterable)

Works just as enumerate, but the returned index is a string.

Parameters

iterable (Iterable) – An (guess what) iterable object.

edflow.util.cached_function(fn)

a very rough cache for function calls. Highly experimental. Only active if activated with environment variable.

class edflow.util.PRNGMixin

Bases: object

Adds a prng property which is a numpy RandomState which gets reinitialized whenever the pid changes to avoid synchronized sampling behavior when used in conjunction with multiprocessing.

property prng

class edflow.util.Printer(string_fn)

Bases: object

For usage with walk: collects strings for printing

__init__(string_fn)

Initialize self. See help(type(self)) for accurate signature.

class edflow.util.TablePrinter(string_fn, names=None, jupyter_style=False)

Bases: object

For usage with walk: Collects string to put in a table.

__init__(string_fn, names=None, jupyter_style=False)

Initialize self. See help(type(self)) for accurate signature.

edflow.util.pprint_str(nested_thing, heuristics=None)

Formats nested objects as string and tries to give relevant information.

Parameters

• nested_thing (dict or list) – Some nested object.

• heuristics (Callable) – If given this should produce the string, which is printed as description of a leaf object.

edflow.util.pprint(nested_thing, heuristics=None)

Prints nested objects and tries to give relevant information.

Parameters

• nested_thing (dict or list) – Some nested object.

• heuristics (Callable) – If given this should produce the string, which is printed as description of a leaf object.

edflow.util.pp2mkdtable(nested_thing, jupyter_style=False)

Turns a formatted string into a markdown table.

edflow.util.edprint(nested_thing)

Prints every leaf variable in nested_thing in the form of a table.

Parameters

nested_thing (dict or list) – Some nested object.

class edflow.util.NoModel(config)

Bases: object

__init__(config)

Initialize self. See help(type(self)) for accurate signature.