edflow.iterators.tf_trainer module

Summary

Classes:

TFBaseTrainer	Same but based on TFHookedModelIterator.
TFFrequencyTrainer
TFListTrainer
TFMultiStageModel
TFMultiStageTrainer	Adds multistage training to Edflow Trainer

Functions:

make_linear_var

Linear from \((a, \alpha)\) to \((b, \beta)\), i.e.

Reference

edflow.iterators.tf_trainer.make_linear_var(step, start, end, start_value, end_value, clip_min=None, clip_max=None, **kwargs)

Linear from \((a, \alpha)\) to \((b, \beta)\), i.e. \(y = (\beta - \alpha)/(b - a) * (x - a) + \alpha\)

Parameters

• step (tf.Tensor) – \(x\)

• start (int) – \(a\)

• end (int) – \(b\)

• start_value (float) – \(\alpha\)

• end_value (float) – \(\beta\)

• clip_min (int) – Minimal value returned.

• clip_max (int) – Maximum value returned.

Returns

:math:`y`

Return type

tf.Tensor

class edflow.iterators.tf_trainer.TFBaseTrainer(config, root, model, **kwargs)

Bases: edflow.iterators.tf_iterator.TFHookedModelIterator

Same but based on TFHookedModelIterator.

__init__(config, root, model, **kwargs)

Constructor.

Parameters

• model (object) – Model class.

• num_epochs (int) – Number of times to iterate over the data.

• hooks (list) – List containing Hook instances.

• hook_freq (int) – Frequency at which hooks are evaluated.

• bar_position (int) – Used by tqdm to place bars at the right position when using multiple Iterators in parallel.

initialize(checkpoint_path=None)

Initialize from scratch or restore and keep restorer around.

step_ops()

Defines ops that are called at each step.

Returns

Return type

The operation run at each step.

make_feeds(batch)

Put global step into batches and add all extra required placeholders from batches.

setup()

Init train_placeholders, log_ops and img_ops which can be added to.

create_train_op()

Default optimizer + optimize each submodule

make_loss_ops()

Return per submodule loss. Can add tensors to log_ops and img_ops

make_run_once_op()

Return op to be run at step zero. Used for custom initialization etc.

get_trainable_variables(submodule)

get_init_variables()

get_restore_variables()

get_checkpoint_variables()

run(fetches, feed_dict)

Runs all fetch ops and stores the results.

Parameters

• fetches (dict) – name: Callable pairs.

• feed_dict (dict) – Passed as kwargs to all fetch ops

Returns

name: results pairs.

Return type

dict

class edflow.iterators.tf_trainer.TFFrequencyTrainer(config, root, model, **kwargs)

Bases: edflow.iterators.tf_trainer.TFBaseTrainer

create_train_op()

Default optimizer + optimize each submodule

run(fetches, feed_dict)

Runs all fetch ops and stores the results.

Parameters

• fetches (dict) – name: Callable pairs.

• feed_dict (dict) – Passed as kwargs to all fetch ops

Returns

name: results pairs.

Return type

dict

class edflow.iterators.tf_trainer.TFListTrainer(config, root, model, **kwargs)

Bases: edflow.iterators.tf_trainer.TFBaseTrainer

create_train_op()

Default optimizer + optimize each submodule

run(fetches, feed_dict)

Runs all fetch ops and stores the results.

Parameters

• fetches (dict) – name: Callable pairs.

• feed_dict (dict) – Passed as kwargs to all fetch ops

Returns

name: results pairs.

Return type

dict

get_learning_rate_multiplier(i)

class edflow.iterators.tf_trainer.TFMultiStageTrainer(config, root, model, **kwargs)

Bases: edflow.iterators.tf_trainer.TFBaseTrainer

Adds multistage training to Edflow Trainer

Stages are defined through the config. For example

stages:

1:

name: pretrain end: 10 losses: []

2:

name: retrain end: 30 losses: [“model”]

3:

name: train losses: [“model”]

The stages are sorted by their key. It is recommended to keep the simple numeric ordering. In each stage, a set of losses can be specified through the losses : [ “loss1”, “loss2”, …] syntax. The duration of each stage is given by the end : num_steps value. Note that the end of a stage is determined in the order of the stages. A later stage has to have a higher end value then the previous one.

The model has to implement the edflowiterators.tf_trainer.TFMultiStageModel interface. Look at the multistage_trainer example.

__init__(config, root, model, **kwargs)

Constructor.

Parameters

• model (object) – Model class.

• num_epochs (int) – Number of times to iterate over the data.

• hooks (list) – List containing Hook instances.

• hook_freq (int) – Frequency at which hooks are evaluated.

• bar_position (int) – Used by tqdm to place bars at the right position when using multiple Iterators in parallel.

create_train_op()

Default optimizer + optimize each submodule

run(fetches, feed_dict)

Runs all fetch ops and stores the results.

Parameters

• fetches (dict) – name: Callable pairs.

• feed_dict (dict) – Passed as kwargs to all fetch ops

Returns

name: results pairs.

Return type

dict

get_current_train_op(current_stage)

determine_current_stage()

class edflow.iterators.tf_trainer.TFMultiStageModel

Bases: object

__init__()

Initialize self. See help(type(self)) for accurate signature.

property stage_update_op

property stage_placeholder

property stage