edflow.custom_logging module¶

Module to handle logging in edflow.

Can be imported by application code to get loggers and find out where outputs should be stored.

Summary¶

Classes:

`ColorLineFormatter`
`LogSingleton`	alias of `edflow.custom_logging.log`
`TqdmHandler`	A logging handler compatible with tqdm progress bars.
`log`	Singleton managing all loggers for a run.
`run`	Singleton managing all directories for a run.

Functions:

get_logger

Get logger.

Reference¶

class edflow.custom_logging.run[source]¶

Bases: object

Singleton managing all directories for a run.

Calling the init method below will set up a logging directory structure that should be used for this run. Application code can import this class and use its attributes to figure out where to store their outputs.

Note

This class is intended to provide run information without the need to pass it through. Thus it behaves like a singleton by storing all information on the class object itself and not an instance of the class.

now¶

Representing time of initialization.

Type: str

postfix¶

User specified postfix of run directory or eval directory.

Type: str

name¶

The name of the current run. Stays consistent on resuming.

Type: str

git_tag¶

If activated and a git repo was found, this attribute contains the tag name pointing to a commit recording the state of the repository when this run was started.

Type: str

resumed¶

True if this run was resumed.

Type: bool

code_root¶

Path where code is copied from.

Type: str

code¶

Path where code is copied to.

Type: str

root¶

Path under which all outputs of the run should be stored.

Type: str

train¶

Path to store train outputs in.

Type: str

eval¶

Path to eval subfolders.

Type: str

latest_eval¶

Path to store eval outputs in.

Type: str

configs¶

Path to store configs in.

Type: str

checkpoints¶

Path to store checkpoints in.

Type: str

exists = False¶: True if log structure was initialized.

classmethod init(log_dir=None, run_dir=None, code_root='.', postfix=None, log_level='info', git=False)[source]¶

Initialize logging for this run.

After execution of this method, the log directory structure was created, code was copied and commited if desired, and some basic system information has been logged. Subsequent use of loggers from log.get_logger will result in log files written to the run directory.

Parameters

log_dir (str) – Create new run directory under this directory.
run_dir (str) – Resume in existing run directory.
code_root (str) – Path to where the code lives. py and yaml files will be copied into run directory.
postfix (str) – Identifier appended to run directory if non-existent else to latest eval directory.
log_level (str) – Default log level for loggers.
git (bool) – If True, put code into tagged commit.

class edflow.custom_logging.TqdmHandler(stream=None)[source]¶

Bases: logging.StreamHandler

A logging handler compatible with tqdm progress bars.

emit(record)[source]¶

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

class edflow.custom_logging.log[source]¶

Bases: object

Singleton managing all loggers for a run.

Note

This class is intended to provide logging facilities without the need to pass it through. Thus it behaves like a singleton by storing all information on the class object itself and not an instance of the class.

target = 'root'¶: Current default target to write log file to.

level = 20¶: Current default log level for new loggers.

loggers = []¶: List of all loggers.

classmethod set_log_target(which)[source]¶: Set default target where log file is written to.

classmethod get_logger(name, which=None, level=None)[source]¶

Get logger.

If run was initialized, returns a logger which is compatible with tqdm progress bars and logs into a file in the run directory. Otherwise, returns a basic logger.

Parameters

name (str or object) – Name of the logger. If not a string, the name of the given object class is used.
which (str) – Subdirectory in the project folder where log file is written to.
level (str) – Log level of the logger.

classmethod set_log_level(level)[source]¶: Set log level of all existing and default log level of all future loggers.

class edflow.custom_logging.ColorLineFormatter(use_color, *args, **kwargs)[source]¶

Bases: logging.Formatter

__init__(use_color, *args, **kwargs)[source]¶

Initialize the formatter with specified format strings.

Initialize the formatter either with the specified format string, or a default as described above. Allow for specialized date formatting with the optional datefmt argument. If datefmt is omitted, you get an ISO8601-like (or RFC 3339-like) format.

Use a style parameter of ‘%’, ‘{‘ or ‘$’ to specify that you want to use one of %-formatting, str.format() ({}) formatting or string.Template formatting in your format string.

Changed in version 3.2: Added the style parameter.

format(record)[source]¶

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

edflow.custom_logging.LogSingleton¶: alias of edflow.custom_logging.log

edflow.custom_logging.get_logger(name, which=None, level=None)¶

Get logger.

If run was initialized, returns a logger which is compatible with tqdm progress bars and logs into a file in the run directory. Otherwise, returns a basic logger.

Parameters

name (str or object) – Name of the logger. If not a string, the name of the given object class is used.
which (str) – Subdirectory in the project folder where log file is written to.
level (str) – Log level of the logger.