edflow.custom_logging module¶
Module to handle logging in edflow.
Can be imported by application code to get loggers and find out where outputs should be stored.
Summary¶
Classes:
alias of |
|
A logging handler compatible with tqdm progress bars. |
|
Singleton managing all loggers for a run. |
|
Singleton managing all directories for a run. |
Functions:
Get logger. |
Reference¶
-
class
edflow.custom_logging.
run
[source]¶ Bases:
object
Singleton managing all directories for a run.
Calling the init method below will set up a logging directory structure that should be used for this run. Application code can import this class and use its attributes to figure out where to store their outputs.
Note
This class is intended to provide run information without the need to pass it through. Thus it behaves like a singleton by storing all information on the class object itself and not an instance of the class.
-
exists
¶ True if log structure was initialized.
- Type
bool
-
now
¶ Representing time of initialization.
- Type
str
-
postfix
¶ User specified postfix of run directory or eval directory.
- Type
str
-
name
¶ The name of the current run. Stays consistent on resuming.
- Type
str
-
git_tag
¶ If activated and a git repo was found, this attribute contains the tag name pointing to a commit recording the state of the repository when this run was started.
- Type
str
-
resumed
¶ True if this run was resumed.
- Type
bool
-
code_root
¶ Path where code is copied from.
- Type
str
-
code
¶ Path where code is copied to.
- Type
str
-
root
¶ Path under which all outputs of the run should be stored.
- Type
str
-
train
¶ Path to store train outputs in.
- Type
str
-
eval
¶ Path to eval subfolders.
- Type
str
-
latest_eval
¶ Path to store eval outputs in.
- Type
str
-
configs
¶ Path to store configs in.
- Type
str
-
checkpoints
¶ Path to store checkpoints in.
- Type
str
-
exists
= False
-
classmethod
init
(log_dir=None, run_dir=None, code_root='.', postfix=None, log_level='info', git=False)[source]¶ Initialize logging for this run.
After execution of this method, the log directory structure was created, code was copied and commited if desired, and some basic system information has been logged. Subsequent use of loggers from log.get_logger will result in log files written to the run directory.
- Parameters
log_dir (str) – Create new run directory under this directory.
run_dir (str) – Resume in existing run directory.
code_root (str) – Path to where the code lives. py and yaml files will be copied into run directory.
postfix (str) – Identifier appended to run directory if non-existent else to latest eval directory.
log_level (str) – Default log level for loggers.
git (bool) – If True, put code into tagged commit.
-
-
class
edflow.custom_logging.
TqdmHandler
(stream=None)[source]¶ Bases:
logging.StreamHandler
A logging handler compatible with tqdm progress bars.
-
emit
(record)[source]¶ Emit a record.
If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.
-
-
class
edflow.custom_logging.
log
[source]¶ Bases:
object
Singleton managing all loggers for a run.
Note
This class is intended to provide logging facilities without the need to pass it through. Thus it behaves like a singleton by storing all information on the class object itself and not an instance of the class.
-
target
¶ Current default target to write log file to.
- Type
str
-
level
¶ Current default log level for new loggers.
-
loggers
¶ List of all loggers.
-
target
= 'root'
-
level
= 20
-
loggers
= []
-
classmethod
get_logger
(name, which=None, level=None)[source]¶ Get logger.
If run was initialized, returns a logger which is compatible with tqdm progress bars and logs into a file in the run directory. Otherwise, returns a basic logger.
- Parameters
name (str or object) – Name of the logger. If not a string, the name of the given object class is used.
which (str) – Subdirectory in the project folder where log file is written to.
level (str) – Log level of the logger.
-
-
edflow.custom_logging.
LogSingleton
¶ alias of
edflow.custom_logging.log
-
edflow.custom_logging.
get_logger
(name, which=None, level=None)¶ Get logger.
If run was initialized, returns a logger which is compatible with tqdm progress bars and logs into a file in the run directory. Otherwise, returns a basic logger.
- Parameters
name (str or object) – Name of the logger. If not a string, the name of the given object class is used.
which (str) – Subdirectory in the project folder where log file is written to.
level (str) – Log level of the logger.