:py:mod:`orcanet.logging` ========================= .. py:module:: orcanet.logging .. autoapi-nested-parse:: Scripts for writing the logfiles. .. !! processed by numpydoc !! Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: orcanet.logging.TrainfileLogger orcanet.logging.SummaryLogger orcanet.logging.BatchLogger Functions ~~~~~~~~~ .. autoapisummary:: orcanet.logging.gen_line_str orcanet.logging.gen_line_cells orcanet.logging.merge_arrays orcanet.logging.log_start_training orcanet.logging.log_start_validation .. py:class:: TrainfileLogger(log_file, column_names) .. py:method:: level_file() Make file with only the head lines. Existing file will be overwritten. .. !! processed by numpydoc !! .. py:method:: write_line(values) Write a line with data to the file. :Parameters: **values** : List The data, in the same order as the column names. .. !! processed by numpydoc !! .. py:function:: gen_line_str(data, widths=None, seperator=' | ', float_precision=4, minimum_cell_width=9) Generate a line in nice human readable format, consisting of multiple spaced and seperated cells. :Parameters: **data** : tuple Strings or floats of what is in each cell. It must be in the same order and have the same length as the column names. **widths** : List or None Optional: The width of every cell. If None, will set it automatically, depending on the data. If widths is given, but what is given in data is wider than the width, the cell will expand without notice. Must have the same length as the column names. **seperator** : str String that seperates two adjacent cells. **float_precision** : int Precision to which floats are rounded if they appear in data. The length of the resulting numbercan be up to 5 characters longer than this value (due to . and e-09) **minimum_cell_width** : int Minimum width of the cells in characters. :Returns: **line** : str The line. **new_widths** : List The widths of the cells. .. !! processed by numpydoc !! .. py:function:: gen_line_cells(data, widths=None, float_precision=4, minimum_cell_width=9) Generate the content of the cells for a line in the summary file. See gen_line_str (above) for doc. :Returns: **cells** : List .. **new_widths** : List .. .. !! processed by numpydoc !! .. py:class:: SummaryLogger(orga, model) For writing the summary logfile made during training. :Parameters: **orga** : orcanet.core.Organizer Contains all the configurable options in the OrcaNet scripts. **model** : ks.model.Model or None Keras model containing the metrics to plot. .. !! processed by numpydoc !! .. py:method:: write_line(epoch_float, lr, history_train=None, history_val=None) Write a line to the summary.txt file in the trained model folder. Will update an existing line if possible. :Parameters: **epoch_float** : float The current epoch and fileno as a float. **lr** : float/str The current learning rate of the model. **history_train** : dict Dict containing the history of the training, averaged over files. Keys: Metric names, e.g. "loss", "accuracy", ... Values: Value of the metric during validation as a float. **history_val** : dict or None Dict of validation losses for all the metrics, averaged over all validation files. Keys: Metric names, e.g. "loss", "accuracy", ... Values: Value of the metric during validation as a float. .. rubric:: Notes In tf 2.2, model.metrics_names is only filled after the model has been used on data, i.e. only after that point this line can be run. Otherwise, _get_column_names will throw a NameError. .. !! processed by numpydoc !! .. py:function:: merge_arrays(base, supp, exclude=None) Fill nans in a list with values from another list. :Parameters: **base** : List .. **supp** : List .. **exclude** : List or int Which indices to ignore. :Returns: np.array .. .. !! processed by numpydoc !! .. py:class:: BatchLogger(orga, epoch, reset_metrics=True) Write logfiles during training. Averages the losses of the model over some number of batches, and then writes that in a line in the logfile. The Batch_float entry in the logfiles gives the absolute position of the batch in the epoch (i.e. taking all files into account). This class is intended to be used only for one epoch = one file. :Parameters: **orga** : orcanet.core.Organizer Contains all the configurable options in the OrcaNet scripts. **epoch** : tuple Epoch and file number. **reset_metrics** : bool Reset internal state of metric after eveery batch? .. !! processed by numpydoc !! .. py:method:: on_epoch_begin(epoch, logs=None) Called at the start of an epoch. Subclasses should override for any actions to run. This function should only be called during TRAIN mode. Args: epoch: Integer, index of epoch. logs: Dict. Currently no data is passed to this argument for this method but that may change in the future. .. !! processed by numpydoc !! .. py:method:: initialize_epoch() Start a new logfile and prepare the logger. .. !! processed by numpydoc !! .. py:method:: on_batch_end(batch, logs=None) A backwards compatibility alias for `on_train_batch_end`. .. !! processed by numpydoc !! .. py:method:: on_epoch_end(batch, logs=None) if self._stored_metrics: # write stats of remaining batches self._write_line() .. !! processed by numpydoc !! .. py:function:: log_start_training(orga) When a training is started for the first time, this logs all the input parameters to the log.txt file. :Parameters: **orga** : orcanet.core.Organizer Contains all the configurable options in the OrcaNet scripts. .. !! processed by numpydoc !! .. py:function:: log_start_validation(orga) Log filenames used for validation. .. !! processed by numpydoc !!