Exceptions

SMDebug is designed to be aware of that tensors required to evaluate a rule may not be available at every step. Hence, it raises a few exceptions which allow us to control what happens when a tensor is missing. These are available in the smdebug.exceptions module. You can import them as follows:

from smdebug.exceptions import *

The following functions are the exceptions (along with others) and their meanings.

exception smdebug.exceptions.InvalidCollectionConfiguration(c_name)

Bases: Exception

exception smdebug.exceptions.StepNotYetAvailable(step, mode)

Bases: Exception

This means that the step has not yet been seen from the training job. It may be available in the future if the training is still going on. We automatically load new data as and when it becomes available. This step may either become available in the future, or the exception might change to StepUnavailable.

exception smdebug.exceptions.MissingCollectionFiles

Bases: Exception

This is raised when no data was saved by the training job. Check that the Hook was configured correctly before starting the training job.

exception smdebug.exceptions.IndexReaderException(message)

Bases: Exception

exception smdebug.exceptions.StepUnavailable(step, mode)

Bases: Exception

This means that the step was not saved from the training job. No tensor will be available for this step.

exception smdebug.exceptions.TensorUnavailableForStep(tname, step, mode=<ModeKeys.GLOBAL: 4>, has_reductions=False)

Bases: Exception

This is raised when the tensor requested is not available for the step. It may have been or will be saved for a different step number. You can check which steps tensor is saved for by trial.tensor('tname').steps() api. Note that this exception implies that the requested tensor will never become available for this step in the future.

exception smdebug.exceptions.ShapeUnavailableForStep(tname, step, mode=<ModeKeys.GLOBAL: 4>)

Bases: Exception

exception smdebug.exceptions.TensorUnavailable(tname)

Bases: Exception

This means that this tensor has not been saved from the training job. Note that if you have a SaveConfig which saves a certain tensor only after the time you queried for the tensor, you might get a TensorUnavailable exception even if the tensor may become available later for some step.

exception smdebug.exceptions.InvalidWorker(worker)

Bases: Exception

exception smdebug.exceptions.NoMoreProfilerData(timestamp)

Bases: Exception

This will be raised when the training ends. Once you see this, you will know that there will be no more steps and no more tensors saved.

exception smdebug.exceptions.NoMoreData(step, mode, last_step)

Bases: Exception

exception smdebug.exceptions.RuleEvaluationConditionMet(rule_name, step, end_of_rule=False)

Bases: Exception

This is raised when the rule invocation returns True for some step.

exception smdebug.exceptions.InsufficientInformationForRuleInvocation(rule_name, message)

Bases: Exception