Shortcuts

ReduceOnPlateauMomentum

class mmengine.optim.ReduceOnPlateauMomentum(optimizer, *args, **kwargs)[source]

Reduce the momentum of each parameter group when a metric has stopped improving. Models often benefit from reducing the momentum by a factor of 2-10 once learning stagnates. This scheduler reads a metrics quantity and if no improvement is seen for a patience number of epochs, the momentum is reduced.

Parameters:
  • optimizer (Optimizer or OptimWrapper) – optimizer or Wrapped optimizer.

  • monitor (str) – Key name of the value to monitor in metrics dict.

  • rule (str) – One of less, greater. In less rule, momentum will be reduced when the quantity monitored has stopped decreasing; in greater rule it will be reduced when the quantity monitored has stopped increasing. Defaults to ‘less’. The rule is the renaming of mode in pytorch.

  • factor (float) – Factor by which the momentum will be reduced. new_param = param * factor. Defaults to 0.1.

  • patience (int) – Number of epochs with no improvement after which momentum will be reduced. For example, if patience = 2, then we will ignore the first 2 epochs with no improvement, and will only decrease the momentum after the 3rd epoch if the monitor value still hasn’t improved then. Defaults to 10.

  • threshold (float) – Threshold for measuring the new optimum, to only focus on significant changes. Defaults to 1e-4.

  • threshold_rule (str) – One of rel, abs. In rel rule, dynamic_threshold = best * ( 1 + threshold ) in ‘greater’ rule or best * ( 1 - threshold ) in less rule. In abs rule, dynamic_threshold = best + threshold in greater rule or best - threshold in less rule. Defaults to ‘rel’.

  • cooldown (int) – Number of epochs to wait before resuming normal operation after momentum has been reduced. Defaults to 0.

  • min_value (float or list[float]) – A scalar or a sequence of scalars. A lower bound on the momentum of each parameter group respectively. Defaults to 0. .

  • eps (float) – Minimal decay applied to momentum. If the difference between new and old momentum is smaller than eps, the update is ignored. Defaults to 1e-8.

  • begin (int) – Step at which to start triggering the scheduler to monitor in val within the interval calculated according to epoch of training. Defaults to 0.

  • end (int) – Step at which to stop triggering the scheduler to monitor in val within the interval calculated according to epoch of training. Defaults to INF.

  • last_step (int) – The index of last step. Used for resume without state dict. Defaults to -1.

  • by_epoch (bool) – Whether the scheduled parameters are updated by epochs. Defaults to True.

  • verbose (bool) – Whether to print the value for each update. Defaults to False.

step(metrics=None)[source]

Adjusts the momentum of each parameter group based on the specified schedule.

Parameters:

metrics (Dict[str, float], optional) – Evaluation results of all metrics on validation dataset. The keys are the names of the metrics, and the values are corresponding results. Defaults to None.

Read the Docs v: latest
Versions
latest
stable
v0.10.3
v0.10.2
v0.10.1
v0.10.0
v0.9.1
v0.9.0
v0.8.5
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.4
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.3.0
v0.2.0
Downloads
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.