Shortcuts

KaimingInit

class mmengine.model.KaimingInit(a=0, mode='fan_out', nonlinearity='relu', distribution='normal', **kwargs)[source]

Initialize module parameters with the values according to the method described in the paper below.

Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015).

Parameters
  • a (int | float) – the negative slope of the rectifier used after this layer (only used with 'leaky_relu'). Defaults to 0.

  • mode (str) – either 'fan_in' or 'fan_out'. Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. Defaults to 'fan_out'.

  • nonlinearity (str) – the non-linear function (nn.functional name), recommended to use only with 'relu' or 'leaky_relu' . Defaults to ‘relu’.

  • bias (int | float) – the value to fill the bias. Defaults to 0.

  • bias_prob (float, optional) – the probability for bias initialization. Defaults to None.

  • distribution (str) – distribution either be 'normal' or 'uniform'. Defaults to 'normal'.

  • layer (str | list[str], optional) – the layer will be initialized. Defaults to None.

Read the Docs v: v0.8.3
Versions
latest
stable
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.4
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.3.0
v0.2.0
Downloads
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.