KaimingInit

class mmengine.model.KaimingInit(a=0, mode='fan_out', nonlinearity='relu', distribution='normal', **kwargs)

Initialize module parameters with the values according to the method described in the paper below.

Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015).

Parameters:

• a (int | float) – the negative slope of the rectifier used after this layer (only used with 'leaky_relu'). Defaults to 0.

• mode (str) – either 'fan_in' or 'fan_out'. Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. Defaults to 'fan_out'.

• nonlinearity (str) – the non-linear function (nn.functional name), recommended to use only with 'relu' or 'leaky_relu' . Defaults to ‘relu’.

• bias (int | float) – the value to fill the bias. Defaults to 0.

• bias_prob (float, optional) – the probability for bias initialization. Defaults to None.

• distribution (str) – distribution either be 'normal' or 'uniform'. Defaults to 'normal'.

• layer (str | list[str], optional) – the layer will be initialized. Defaults to None.