MessageHub

class mmengine.logging.MessageHub(name, log_scalars=None, runtime_info=None, resumed_keys=None)

Message hub for component interaction. MessageHub is created and accessed in the same way as ManagerMixin.

MessageHub will record log information and runtime information. The log information refers to the learning rate, loss, etc. of the model during training phase, which will be stored as HistoryBuffer. The runtime information refers to the iter times, meta information of runner etc., which will be overwritten by next update.

参数

• name (str) – Name of message hub used to get corresponding instance globally.

• log_scalars (dict, optional) – Each key-value pair in the dictionary is the name of the log information such as “loss”, “lr”, “metric” and their corresponding values. The type of value must be HistoryBuffer. Defaults to None.

• runtime_info (dict, optional) – Each key-value pair in the dictionary is the name of the runtime information and their corresponding values. Defaults to None.

• resumed_keys (dict, optional) – Each key-value pair in the dictionary decides whether the key in _log_scalars and _runtime_info will be serialized.

备注

Key in _resumed_keys belongs to _log_scalars or _runtime_info. The corresponding value cannot be set repeatedly.

实际案例

>>> # create empty `MessageHub`.
>>> message_hub1 = MessageHub('name')
>>> log_scalars = dict(loss=HistoryBuffer())
>>> runtime_info = dict(task='task')
>>> resumed_keys = dict(loss=True)
>>> # create `MessageHub` from data.
>>> message_hub2 = MessageHub(
>>>     name='name',
>>>     log_scalars=log_scalars,
>>>     runtime_info=runtime_info,
>>>     resumed_keys=resumed_keys)

classmethod get_current_instance()

Get latest created MessageHub instance.

MessageHub can call get_current_instance() before any instance has been created, and return a message hub with the instance name “mmengine”.

返回

Empty MessageHub instance.

返回类型

MessageHub

get_info(key, default=None)

Get runtime information by key. If the key does not exist, this method will return default information.

参数

• key (str) – Key of runtime information.

• default (Any, optional) – The default returned value for the given key.

返回

A copy of corresponding runtime information if the key exists.

返回类型

Any

get_scalar(key)

Get HistoryBuffer instance by key.

备注

Considering the large memory footprint of history buffers in the post-training, get_scalar() will not return a reference of history buffer rather than a copy.

参数

key (str) – Key of HistoryBuffer.

返回

Corresponding HistoryBuffer instance if the key exists.

返回类型

HistoryBuffer

load_state_dict(state_dict)

Loads log scalars, runtime information and resumed keys from state_dict or message_hub.

If state_dict is a dictionary returned by state_dict(), it will only make copies of data which should be resumed from the source message_hub.

If state_dict is a message_hub instance, it will make copies of all data from the source message_hub. We suggest to load data from dict rather than a MessageHub instance.

参数

state_dict (dict or MessageHub) – A dictionary contains key log_scalars runtime_info and resumed_keys, or a MessageHub instance.

返回类型

None

property log_scalars: collections.OrderedDict

Get all HistoryBuffer instances.

备注

Considering the large memory footprint of history buffers in the post-training, get_scalar() will return a reference of history buffer rather than a copy.

返回

All HistoryBuffer instances.

返回类型

OrderedDict

pop_info(key, default=None)

Remove runtime information by key. If the key does not exist, this method will return the default value.

参数

• key (str) – Key of runtime information.

• default (Any, optional) – The default returned value for the given key.

返回

The runtime information if the key exists.

返回类型

Any

property runtime_info: collections.OrderedDict

Get all runtime information.

返回

A copy of all runtime information.

返回类型

OrderedDict

state_dict()

Returns a dictionary containing log scalars, runtime information and resumed keys, which should be resumed.

The returned state_dict can be loaded by load_state_dict().

返回

A dictionary contains log_scalars, runtime_info and resumed_keys.

返回类型

dict

update_info(key, value, resumed=True)

Update runtime information.

The key corresponding runtime information will be overwritten each time calling update_info.

备注

The resumed argument needs to be consistent for the same key.

实际案例

>>> message_hub = MessageHub(name='name')
>>> message_hub.update_info('iter', 100)

参数

• key (str) – Key of runtime information.

• value (Any) – Value of runtime information.

• resumed (bool) – Whether the corresponding HistoryBuffer could be resumed.

返回类型

None

update_info_dict(info_dict, resumed=True)

Update runtime information with dictionary.

The key corresponding runtime information will be overwritten each time calling update_info.

备注

The resumed argument needs to be consistent for the same info_dict.

实际案例

>>> message_hub = MessageHub(name='name')
>>> message_hub.update_info({'iter': 100})

参数

• info_dict (str) – Runtime information dictionary.

• resumed (bool) – Whether the corresponding HistoryBuffer could be resumed.

返回类型

None

update_scalar(key, value, count=1, resumed=True)

Update :attr:_log_scalars.

Update HistoryBuffer in _log_scalars. If corresponding key HistoryBuffer has been created, value and count is the argument of HistoryBuffer.update, Otherwise, update_scalar will create an HistoryBuffer with value and count via the constructor of HistoryBuffer.

实际案例

>>> message_hub = MessageHub(name='name')
>>> # create loss `HistoryBuffer` with value=1, count=1
>>> message_hub.update_scalar('loss', 1)
>>> # update loss `HistoryBuffer` with value
>>> message_hub.update_scalar('loss', 3)
>>> message_hub.update_scalar('loss', 3, resumed=False)
AssertionError: loss used to be true, but got false now. resumed
keys cannot be modified repeatedly'

备注

The resumed argument needs to be consistent for the same key.

参数

• key (str) – Key of HistoryBuffer.

• value (torch.Tensor or np.ndarray or int or float) – Value of log.

• count (torch.Tensor or np.ndarray or int or float) – Accumulation times of log, defaults to 1. count will be used in smooth statistics.

• resumed (str) – Whether the corresponding HistoryBuffer could be resumed. Defaults to True.

返回类型

None

update_scalars(log_dict, resumed=True)

Update _log_scalars with a dict.

update_scalars iterates through each pair of log_dict key-value, and calls update_scalar. If type of value is dict, the value should be dict(value=xxx) or dict(value=xxx, count=xxx). Item in log_dict has the same resume option.

备注

The resumed argument needs to be consistent for the same log_dict.

参数

• log_dict (str) – Used for batch updating _log_scalars.

• resumed (bool) – Whether all HistoryBuffer referred in log_dict should be resumed. Defaults to True.

返回类型

None

实际案例

>>> message_hub = MessageHub.get_instance('mmengine')
>>> log_dict = dict(a=1, b=2, c=3)
>>> message_hub.update_scalars(log_dict)
>>> # The default count of  `a`, `b` and `c` is 1.
>>> log_dict = dict(a=1, b=2, c=dict(value=1, count=2))
>>> message_hub.update_scalars(log_dict)
>>> # The count of `c` is 2.