mmengine.dist

dist

gather	Gather data from the whole group to dst process.
gather_object	Gathers picklable objects from the whole group in a single process.
all_gather	Gather data from the whole group in a list.
all_gather_object	Gather picklable objects from the whole group into a list.
all_reduce	Reduces the tensor data across all machines in such a way that all get the final result.
all_reduce_dict	Reduces the dict across all machines in such a way that all get the final result.
all_reduce_params	All-reduce parameters.
broadcast	Broadcast the data from src process to the whole group.
sync_random_seed	Synchronize a random seed to all processes.
broadcast_object_list	Broadcasts picklable objects in object_list to the whole group.
collect_results	Collected results in distributed environments.
collect_results_cpu	Collect results under cpu mode.
collect_results_gpu	Collect results under gpu mode.

utils

get_dist_info	Get distributed information of the given process group.
init_dist	Initialize distributed environment.
init_local_group	Setup the local process group.
get_backend	Return the backend of the given process group.
get_world_size	Return the number of the given process group.
get_rank	Return the rank of the given process group.
get_local_size	Return the number of the current node.
get_local_rank	Return the rank of current process in the current node.
is_main_process	Whether the current rank of the given process group is equal to 0.
master_only	Decorate those methods which should be executed in master process.
barrier	Synchronize all processes from the given process group.
is_distributed	Return True if distributed environment has been initialized.
get_local_group	Return local process group.
get_default_group	Return default process group.
get_data_device	Return the device of data.
get_comm_device	Return the device for communication among groups.
cast_data_device	Recursively convert Tensor in data to device.