Visualizer¶
- class mmengine.visualization.Visualizer(name='visualizer', image=None, vis_backends=None, save_dir=None, fig_save_cfg={'frameon': False}, fig_show_cfg={'frameon': False})[source]¶
MMEngine provides a Visualizer class that uses the
Matplotlib
library as the backend. It has the following functions:Basic drawing methods
draw_bboxes: draw single or multiple bounding boxes
draw_texts: draw single or multiple text boxes
draw_points: draw single or multiple points
draw_lines: draw single or multiple line segments
draw_circles: draw single or multiple circles
draw_polygons: draw single or multiple polygons
draw_binary_masks: draw single or multiple binary masks
draw_featmap: draw feature map
Basic visualizer backend methods
add_configs: write config to all vis storage backends
add_graph: write model graph to all vis storage backends
add_image: write image to all vis storage backends
add_scalar: write scalar to all vis storage backends
add_scalars: write scalars to all vis storage backends
add_datasample: write datasample to all vis storage backends. The abstract drawing interface used by the user
Basic info methods
set_image: sets the original image data
get_image: get the image data in Numpy format after drawing
show: visualization
close: close all resources that have been opened
get_backend: get the specified vis backend
All the basic drawing methods support chain calls, which is convenient for overlaydrawing and display. Each downstream algorithm library can inherit
Visualizer
and implement the add_datasample logic. For example,DetLocalVisualizer
in MMDetection inherits fromVisualizer
and implements functions, such as visual detection boxes, instance masks, and semantic segmentation maps in the add_datasample interface.- Parameters
name (str) – Name of the instance. Defaults to ‘visualizer’.
image (np.ndarray, optional) – the origin image to draw. The format should be RGB. Defaults to None.
vis_backends (list, optional) – Visual backend config list. Default to None.
save_dir (str, optional) – Save file dir for all storage backends. If it is None, the backend storage will not save any data.
fig_save_cfg (dict) – Keyword parameters of figure for saving. Defaults to empty dict.
fig_show_cfg (dict) – Keyword parameters of figure for showing. Defaults to empty dict.
- Return type
Examples
>>> # Basic info methods >>> vis = Visualizer() >>> vis.set_image(image) >>> vis.get_image() >>> vis.show()
>>> # Basic drawing methods >>> vis = Visualizer(image=image) >>> vis.draw_bboxes(np.array([0, 0, 1, 1]), edge_colors='g') >>> vis.draw_bboxes(bbox=np.array([[1, 1, 2, 2], [2, 2, 3, 3]]), >>> edge_colors=['g', 'r']) >>> vis.draw_lines(x_datas=np.array([1, 3]), >>> y_datas=np.array([1, 3]), >>> colors='r', line_widths=1) >>> vis.draw_lines(x_datas=np.array([[1, 3], [2, 4]]), >>> y_datas=np.array([[1, 3], [2, 4]]), >>> colors=['r', 'r'], line_widths=[1, 2]) >>> vis.draw_texts(text='MMEngine', >>> position=np.array([2, 2]), >>> colors='b') >>> vis.draw_texts(text=['MMEngine','OpenMMLab'], >>> position=np.array([[2, 2], [5, 5]]), >>> colors=['b', 'b']) >>> vis.draw_circles(circle_coord=np.array([2, 2]), radius=np.array[1]) >>> vis.draw_circles(circle_coord=np.array([[2, 2], [3, 5]), >>> radius=np.array[1, 2], colors=['g', 'r']) >>> vis.draw_polygons(np.array([0, 0, 1, 0, 1, 1, 0, 1]), >>> edge_colors='g') >>> vis.draw_polygons(bbox=[np.array([0, 0, 1, 0, 1, 1, 0, 1], >>> np.array([2, 2, 3, 2, 3, 3, 2, 3]], >>> edge_colors=['g', 'r']) >>> vis.draw_binary_masks(binary_mask, alpha=0.6) >>> heatmap = vis.draw_featmap(featmap, img, >>> channel_reduction='select_max') >>> heatmap = vis.draw_featmap(featmap, img, channel_reduction=None, >>> topk=8, arrangement=(4, 2)) >>> heatmap = vis.draw_featmap(featmap, img, channel_reduction=None, >>> topk=-1)
>>> # chain calls >>> vis.draw_bboxes().draw_texts().draw_circle().draw_binary_masks()
>>> # Backend related methods >>> vis = Visualizer(vis_backends=[dict(type='LocalVisBackend')], >>> save_dir='temp_dir') >>> cfg = Config(dict(a=1, b=dict(b1=[0, 1]))) >>> vis.add_config(cfg) >>> image=np.random.randint(0, 256, size=(10, 10, 3)).astype(np.uint8) >>> vis.add_image('image',image) >>> vis.add_scaler('mAP', 0.6) >>> vis.add_scalars({'loss': 0.1,'acc':0.8})
>>> # inherit >>> class DetLocalVisualizer(Visualizer): >>> def add_datasample(self, >>> name, >>> image: np.ndarray, >>> gt_sample: >>> Optional['BaseDataElement'] = None, >>> pred_sample: >>> Optional['BaseDataElement'] = None, >>> draw_gt: bool = True, >>> draw_pred: bool = True, >>> show: bool = False, >>> wait_time: int = 0, >>> step: int = 0) -> None: >>> pass
- add_config(config, **kwargs)[source]¶
Record the config.
- Parameters
config (Config) – The Config object.
- add_datasample(name, image, data_sample=None, draw_gt=True, draw_pred=True, show=False, wait_time=0, step=0)[source]¶
Draw datasample.
- Parameters
image (numpy.ndarray) –
data_sample (Optional[mmengine.structures.base_data_element.BaseDataElement]) –
draw_gt (bool) –
draw_pred (bool) –
show (bool) –
wait_time (int) –
step (int) –
- Return type
- add_graph(model, data_batch, **kwargs)[source]¶
Record the model graph.
- Parameters
model (torch.nn.Module) – Model to draw.
data_batch (Sequence[dict]) – Batch of data from dataloader.
- Return type
- add_scalars(scalar_dict, step=0, file_path=None, **kwargs)[source]¶
Record the scalars’ data.
- Parameters
- Return type
- draw_bboxes(bboxes, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶
Draw single or multiple bboxes.
- Parameters
bboxes (Union[np.ndarray, torch.Tensor]) – The bboxes to draw with the format of(x1,y1,x2,y2).
edge_colors (Union[str, tuple, List[str], List[tuple]]) – The colors of bboxes.
colors
can have the same length with lines or just single value. Ifcolors
is single value, all the lines will have the same colors. Refer to matplotlib. colors for full list of formats that are accepted. Defaults to ‘g’.line_styles (Union[str, List[str]]) – The linestyle of lines.
line_styles
can have the same length with texts or just single value. Ifline_styles
is single value, all the lines will have the same linestyle. Reference to https://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestyle for more details. Defaults to ‘-‘.line_widths (Union[Union[int, float], List[Union[int, float]]]) – The linewidth of lines.
line_widths
can have the same length with lines or just single value. Ifline_widths
is single value, all the lines will have the same linewidth. Defaults to 2.face_colors (Union[str, tuple, List[str], List[tuple]]) – The face colors. Default to None.
alpha (Union[int, float]) – The transparency of bboxes. Defaults to 0.8.
- Return type
- draw_binary_masks(binary_masks, colors='g', alphas=0.8)[source]¶
Draw single or multiple binary masks.
- Parameters
binary_masks (np.ndarray, torch.Tensor) – The binary_masks to draw with of shape (N, H, W), where H is the image height and W is the image width. Each value in the array is either a 0 or 1 value of uint8 type.
colors (np.ndarray) – The colors which binary_masks will convert to.
colors
can have the same length with binary_masks or just single value. Ifcolors
is single value, all the binary_masks will convert to the same colors. The colors format is RGB. Defaults to np.array([0, 255, 0]).alphas (Union[int, List[int]]) – The transparency of masks. Defaults to 0.8.
- Return type
- draw_circles(center, radius, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶
Draw single or multiple circles.
- Parameters
center (Union[np.ndarray, torch.Tensor]) – The x coordinate of each line’ start and end points.
radius (Union[np.ndarray, torch.Tensor]) – The y coordinate of each line’ start and end points.
edge_colors (Union[str, tuple, List[str], List[tuple]]) – The colors of circles.
colors
can have the same length with lines or just single value. Ifcolors
is single value, all the lines will have the same colors. Reference to https://matplotlib.org/stable/gallery/color/named_colors.html for more details. Defaults to ‘g.line_styles (Union[str, List[str]]) – The linestyle of lines.
line_styles
can have the same length with texts or just single value. Ifline_styles
is single value, all the lines will have the same linestyle. Reference to https://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestyle for more details. Defaults to ‘-‘.line_widths (Union[Union[int, float], List[Union[int, float]]]) – The linewidth of lines.
line_widths
can have the same length with lines or just single value. Ifline_widths
is single value, all the lines will have the same linewidth. Defaults to 2.face_colors (Union[str, tuple, List[str], List[tuple]]) – The face colors. Default to None.
alpha (Union[int, float]) – The transparency of circles. Defaults to 0.8.
- Return type
- static draw_featmap(featmap, overlaid_image=None, channel_reduction='squeeze_mean', topk=20, arrangement=(4, 5), resize_shape=None, alpha=0.5)[source]¶
Draw featmap.
If overlaid_image is not None, the final output image will be the
weighted sum of img and featmap.
If resize_shape is specified, featmap and overlaid_image
are interpolated.
If resize_shape is None and overlaid_image is not None,
the feature map will be interpolated to the spatial size of the image in the case where the spatial dimensions of overlaid_image and featmap are different.
If channel_reduction is “squeeze_mean” and “select_max”,
it will compress featmap to single channel image and weighted sum to overlaid_image.
if channel_reduction is None
If topk <= 0, featmap is assert to be one or three
channel and treated as image and will be weighted sum to
overlaid_image
. - If topk > 0, it will select topk channel to show by the sum of each channel. At the same time, you can specify the arrangement to set the window layout.- Parameters
featmap (torch.Tensor) – The featmap to draw which format is (C, H, W).
overlaid_image (np.ndarray, optional) – The overlaid image. Default to None.
channel_reduction (str, optional) – Reduce multiple channels to a single channel. The optional value is ‘squeeze_mean’ or ‘select_max’. Defaults to ‘squeeze_mean’.
topk (int) – If channel_reduction is not None and topk > 0, it will select topk channel to show by the sum of each channel. if topk <= 0, tensor_chw is assert to be one or three. Defaults to 20.
arrangement (Tuple[int, int]) – The arrangement of featmap when channel_reduction is not None and topk > 0. Defaults to (4, 5).
resize_shape (tuple, optional) – The shape to scale the feature map. Default to None.
alpha (Union[int, List[int]]) – The transparency of featmap. Defaults to 0.5.
- Returns
RGB image.
- Return type
np.ndarray
- draw_lines(x_datas, y_datas, colors='g', line_styles='-', line_widths=2)[source]¶
Draw single or multiple line segments.
- Parameters
x_datas (Union[np.ndarray, torch.Tensor]) – The x coordinate of each line’ start and end points.
y_datas (Union[np.ndarray, torch.Tensor]) – The y coordinate of each line’ start and end points.
colors (Union[str, tuple, List[str], List[tuple]]) – The colors of lines.
colors
can have the same length with lines or just single value. Ifcolors
is single value, all the lines will have the same colors. Reference to https://matplotlib.org/stable/gallery/color/named_colors.html for more details. Defaults to ‘g’.line_styles (Union[str, List[str]]) – The linestyle of lines.
line_styles
can have the same length with texts or just single value. Ifline_styles
is single value, all the lines will have the same linestyle. Reference to https://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestyle for more details. Defaults to ‘-‘.line_widths (Union[Union[int, float], List[Union[int, float]]]) – The linewidth of lines.
line_widths
can have the same length with lines or just single value. Ifline_widths
is single value, all the lines will have the same linewidth. Defaults to 2.
- Return type
- draw_points(positions, colors='g', marker=None, sizes=None)[source]¶
Draw single or multiple points.
- Parameters
positions (Union[np.ndarray, torch.Tensor]) – Positions to draw.
colors (Union[str, tuple, List[str], List[tuple]]) – The colors of points.
colors
can have the same length with points or just single value. Ifcolors
is single value, all the points will have the same colors. Reference to https://matplotlib.org/stable/gallery/color/named_colors.html for more details. Defaults to ‘g.marker (str, optional) – The marker style. See
matplotlib.markers
for more information about marker styles. Default to None.sizes (Optional[Union[np.ndarray, torch.Tensor]]) – The marker size. Default to None.
- draw_polygons(polygons, edge_colors='g', line_styles='-', line_widths=2, face_colors='none', alpha=0.8)[source]¶
Draw single or multiple bboxes.
- Parameters
(Union[Union[np.ndarray (polygons) – List[Union[np.ndarray, torch.Tensor]]]): The polygons to draw with the format of (x1,y1,x2,y2,…,xn,yn).
torch.Tensor] – List[Union[np.ndarray, torch.Tensor]]]): The polygons to draw with the format of (x1,y1,x2,y2,…,xn,yn).
polygons (Union[numpy.ndarray, torch.Tensor, List[Union[numpy.ndarray, torch.Tensor]]]) –
- Return type
- :paramList[Union[np.ndarray, torch.Tensor]]]): The polygons to draw
with the format of (x1,y1,x2,y2,…,xn,yn).
- Parameters
edge_colors (Union[str, tuple, List[str], List[tuple]]) – The colors of polygons.
colors
can have the same length with lines or just single value. Ifcolors
is single value, all the lines will have the same colors. Refer to matplotlib.colors for full list of formats that are accepted. Defaults to ‘g.line_styles (Union[str, List[str]]) – The linestyle of lines.
line_styles
can have the same length with texts or just single value. Ifline_styles
is single value, all the lines will have the same linestyle. Reference to https://matplotlib.org/stable/api/collections_api.html?highlight=collection#matplotlib.collections.AsteriskPolygonCollection.set_linestyle for more details. Defaults to ‘-‘.line_widths (Union[Union[int, float], List[Union[int, float]]]) – The linewidth of lines.
line_widths
can have the same length with lines or just single value. Ifline_widths
is single value, all the lines will have the same linewidth. Defaults to 2.face_colors (Union[str, tuple, List[str], List[tuple]]) – The face colors. Default to None.
alpha (Union[int, float]) – The transparency of polygons. Defaults to 0.8.
polygons (Union[numpy.ndarray, torch.Tensor, List[Union[numpy.ndarray, torch.Tensor]]]) –
- Return type
- draw_texts(texts, positions, font_sizes=None, colors='g', vertical_alignments='top', horizontal_alignments='left', font_families='sans-serif', bboxes=None)[source]¶
Draw single or multiple text boxes.
- Parameters
positions (Union[np.ndarray, torch.Tensor]) – The position to draw the texts, which should have the same length with texts and each dim contain x and y.
font_sizes (Union[int, List[int]], optional) – The font size of texts.
font_sizes
can have the same length with texts or just single value. Iffont_sizes
is single value, all the texts will have the same font size. Defaults to None.colors (Union[str, tuple, List[str], List[tuple]]) – The colors of texts.
colors
can have the same length with texts or just single value. Ifcolors
is single value, all the texts will have the same colors. Reference to https://matplotlib.org/stable/gallery/color/named_colors.html for more details. Defaults to ‘g.vertical_alignments (Union[str, List[str]]) – The verticalalignment of texts. verticalalignment controls whether the y positional argument for the text indicates the bottom, center or top side of the text bounding box.
vertical_alignments
can have the same length with texts or just single value. Ifvertical_alignments
is single value, all the texts will have the same verticalalignment. verticalalignment can be ‘center’ or ‘top’, ‘bottom’ or ‘baseline’. Defaults to ‘top’.horizontal_alignments (Union[str, List[str]]) – The horizontalalignment of texts. Horizontalalignment controls whether the x positional argument for the text indicates the left, center or right side of the text bounding box.
horizontal_alignments
can have the same length with texts or just single value. Ifhorizontal_alignments
is single value, all the texts will have the same horizontalalignment. Horizontalalignment can be ‘center’,’right’ or ‘left’. Defaults to ‘left’.font_families (Union[str, List[str]]) –
The font family of texts.
font_families
can have the same length with texts or just single value. Iffont_families
is single value, all the texts will have the same font family. font_familiy can be ‘serif’, ‘sans-serif’, ‘cursive’, ‘fantasy’or ‘monospace’. Defaults to ‘sans-serif’.
bboxes (Union[dict, List[dict]], optional) – The bounding box of the texts. If bboxes is None, there are no bounding box around texts.
bboxes
can have the same length with texts or just single value. Ifbboxes
is single value, all the texts will have the same bbox. Reference to https://matplotlib.org/stable/api/_as_gen/matplotlib.patches.FancyBboxPatch.html#matplotlib.patches.FancyBboxPatch for more details. Defaults to None.
- Return type
- get_backend(name)[source]¶
get vis backend by name.
- Parameters
name (str) – The name of vis backend
- Returns
The vis backend.
- Return type
- get_image()[source]¶
Get the drawn image. The format is RGB.
- Returns
the drawn image which channel is RGB.
- Return type
np.ndarray
- classmethod get_instance(name, **kwargs)[source]¶
Make subclass can get latest created instance by
Visualizer.get_current_instance()
.Downstream codebase may need to get the latest created instance without knowing the specific Visualizer type. For example, mmdetection builds visualizer in runner and some component which cannot access runner wants to get latest created visualizer. In this case, the component does not know which type of visualizer has been built and cannot get target instance. Therefore,
Visualizer
overrides theget_instance()
and its subclass will register the created instance to_instance_dict
additionally.get_current_instance()
will return the latest created subclass instance.Examples
>>> class DetLocalVisualizer(Visualizer): >>> def __init__(self, name): >>> super().__init__(name) >>> >>> visualizer1 = DetLocalVisualizer.get_instance('name1') >>> visualizer2 = Visualizer.get_current_instance() >>> visualizer3 = DetLocalVisualizer.get_current_instance() >>> assert id(visualizer1) == id(visualizer2) == id(visualizer3)
- set_image(image)[source]¶
Set the image to draw.
- Parameters
image (np.ndarray) – The image to draw.
- Return type
- show(drawn_img=None, win_name='image', wait_time=0, continue_key=' ')[source]¶
Show the drawn image.
- Parameters
drawn_img (np.ndarray, optional) – The image to show. If drawn_img is None, it will show the image got by Visualizer. Defaults to None.
win_name (str) – The image title. Defaults to ‘image’.
wait_time (int) – Delay in milliseconds. 0 is the special value that means “forever”. Defaults to 0.
continue_key (str) – The key for users to continue. Defaults to the space key.
- Return type