hvplot_docs/Annotators.md

import holoviews as hv
import numpy as np
import panel as pn

hv.extension('bokeh')

HoloViews-generated plots generally convey information from Python to a viewer of the plot, but there are also circumstances where information needs to be collected from the viewer and made available for processing in Python:

• annotating data with contextual information to aid later interpretation
• labeling or tagging data for automated machine-learning or other processing pipelines
• indicating regions of interest, outliers, or other semantic information
• specifying inputs to a query, command, or simulation
• testing sensitivity of analyses to adding, changing, or deleting user-selected data points

In such cases, it is important to be able to augment, edit, and annotate datasets and to access those values from Python. To perform these actions, HoloViews provides an annotate helper using Bokeh's drawing tools to make it easy to edit HoloViews Elements and add additional information using an associated table. The annotate helper:

• Adds plot tools that allow editing and adding new elements to a plot
• Adds table(s) to allow editing the element in a tabular format
• Returns a layout of these two components
• Makes the edits, annotations, and selections available on a property of the annotate object so that they can be utilized in Python

Basics

Let us start by annotating a small set of Points. To do this, we need two things:

1. A Points Element to annotate or edit
2. An annotator object to collect and store annotations

The annotator is a callable object with its own state that can be called to return a Layout consisting of the object to be annotated and an Overlay of editable table(s):

points = hv.Points([(0.0, 0.0), (1.0, 1.0), (200000.0, 2000000.0)]).opts(size=10, min_height=500)

annotator = hv.annotate.instance()
layout = annotator(hv.element.tiles.OSM() * points, annotations=['Label'], name="Point Annotations")

print(layout)

This layout of a DynamicMap (the user-editable Element data) and an Overlay (the user-editable table) lets a user input the required information:

layout

Here we have pre-populated the Element with three points. Each of the points has three bits of information that can be edited using the table: the x location, y location, and a "Label", which was initialized to dummy values when we called annotate and asked that there be a Label column. Try clicking on one of the rows and editing the location or the label to anything you like. As long as Python is running and the new location is in the viewport, you should see the dot move when you edit the location, and any labels you entered should be visible in the table.

You can also edit the locations graphically using the PointDraw tool in the toolbar:

Once you select that tool, you should be able to click and drag any of the existing points and see the location update in the table. Whether you click on the table or the points, the same object should be selected in each, so that you can see how the graphical and tabular representations relate.

The PointDraw tool also allows us to add completely new points; once the tool is selected, just click on the plot above in locations not already containing a point and you can see a new point and a new table row appear ready for editing. You can also delete points by selecting them in the plot then pressing Backspace or Delete (depending on operating system).

All the above editing and interaction could have been done if we had simply called hv.annotate(points, annotations=['Label']) directly, but instead we first saved an "instance" of the annotator object so that we'd also be able to access the resulting data. So, once we are done collecting data from the user, let's use the saved annotator object handle to read out the values (by re-evaluating the following line):

annotator.annotated.dframe()

You should see that you can access the current set of user-provided or user-modified points and their user-provided labels from within Python, ready for any subsequent processing you need to do.

We can also access the currently selected points, in case we care only about a subset of the points (which will be empty if no points/rows are selected):

annotator.selected.dframe()

Configuring the Annotator

In addition to managing the list of annotations, the annotate helper exposes a few additional parameters. Remember like most Param-based objects you can get help about annotate parameters using the hv.help function:

hv.help(hv.annotate)

Annotation types

The default annotation type is a string, to allow you to put in arbitrary information that you later process in Python. If you want to enforce a more specific type, you can specify the annotation-value types explicitly using a dictionary mapping from column name to the type:

hv.annotate(points, annotations={'int': int, 'float': float, 'str': str})

This example also shows how to collect multiple columns of information for the same data point.

Types of Annotators

Currently only a limited set of Elements may be annotated, which include:

• Points/Scatter
• Curve
• Path
• Polygons
• Rectangles

Adding support for new elements, in most cases, requires adding corresponding drawing/edit tools to Bokeh itself. But if you have data of other types, you may still be able to annotate it by casting it to one of the indicated types, collecting the data, then casting it back.

Annotating Curves

To allow dragging the vertices of the Curve, the Curve annotator uses the PointDraw tool in the toolbar: The vertices will appear when the tool is selected or a vertex is selected in the table. Unlike most other annotators the Curve annotator only allows editing the vertices and does not allow adding new ones.

curve = hv.Curve(np.random.randn(50).cumsum())

curve_annotator = hv.annotate.instance()

curve_annotator(curve.opts(width=800, height=400, responsive=False), annotations={'Label': str})

To access the data you can make use of the annotated property on the annotator:

curve_annotator.annotated.dframe().head(5)

Annotating Rectangles

The Rectangles annotator behaves very similarly to the Points annotator. It allows adding any number of annotation columns, using Bokeh's BoxEdit tool that allows both drawing and moving boxes. To see how to use the BoxEdit tool, refer to the HoloViews BoxEdit stream reference, but briefly:

• Select the BoxEdit tool in the toolbar:
• Click and drag on an existing Rectangle to move it
• Double click to start drawing a new Rectangle at one corner, and double click to complete the rectangle at the opposite corner
• Select a rectangle and press the Backspace or Delete key (depending on OS) to delete it
• Edit the box coordinates in the table to resize it

boxes = hv.Rectangles([(0, 0, 1, 1), (1.5, 1.5, 2.5, 2.5)])

box_annotator = hv.annotate.instance()

box_annotator(boxes.opts(width=800, height=400, responsive=False), annotations=['Label'])

To access the data we can make use of the annotated property on the annotator instance:

box_annotator.annotated.dframe()

Annotating paths/polygons

Unlike the Points and Boxes annotators, the Path and Polygon annotators allow annotating not just each individual entity but also the vertices that make up the paths and polygons. For more information about using the editing tools associated with this annotator refer to the HoloViews PolyDraw and PolyEdit stream reference guides, but briefly:

Drawing/Selecting Deleting Paths/Polygons

• Select the PolyDraw tool in the toolbar:
• Double click to start a new object, single click to add each vertex, and double-click to complete it.
• Delete paths/polygons by selecting and pressing Delete key (OSX) or Backspace key (PC)

Editing Paths/Polygons

• Select the PolyEdit tool in the toolbar:
• Double click a Path/Polygon to start editing
• Drag vertices to edit them, delete vertices by selecting them

To edit and annotate the vertices, use the draw tool or the first table to select a particular path/polygon and then navigate to the Vertices tab.

path = hv.Path([hv.Box(0, 0, 1), hv.Ellipse(1, 1, 1)])

path_annotator = hv.annotate.instance()

path_annotator(path.opts(width=800, height=400, responsive=False), annotations=['Label'], vertex_annotations=['Value'])

To access the data we can make use of the iloc method on Path objects to access a particular path, and then access the .data or convert it to a dataframe:

path_annotator.annotated.iloc[0].dframe()

Composing Annotators

Often we will want to add some annotations on top of one or more other elements which provide context, e.g. when annotating an image with a set of Points. As long as only one annotation layer is required you can pass an overlay of multiple elements to the annotate operation and it will automatically associate the annotator with the layer that supports annotation. Note however that this will error if multiple elements that support annotation are provided. Below we will annotate a two-photon microscopy image with a set of Points, e.g. to mark the location of each cell:

img = hv.Image(np.load('../assets/twophoton.npz')['Calcium'][..., 0])
cells = hv.Points([]).opts(width=500, height=500, responsive=False, padding=0)

hv.annotate(img * cells, annotations=['Label'], name="Cell Annotator")

Multiple Annotators

If you want to work with multiple annotators in the same plot, you can recompose and rearrange the components returned by each annotate helper manually, but doing so can get confusing. To simplify working with multiple annotators at the same time, the annotate helper provides a special classmethod that allows composing multiple annotators and other elements, e.g. making a set of tiles into a combined layout consisting of all the components:

point_annotate = hv.annotate.instance()
points = hv.Points([(500000, 500000), (1000000, 1000000)]).opts(size=10, color='red', line_color='black')
point_layout = point_annotate(points, annotations=['Label'])

poly_annotate = hv.annotate.instance()
poly_layout = poly_annotate(hv.Polygons([]), annotations=['Label'])

hv.annotate.compose(hv.element.tiles.OSM(), point_layout, poly_layout)

Internals

The annotate function builds on Param and Panel, creating and wrapping Panel Annotator panes internally. These objects make it easy to include the annotator in Param-based workflows and trigger actions when parameters change and/or update the annotator in response to external events. The Annotator of a annotate instance can be accessed using the annotator attribute:

print(point_annotate.annotator)

This object can be included directly in a Panel layout, be used to watch for parameter changes, or updated directly. To see the effect of updating directly, uncomment the line below, execute that cell, and then look at the previous plot of Africa, which should get updated with 10 randomly located blue dots.

#point_annotate.annotator.object = hv.Points(np.random.randn(10, 2)*1000000).opts(color='blue')