Numerical features and tags. The table and grapher view.

Mastodon is mainly a tracking and lineaging tool. Its output is a collection of tracks, and the analysis of these tracks to yield statistics on aspects, such as velocity, displacement, etc. is often carried out in another software package such as MATLAB or Python. Nonetheless, you will find in Mastodon tools to compute numerical features as data items. Numerical features are numbers that can be calculated on spots, links and branches (i.e. an individual spot during its lifetime) of the data. For instance, there are features for the number of links that touch a spot, or the displacement of a link or the number of spots in a track. You can find them within Mastodon because it is convenient but also because they are very useful for the interactive exploration of your data. See also the grapher view below. Coupled with feature-based coloring, the display and sorting of values in the table view and the selection creator tool can considerably speed up making sense of the data.

Overview of available numerical features: numerical features

Numerical features are numbers that classically relate to a physical quantity. When we need to categorize items, we rely on tags. We describe them just below. This chapter will also show you how to compute numerical features and create a coloring view from the feature values and tags. Doing so, we will introduce the third kind of data view in Mastodon: the data tables.

Tags and tag-sets.

As we said above, every-time you need to categorize certain data items, or need to visualize categories, you should rely on tags. Let’s suppose that you are investigating the trajectories of cells in a developing embryo from an early stage to a stage where the embryo is polarized. Some cells will migrate to the anterior part, some others to the posterior part, etc. You might want to tag cell tracks with the Anterior or Posterior tag, to investigate where these cells come from in the early embryo. Or let’s say that you are curating the results of the automated tracking on large images. The tracking results might have some inaccuracies, and you want to correct them for important tracks. Because there are a lot of tracks, you might share the workload with some colleagues. You work asynchronously with them, editing the Mastodon file one after another. Doing so, you can use tags in Mastodon to mark some tracks as reviewed by you. Your colleagues will use a tag for themselves to ensure that no two scientists are reviewing the same track twice. All the cells that are not tagged in this categorization are still waiting to be reviewed.

In Mastodon, a categorization corresponds to a tag-set. A tag-set defines a property that can have a reasonable number of discrete values, or tags. In the first of the two examples above, Location would be a tag-set to specify the location of cells. Anterior and Posterior would be two tags belonging to the Location tag-set. In the second example, Reviewed by would be a tag-set, and Mette, Pavel, Tobias and Jean-Yves would be 4 tags of this tag-set.

You can assign tags to spots and links. To assign a tag to a whole track, you have to assign this tag to all the spots and links of this track. One data item (a spot or a link) can have 1 or 0 tags per existing tag-set, meaning that it can have only one category per tag-set. They can, however, be categorized by as many tag-sets as there are in the data set. For instance, a spot can have the tag Anterior in the Location tag-set, and the tag Pavel in the Reviewed by tag-set. Or it can be not tagged at all in the Reviewed by tag-set. But it cannot have both the tag Mette and the tag Tobias because they belong to the same tag-set and are mutually exclusive. Each tag-set works independently, and clearing a tag-set does not affect the others even for one data item. Now that we set things straight, let’s see how to create tag-sets. We will base the demonstration in this chapter on the data we generated in the first tutorial.

Creating tag-sets.

On the main Mastodon window, there is a configure tags button.

Pressing it opens the tag-set dialog. Right now, it appears as an empty table made of two columns. This is where you enter tag-sets and tags.

Creating tag-sets and tags. 1. The empty tag-set dialog. 2. After adding two tag-sets and six tags.

Press the green + button on the left column to create a new tag-set. A default name is shown for the tag-set, that you can edit. You can also directly press the Enter key to immediately start editing the new tag-set. Let’s say we want to create the location tag we talked about before. Type Location in the text field. An empty line followed by a green + button should appear on the right column. This is where you will enter the tags of this tag-set. Click on this button, or press the ↹ key followed by the Enter key to create a new tag. For instance, the Anterior tag. Note that a tag is only made of a label (the text) and a color. The color will be used in Mastodon views. Create a second tag for the same tag-set called Posterior. Pick the color as you like. Now try to create another tag-set called Reviewed by and create some tags in this tag-set. The tag-set dialog is normally fully navigable with the cursor keys, so that you can enter tags quickly if you have a lot of them. You can create new tag-set or new tags with the Enter key, and delete them either with the ⌦ key or by pressing the red - button.

When you are finished, press the OK button.

Assigning tags to data items.

Tags are set via the selection tool, presented in the second tutorial . Once you have some spots and links in the selection, you can assign a tag to it via the menu. The menu content will be updated with the tag-sets and tags you defined in the tag-set dialog, described above. This will work in any Mastodon views, including BDV, TrackScheme, Table and Grapher.

TrackScheme ships a second way to set tags quickly from the keyboard. After selecting the spots and links of interest, press the key Y. A floating menu should appear on the left part of the view panel. There is a bit of naming clash in the floating menu. Here, tag-sets are called tags and tags are called labels. Select the desired tag-set with the 1, 2, .. keys. The menu now shows the tags defined within this tag-set, that you can select the same way. Note that there is a way to remove all the tags over all the tag-sets on the selection by pressing on the first menu, or just the tags of the selected tag-set by pressing on the second menu.

Assigning tags in TrackScheme. After pressing the `Y` key this floating menu is shown, that can be navigated with the digit keys.

Coloring views by tag-sets.

The tags we just defined and assigned can be used in the views to highlight the items that are tagged. In the View > Coloring menu of any view in Mastodon, you will find a sub-menu updated with the tag-sets you created among other choices. By default, newly created views are colored with the coloring mode, which simply colors all the spots and links the same way, taking colors from display settings. If you select a mode corresponding to a tag-set, tagged spots and links will appear painted with the color you chose for the tags of this tag-set. This is very handy to mark some locations in the image or highlight interesting tracks in the data. Later we will see that tags can be used to retrieve specific items for further processing. Finally, there is an option to show a legend of the current coloring mode. You can toggle it on or off and set the location of this legend in the View > Colorbar menu.

Numerical features.

Numerical features are values that are calculated from the data. For instance, the mean intensity within a spot, or the displacement along a link. They are very generic: the main restrictions are that there must be a data item (a spot or a link) per feature value. But the feature itself can be scalar, non-scalar, real, integer, a string, a vector, etc. They are labile. Because they are defined for a data item, they will become invalid as soon as the data item changes. Think of what happens to the spot mean intensity if the spot is moved over the image for instance. Because we want to accommodate extensibility and large data, we have to use a special system that we describe below.

Feature computation.

Some numerical feature values are calculated by feature computers. Feature computers are actually specialized Mastodon plugins, made so that it is easy for a 3rd party (you) to implement their own features in Mastodon. We explain how to write your own feature computer in the second part of this documentation, dedicated to technical information.

Because some feature computation can take very long on large images, you have to trigger it manually. On Mastodon main window, you can find a button compute features. Pressing it will show the feature computation dialog. The feature computers are listed on the left panel.

You see in the panel only the feature computers that takes a significant time to compute. The other ones are computed on the fly and are always visible in the data tables (see below). For instance, the Link displacement and Link velocity are computed on the fly and do not appear in this panel.

Clicking on the computer names displays some information about the feature they compute in the right panel. Note that they are named ‘features’ on this panel, but they are in reality the feature computers. For instance if you click on the Spot intensity, you will see in the information panel that this computer generates a feature for the mean intensity, median intensity, etc. Note also that they can have dependencies. For instance, the Track N spots feature computer depends on the Spot track ID feature to be present at the time of computation.

The check-box on the left of each feature computer name triggers whether they will be part of the next feature computation. Press the Compute button to trigger computation of features.

Once the computation of all the features is complete, all the small clock icons that were shown right to the feature computer names now turned to a green dot. This is how we keep track of the validity of the feature values. Since the feature computation is triggered manually, and that a feature value might have become invalid after data changes (new spots added, removed, moved, changed the radius, added or removed some links), this icon serves as a signal for feature value de-synchronization. If the icon shows as a clock, it means that the data has changed since the last feature computation, and that the feature values are out of sync. If it shows a green dot, then the data has not changed since the last computation, and the feature values are sure to be valid. This is very important for proper interpretation of the data, and you will have to show the computation dialog often just to check the feature values validity. By the way, you can check now how the validity flag works. While keeping the feature computation dialog open, move a spot in a BDV view. You should see that all the green dot icons now turn to the clock icon. Also, if you now deselect some feature computers before launching a new computation, the validity flag will not turn to the green dot icon for those feature computers.

You probably have noticed, thanks to the progress bar, that the intensity-related features are the ones that take the most time to compute. Indeed, they require loading all of the data blocks on which there are spots. This could rapidly become cumbersome for large datasets, if you have to recompute all of the features every-time a spot is added or edited. Fortunately, we implemented an update mechanism for feature computation. After the first computation, which possibly took very long, only the spots that have been added or modified since the last computation are considered for computation.

Now that we have feature values computed, we would like to inspect them and export them for further analysis. This is the role of the table view, but before getting to it, we will make a little detour to showing how to use features to generate coloring, accelerating updates of computation and saving them to disk.

Coloring views by numerical features.

We have seen above that tag-sets could be used to generate coloring of the data items shown in a view. Indeed, in the View > Coloring menu of each view, that tag-sets are listed and when selected, are used to assign a color to each data item. We can do something similar with feature values, except that coloring based on features requires more input from us.

Feature color modes.

Feature color modes need to be created first, and this is done in a dedicated user interface. Select the File > Preferences menu item in the main window or any view. This shows the dialog preferences. It is organized with a side-bar on the left that contains the various items that can be configured in Mastodon. Parenthetically, you can see that you can configure the display style of the TrackScheme, Grapher and BDV views, and the keymaps. But we will see this later. In the sidebar select Feature Color Modes. The panel on the right now displays the feature color mode configuration panel:

Its top line has a drop-down list that contains all the color modes already defined. Right now, there is only one, called Number of links. As the built-in suffix indicates, it is a built-in color mode, and it cannot be edited. To create a new color mode, you must duplicate it and rename the new one. Do so by clicking on the Duplicate button, then on Rename. Let’s create a color mode that colors spots and links based on the mean intensity inside the spots, that we would call * Mean intensity*.

The rest of the configuration panel is made of two parts. The top part configures the spot coloring, or how we color spots. The bottom part configures the link coloring, or how we color links. In Mastodon, the data is organized in a mathematical graph, in which the vertices are the spots, and the edges are the links that connect spots from one frame to another, so you will sometimes find in Mastodon and in this manual the vocables vertex and edge to design a spot and a link respectively. The vertex color mode specifies where we take colors from. You can choose between:

• Spot, which means a spot will take its color from a feature value it owns.

• Incoming link, which means a spot will take its color from a feature value owned by the single incoming link that targets this spot. This is the link backward in time. If there are no such links or more than one, then the default color is used.

• Outgoing link is the same thing, but for the link forward in time.

• Default mode does not rely on feature values but simply uses the default color in the view.

There are also modes that are related to the branch graph (Branch-spot, etc.) but we will discuss them in its dedicated tutorial.

Depending on the mode you chose, the content of the drop-down list below, called Spot feature, will change to reflect either the list of spot features or the link features. Select Spot as a color mode and Spot intensity as feature. Two new drop-down lists appear on the right of the feature list. One contains the list of projections in the feature, and the second one contains the list of channels in the dataset.

Feature projections.

We need to explain feature projections a bit. We said above that a feature could be roughly anything numerical and was not necessarily a scalar. It could be a vector, a tensor, a complex number, etc. However, to be usable and useful in Mastodon, features are required to expose a sensible list of projections that compose them. Feature projections are scalar and real values that can decompose or project a feature on a real axis. How they are defined is up to the person that created the feature computer, but we can rely on the hope that they choose wisely. For instance, a feature that gives the velocity vector of a link will reasonably expose 3 projections, one for each of the X, Y and Z component of the vector. Or maybe the polar angle, azimuthal angle and norm of this vector. Or maybe the 6 projections since they can be calculated on the fly. A complex feature value will reasonably expose 2 projections, one for the real part, one of the imaginary part. Etc. The Spot intensity feature has six projections: mean, max, min, median, sum and standard deviation of the intensity inside a spot. Since all can be computed on any of the channel present in the dataset, their number is multiplied by the number of channels. The configuration panel changes, according to the number of projections in a feature and its multiplicity. For features that are made of one real value with no multiplicity, the projection list is superfluous and not shown. In our case, we simply want the mean of the only channel in the dataset.

Color mode configuration.

Coming back to the color mode configuration, next we need to pick a color-map. A color-map acts as the LUT for an image, and maps a color to a certain value. Mastodon ships about 20 of them, many taken from the MatPlotLib project. Finally, you have to specify a min value and a max value that will act as the brightness and contrast values for an image. Values below the min you defined will all be displayed with the first color of the color-map and values larger than the max with the last. The black square you see next to the graded representation of the color-map is the color used for data items for which the feature value is not present or undefined (division by zero, etc). The autoscale button computes the min and max automatically from the feature currently selected with the values taken from the last feature computation.

The link coloring works exactly the same, except for the link color mode.

• Link means that a link will be colored by a feature value it owns.

• Source spot will take a feature value from the source spot of this link, that is, the first in time.

• Target spot does the same but for the last spot in time of this link.

• Default mode does not rely on feature values but simply uses the default color in the view.

To build our example feature color mode, choose Source spot as color mode and use for instance the viridis color-map along with 700 and 1100 for min and max values. You can also click on the Copy from Spot settings that will automatically configure the link color mode so that it matches what is set for spots.

Like for tag-sets, the menu is now updated with items corresponding to the color modes we created. If they are grayed-out, it means that the feature values they depend on are not yet computed. This kind of view immediately reveals the important aspect of the data, even at a very high level. For instance, with our custom color mode, we can quickly find cells that are the brightest and visually inspect how the intensity in cells change over time.

The data table views.

The main table view.

The BDV and trackscheme views are not suitable to display all the feature values we computed. The coloring we have been using with them is good only for visualization purposes. There is a nice view to properly inspect and exploit feature values in later steps in your analysis: the table view. In practice, the table view is simply a tabular representation of the data items in Mastodon. Spots and links are displayed in a list where a single row corresponds to a data item, and columns to feature values and tags. You can create a new table view by using the menu. If you did compute all the features, it should look like the table below:

The table view, with no features computed and no tags defined. Left: the table for spots. Right: the table for links.

The view is made of several tables, the first one for spots and the second one for links. We will discuss the others, made for the branch-graph, later. Right now it is pretty empty. The spot and link tables only show the label and ID of the spots and a few feature values that are built-in. But they do not show the spot intensity features we configured above. Navigating in this table is done classically: the arrow keys ↑ and ↓ jump from one row to the next, and ← and → from one column to the next. ⇞ and ⇟ jump page per page. Control ⇞ and Control ⇟ moves across tables.

After computing some features and defining some tag-sets, the table shows new columns Note that the column headers represent the feature with their projection and physical units on several rows. For instance, the Spot intensity feature name is displayed on the first row of the column header. The header is split in two columns on the second row, one for each projection included in the feature. And in the third and last row, the units of each projection are displayed in brackets (Counts in this case). The headers of the tag-set columns are similar. The first row shows the name of the tag-set, and the second row shows each tag the set contains, with the tag chosen color as background.

The table view can be used to edit in part the data. For instance, you can edit the spot label directly in the table. Just navigate to the row of a spot you want to change the name of and the Label column, then press F2. The label field becomes editable. When you are done editing, press Enter. The tags are displayed as check-boxes in the table that you can set directly by clicking on them. Or you can navigate the desired row and column and set them with the space key.

The highlight and selection are also shared with table views. When the table view is not active, selected items are shown with a gray background. The highlight spot or link is shown in the table with a thick black border.

To add rows to the selection, the default key-bindings are again standard. Press Shift Left-click to add a range of rows to the selection from a table view, or use Shift ↑ or Shift ↓ or Shift ⇞ and Shift ⇟. By pressing Control Left-click or Command Left-click you can toggle single rows in and out of the selection. Of course, all commands related to the selection we have seen before also apply to the table views. The shortcuts to navigate in the table views are summarized in the table of table shortcuts.

Another feature of data tables is that they can be made slave of a spatial context, like for TrackScheme. When another BDV view is active, you can select its name in the drop-down list on the top-right part of the table (named context). Then the table only shows the data items that of the spots that are currently displayed in the master BDV view. The notion of spatial context is explained in the previous tutorial.

Sorting rows.

The table can be sorted by clicking on the header of the column you want to use for sorting. It works for labels, IDs, feature values and tags

The selection table.

There exists a variation of the table view, but that only displays what is currently in the selection. To display such a table, go to Window > New selection table view in the menu. The selection table that appears only shows what is in the selection and is constantly updated to reflect changes in the selection. You cannot use it to edit the selection like in the main table. However, the row you pick in this table will set the focus and highlight in other views. Everything else applies to the selection table.

Feature-based coloring in table views.

Of course, feature-based coloring works with the table views. And it can give a pleasant display when combined with sorting rows by a feature column.

Exporting table data.

The data currently displayed in a table view can be exported to CSV. When a table window is active, select the menu item File > Export > Export to CSV. You will have to specify a saving location and a name. Only the data displayed in the currently visible table is saved and ordered as in the view. This means that if you call the command from a selection table, only the current selection will be saved.

The grapher views.

Besides the table view that allows you to inspect the data tabularly, Mastodon ships a second view that is more suitable for visualizing the data: the grapher view. You can open a new grapher view by selecting the menu item Window > New grapher or via the main window.

Plotting feature values.

The grapher view is a plotting tool that can be used to visualize the feature values of the data items. In the left area of the grapher view, you can select the feature values to plot. You need to select a feature for the X axis and a feature for the Y axis. After making this choice, you have to press the Plot button to trigger the painting of the plot. After that you can Zoom in and out and pan the plot with the mouse.

• Zoom in/out: in X axis: Shift + Mouse Wheel.

• Zoom in/out: in Y axis: Control + Mouse Wheel.

• Pan: Right Mouse Button and dragging the mouse.

You can use the tags like in the table views to visualize categories in the plotted data. You can also use the feature color modes in the plotted data to visualize a third dimension. Both options can be reached via the View > Coloring menu. Finally, there is an option to show a legend of the current coloring mode. You can toggle it on or off and set the location of this legend in the View > Colorbar menu.

The grapher view with spot radius as X axis and mean of channel 1 as Y axis. The link velocity is used to color code the points.

The grapher view with spot radius as X axis and velocity as Y axis. A tag set called lineage classification is used to color code the points.

Filtering items.

Additionally, you can set some filtering for the items to plot. You can choose between the following options:

Plot currently selected items

This option will plot only the items that are currently selected in the grapher view. This can be useful if you make a selection in the table or track scheme view, or the selection creator and want to visualize only the selected items in the grapher view.

The grapher view with spot radius as X axis and mean of channel 1 as Y axis. Only the points that are about to divide in the next time frame are shown.

Plot track(s) of selected items

This option will plot all tracks that contain the selected items. This can be useful if you want to visualize a feature value of a spot over time.

The grapher view with frame as X axis and radius as Y axis showing the tracks of the selected spots. Spots are color-coded with velocity. The plot allows seeing the change of the radius over time

Plot spots of (spatial) context

This option allows you to select either the full graph (i.e. all spots and links) or the spots that are currently visible in a selectable Big Data Viewer (BDV) view.

Spot center intensity as X axis and Spot radius as Y axis. Since the BigDataViewer is chosen as context, the Grapher updates while we are sliding through time in the BigDataViewer.

Keep current

The default option is to keep the current selection of items in the grapher view. After pressing the Plot button, the side panel falls back to this option.

Plotting edges.

You can visualize the edges between the spots of the graph. This makes particularly sense when you are plotting features over time, i.e. with the frame as X axis.

The grapher view with frame as X axis and radius as Y axis showing the tracks of the selected spots. Spots are color-coded with velocity. The plot allows seeing the change of the radius over time.

Selecting items.

Rectangle selection.

Press the left mouse button and drag the mouse to create a rectangle. Spots and links that are fully contained in the rectangle will be selected.

The grapher view with frame as X axis and radius as Y axis. Spots that are selected with the rectangle are also selected in track scheme view and the BDV view.

Lasso selection.

Press Control and the left mouse button and drag to create a lasso selection. Spots and links that are fully contained in the lasso will be selected.

The grapher view with spot ellipsoid aspect ratio as X axis and intensity as Y axis. Spots that are selected with the lasso are also selected in track scheme view and the BDV view.

Grapher stlye configuration.

You can configure the grapher style in the preference dialog. You find it in the menu File > Preferences. Besides default colors, you can configure the size of the points in the plot. There are two modes:

• Auto vertex size: This option will use large circles in sparse regions of the plot and small circles in dense regions.

• Fixed vertex size: This option will use the same size for all points in the plot. You can set the size in pixels.

• Draw Labels: This option will draw the labels of the points in the plot.