Manually editing tracks in Mastodon. TrackScheme.¶
One of the key feature of Mastodon, the implementation of which made our lives extremely hard and rich, is the ability to edit manually at any time any spot or link within an annotation that can contains billions of them, while retaining a good and pleasant response time from the software.
In this tutorial, we will show how to do just that, from existing tracking data, the one we generated in the previous section. We will use it as an opportunity to present TrackScheme , the track visualizer of Mastodon, as well as introduce several useful editing features such as the undo/redo mechanism. We will also introduce the various visual hints that help the user knowing what spot or link is currently edited or inspected, such as the focus, the highlight and the selection.
Manual editing is often not desirable, first because it is not objective which might be detrimental in situations where the performance of an algorithm is evaluated, or when an unknown motion characteristics is investigated. Second, because it does not look like a realistic solution when e.g. thousands of cells are to be followed over thousands of time-points. Regarding the latter however, several courageous individuals sacrificed their time, energy and sometimes sanity in doing so, for the sake of finally a scientific answer, or curating the results of an automated analysis, when there was no other working solutions. See for instance Wolff, Tinevez, Pietzsch et al, 2018 or McDole et al, 2018. If you are going this way, Mastodon aims at providing tools to do so, in the hopefully fastest and least painful way possible. Also, you can combine the fully automated approach of the previous chapter with manual editing to correct hopefully a few numbers of mistakes. Finally, there is also a semi-automated tracking tool, that we will introduce later.
This tutorial will start by presenting you the manual editing tools of Mastodon, as well as and the focus, highlight and selection tools. Because this catalogue of actions can be a bit dry and long to swallow we will in the last paragraph use the techniques presented here to remove and rebuild some tracks from the results of the last tutorial.
TrackScheme, the lineage view and editor.¶
First, load or retrieve the tracking data of the previous tutorial.
You should have a few hundreds of tracks.
To open a TrackScheme view window, press the
trackscheme button on the main window.
The TrackScheme view offers a special way of displaying tracks. If you are familiar with TrackMate, you will see that we brought the same kind of features here, but scaled to large data. You can think of TrackScheme as a workbench for tracks, where you will edit, cut, stitch and rename them. It displays a kind of “track map”, where a track is laid on a panel, arranged vertically over time, as a Parisian subway train map. Tracks are displayed hierarchically, discarding the spatial location of each spot. Each track is laid out going through time from top to bottom. One horizontal line corresponds to a single time-point in the movie. One vertical column corresponds to a single track, that is all the spots that are connected by links over time, including divisions and merges. It is a great tool inp artiular when studying and editing cell lineages.
When opened, the view in TrackScheme is scaled so that the full data is shown, all tracks over all time-points. Even on our small dataset, most of the tracks and spots appear as single lines or dense boxes. To see the details of each track, you need to zoom in and navigate around the data. The navigation actions their mappings are listed in the TrackScheme keyboard shortcuts table.
Try to zoom in until you can see the label of a few spots. TrackScheme implements adaptive level of details depending on the zoom level, so that we can accommodate plotting a large amount of data without compromising the responsiveness of Mastodon. On the finer level of details, spots are plotted as circles, with the spot label shown. As you zoom out, they becomes just empty circles, then points, then they disappear to only show the track as a line. When the zoom level is so low that several tracks coalesce, they are drawn as a gray box.
The focus and the spot labels.¶
You probably noticed that some spots are painted differently from their neighbors. Mastodon manages three special collections of spots and links to facilitate making sense of the data across views:
the selection, painted in green, that manages a classical selection of spots and links;
the highlight, used to highlight the spot or link currently under the mouse;
the focus, particularly used in TrackScheme, to indicate what spot is currently focused on by the keyboard interaction.
We will first present the focus.
Moving the focus.¶
The spot that has the focus is painted with a dashed contour (see figure below). Only one spot can have the focus and it is set by mouse or keyboard interaction. Click on one spot in TrackScheme when fully zoomed, and move across track and time with the arrow cursor keys. If the focus reaches the border of the window, the view will be moved to follow it. You can think of the focus as the caret in a text editor. It is meant to facilitate keyboard interaction. See the focus navigation keyboard shortcuts.
You can also jumps across branches.
A branch in a track is a linear section of the track between divisions, fusions or the track end or start.
Alt ↑ and
Alt ↓ will move the focus to the branch start and end.
In the case where you could navigate to several branches, for instance you are currently at a cell division and you can go the end of either daughter cells branches, you can navigate to one or the other with or
Alt ↓ or
Control Alt ↓.
Editing the spot labels.¶
By the way, the focus is used to rename individual spots.
Zoom in so that we can see the label of the spots and move the focus to a spot.
A small editing box appears inside the spot and lets you change its label.
By default the spot label display the spot ID. If you edit the label then it shows the new label you entered. Editing the label of a spot does not affect its ID or any other properties. The spot label is just a convenient text field that you can use to annotate cells and search them later. Only the spots have a label. The links don’t.
The order of tracks in TrackScheme.¶
The spot labels also control how the tracks are ordered in TrackScheme. We said before that in TrackScheme all the spatial information is discarded. However the tracks are laid out in a deterministic order. This order is set by the label of of a track.
There is no special structure to follow individual tracks in Mastodon.
For this, we simply use the first spot of a track.
So when we speak of the label of a track, we simply means the label of the first spot (in
time) in the track.
The tracks are arranged from left to right following the alphanumerical order of the track labels.
So you can change the tracks arrangement by editing theirs first spot’s label.
For instance a track named
A will be laid out to the left of a track named
B, and a track named
D9 will be put to the left of a track named
If you change the track labels now, the tracks will not move immediately in TrackScheme. For the new arrangement to happen, you need to either open another TrackScheme window, or to edit the data, which we will see soon.
The focus in BDV views.¶
The focus also works in the BDV views. In these views, the spot that has the focus is also painted with a thick dashed circle. Parenthetically, notice that the focus is shared across all opened views. If you have a TrackScheme view and a BDV view opened, and that they both contain a display of the same spot, setting the focus in one view will update this very spot display in the other views.
When you are in a BDV view and have set the focus, you can also navigate with the arrow keys. After clicking inside a cell, you can navigate in time and follow a cell lineage through the movie with
If you reach the border of the view or if the cell moves in Z, the view will be translated to keep the cell in focus.
However, you cannot edit the spot labels in a BDV view nor jump from one track to another with the arrow keys.
Navigating across branch with
Alt ← and
Alt → however works.
Synchronizing several views together.¶
You probably noticed that both BDV and views have a toolbar on the of the view itself.
This toolbar can be hidden and shown by a press of the
In both view types, the toolbar contains 3 gray locks , that are used to link several views together for navigation.
For instance, if you click on the lock 1 on a BDV view and the same lock on a view, they will be synchronized.
If you move with the focus in the view, the BDV view will be translated to show the focused spot.
Double click on a spot in any view, all views in sync will translate to show the spot.
This is very handy to navigate around in TrackScheme, for instance following a cell over time while the BDV view displays where it is in the sample.
You can even combine several BDV views in sync at different magnification to have both an overview of the cell position in the sample and a close view of the cell itself.
The highlight is the second visual hint we will introduce. You probably already noticed it and used it: the display of spots and links that are just below the mouse cursor changes. Highlighted spots and links are painted with a thick continuous line. To highlight a spot or link you just have to lay the mouse over it. As for the focus, the highlight is common to all views, and if you highlight a spot or a link in a view, its display is changed in all the views that show it.
The selection is the third visual cue of Mastodon and probably the most important one. It behaves and has the role of a classical selection tool you would find in another software that manages a collection of data items. It is meant to select a subset of spots and links and apply operations to this subset. Contrary to the focus and the highlight, any number of spots and links can be put in the selection.
The selection in TrackScheme is built in a classical way.
Click and drag from an empty part of the view.
A red box appears to let you select an area of the view.
Release it when it contains the part of the tracks you want to select, and selected items should now be painted in green.
All spots and tracks in the selection box are put in the selection, even if the zoom level is too small for them to be painted.
Control A selects the whole data.
There are variants to select only all the spots (
Control Alt A) or only all the links (
Control Shift A).
Shift click on a spot or a link to toggle it in the selection.
To clear the selection, click on an empty part of the TrackScheme view.
In you can also add spots - but not links - to the selection using the focus.
For instance, if you combine navigating with the focus while holding the
Shift key, spots that you navigate to will be added to the selection.
Shift Alt ↓ will add all the
spots from the focused spot to the end of the branch it belongs to.
Finally there are commands to add a full track from the currently focused spot.
Shift space will select the whole track of the focused spot.
Shift ⇞ selects all the spots and links in the track that are forward in time relative
to the focused spot, and
Shift ⇟ does the converse backward in time.
The selection is shown in BDV views as red or magenta spots and links by default.
In BDV views you can only edit the selection via
The interaction with the focus also work as described above.
These default key-bindings are recapitulated in the table of shortcuts for the selection actions.
Moving spots around.¶
To move an existing spot we use a similar interaction with the spots.
In a BDV view, place the mouse cursor inside the spot you want to move, then press and hold
While you hold
space pressed the spot will move where you move the mouse.
space key at the desired location.
The undo/redo mechanism.¶
One of the most useful actions is undoubtedly undo and redo.
To undo just press
Control Z and to redo use
Control Shift Z .
There is no limit to the number of undos you can do.
And everything can be undone in Mastodon, even tag definitions.
In our humble opinion, this is one of the nice reasons for Mastodon to exits: a tracking and lineaging software that aims at combining automated and manual approaches, in a very efficient way..
Putting things in practice.¶
Shift+K3. Backtracking a cell with
Make sure you have one BDV view and one view open, and that both are linked using the same lock.
In TrackScheme, select one of the spot or link that belongs to one of the long tracks; the ones that extend from the first time-point to the last.
You selected the whole track to which the spot or link you selected belong to.
The track has been removed.
Let’s remove a couple of other tracks another way.
Click and drag in to draw a selection box around some long tracks.
Our model should now lack several good tracks we will try to put back manually.
For clarity, also remove all the tracks on the right part of the view, that do not start at the first time-point.
In the BDV view, go to the first time-point, and look for a cell whose spot has been removed.
Pan the view (right-click and drag), adjust the zoom (
Control Shift Mouse-wheel) and the Z position (
Mouse-wheel with or without
Shift ) so that the cell is well in view.
Then put the mouse cursor over it and press
A spot has been added on the cell.
The fine adjustment of the spot position in 3D can be made easier with a second BDV view.
Open another BDV view, zoom and rotate it so that it is aligned with the XZ plane by pressing
Add it to the lock group by toggling the right lock in the toolbar.
Go back to the first BDV view, and align it with the XY plane by pressing
You must now find the spot we just added.
If you cannot find it in any of the two BDV view, look for it in TrackScheme.
It will be in the rightmost column at the first line, since we added the spot to the first time-point.
Double-clicking on it in TrackScheme will center the two BDV views on it.
Now that the two BDV views are centered on the spot, you can adjust its XY and Z positions over the cell using the two views.
In any of the two BDV views, put the mouse cursor, then press and hold
Move the spot the desired location and release
The spot might not have the right diameter to encompass the cell.
Change its radius by pressing
Shift Q and
Shift E (without
Shift for finer adjustments) until you are happy with it.
You must now do it for the same cell in the subsequent time-points. Press in a BDV view to move to the next time-point. Again, put the mouse cursor roughly at the center of the cell and press . Adjust its position with and move to the next time-point.
We have to repeat this for all the time-points of the movie.
This might be tedious but does not have to be too long.
When I need to do this I adopt a posture similar to what I have when I play PC video-games: the right hand on the mouse, the left hand over the left side of the keyboard, over the
Shift key can be accessed with the little finger, the
space key with the thumb.
M keys to navigate over time are accessible with the left thumb also.
We created a bunch of spots, but they are not linked. Move to the TrackScheme view. You will find the spots you last created there always at rightmost part of the view. Since they are not linked, they should appear each in their separate column, in a stairway manner. Select them all by dragging a selection box around them and press . They should now link together into a new track. With one of the spot of this newly created track selected, press to jump to the first spot in the track. Then press to edit its name into something like My first track.
We can create a track directly. Let’s do this by backtracking a cell from the last time-point to the first. In a BDV view, move to the last time-point, and place the mouse cursor over an annotated cell. Create a spot over it by pressing and center and resize it until satisfaction. Now keep the mouse cursor inside this spot, and press and hold the key. The BDV view moves to the previous time-point and creates a new spot there, already linked to the other one. By repeating this you can create a track that will backtrack the cell until it appears in the movie.
We have now restored two of the tracks we removed at the beginning of this paragraph.
Hopefully now you feel enough at ease to do these manipulations quickly, efficiently and without too much hassle.
But one of the concluding beauty of this tutorial is that you can restore the data as we had it at the beginning by roughly 100 presses of the
Control Z keys.