Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality.

First Authors Ulrik Günther
Authors Ulrik Günther, Kyle Harrington, Raimund Dachselt, Ivo F. Sbalzarini
Corresponding Authors Ivo F. Sbalzarini
Last Authors Ivo F. Sbalzarini
Conference Proceedings Volume Title Computer vision - ECCV 2020 workshops : Glasgow, UK, August 23-28, 2020 : proceedings : Part 1
Series Title (Lecture notes in computer science ; 12535)
Conference Name 16th european conference on COMPUTER VISION 23-28 August 2020
Conference Location online
Conference Start Date 2020-08-23
Conference End Date 2020-08-28
Chapter Number
Publisher Springer International Publishing
Conference Proceedings Editors Adrien Bartoli
ISBN 978-3-030-66414-5
First Page 280
Last Page 297
Print Publication Date 2020-08-28
Online Publication Date 2020-08-28
Abstract We present Bionic Tracking, a novel method for solving biological cell tracking problems with eye tracking in virtual reality using commodity hardware. Using gaze data, and especially smooth pursuit eye movements, we are able to track cells in time series of 3D volumetric datasets. The problem of tracking cells is ubiquitous in developmental biology, where large volumetric microscopy datasets are acquired on a daily basis, often comprising hundreds or thousands of time points that span hours or days. The image data, however, is only a means to an end, and scientists are often interested in the reconstruction of cell trajectories and cell lineage trees. Reliably tracking cells in crowded three-dimensional space over many time points remains an open problem, and many current approaches rely on tedious manual annotation or curation. In the Bionic Tracking approach, we substitute the usual 2D point-and-click interface for annotation or curation with eye tracking in a virtual reality headset, where users follow cells with their eyes in 3D space in order to track them. We detail the interaction design of our approach and explain the graph-based algorithm used to connect different time points, also taking occlusion and user distraction into account. We demonstrate Bionic Tracking using examples from two different biological datasets. Finally, we report on a user study with seven cell tracking experts, highlighting the benefits and limitations of Bionic Tracking compared to point-and-click interfaces.
PDF Günther_2020_7862.pdf (6.7 MB)
Cover Image
Affiliated With Sbalzarini, CSBD
Selected By
Acknowledged Services
Publication Status Published
Edoc Link
Sfx Link
DOI 10.1007/978-3-030-66415-2
PubMed ID
WebOfScience Link
Alternative Full Text URL
Display Publisher Download Only false
Visible On MPI-CBG Website true
PDF Downloadable true
Created By sbalzari
Added Date 2020-12-09
Last Edited By herbst
Last Edited Date 2022-02-18 15:43:47.053
Library ID 7862
Document ID
Entry Complete true
eDoc Compliant true
Include in Edoc Report true
In Pure false
Ready for eDoc Export false
Author Affiliations Complete false
Project Name
Project URL
Grant ID
Funding Programme
Funding Organisation