How can deep learning enhance microscopy?

by Giovanni Volpe

Thursday 30 Jan 2025, 15:15 → 16:15 Europe/Stockholm

Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we have introduced a software, currently at version DeepTrack 2.2, to design, train and validate deep-learning solutions for digital microscopy.

[1]   S. Helgadottir, A. Argun, G. Volpe. Digital video microscopy enhanced by deep learning. Optica 6, 506 (2019).
[2]   B. Midtvedt, S. Helgadottir, A. Argun, J. Pineda, D. Midtvedt, G. Volpe. Quantitative digital microscopy with deep learning. Applied Physics Reviews 8, 011310 (2021).
[3]   S. Helgadottir, B. Midtvedt, J. Pineda, A. Sabirsh, C. B. Adiels, S. Romeo, D. Midtvedt, G. Volpe. Extracting quantitative biological information from brightfield cell images using deep learning. Biophysics Reviews 2, 031401 (2021).
[4]   J. Pineda, B. Midtvedt, H. Bachimanchi, S. Noé, D. Midtvedt, G. Volpe, C. Manzo. Geometric deep learning reveals the spatiotemporal fingerprint of microscopic motion. Nature Machine Intelligence 5, 71–82 (2023).
[5]   B. Midtvedt, J. Pineda, F. Skärberg, E. Olsén, H. Bachimanchi, E. Wésen, E. K. Esbjörner, E. Selander, F. Höök, D. Midtvedt, G. Volpe. Single-shot self-supervised object detection in microscopy. Nature Communications 13, 7492 (2022).
[6]   H. Bachimanchi, B. Midtvedt, D. Midtvedt, E. Selander, G. Volpe. Microplankton life histories revealed by holographic microscopy and deep learning. eLife 11, e79760 (2022).
[7]   B. Špačková, H. K. Moberg, J. Fritzsche, J. Tenghamn, G. Sjösten, H. Šípová-Jungová, D. Albinsson, Q. Lubart, D. van Leeuwen, F. Westerlund, D. Midtvedt, E. K. Esbjörner, M. Käll, G. Volpe, C. Langhammer. Label-free nanofluidic scattering microscopy of size and mass of single diffusing molecules and nanoparticles. Nature Methods 19, 751 (2022).
[8]   B. Midtvedt, J. Pineda, H. K. Moberg, H. Bachimanchi, J. B. Pereira, C. Manzo, G. Volpe. Deep Learning Crash Course. (No Starch Press, San Francisco (CA), 2025).

About the speaker:

Giovanni Volpe is Full Professor at the Physics Department of the University of Gothenburg University, where he leads the Soft Matter Lab (http://softmatterlab.org). His research interests include soft matter, optical trapping and manipulation, statistical mechanics, brain connectivity, and machine learning. He has authored more than 100 articles and reviews on soft matter, statistical physics, optics, physics of complex systems, brain network analysis, and machine learning. He co-authored the books "Optical Tweezers: Principles and Applications" (Cambridge University Press, 2015) and “Simulation of Complex Systems” (IOP Press, 2021). He has developed several software packages for optical tweezers (OTS — Optical Tweezers Software), brain connectivity (BRAPH—Brain Analysis Using Graph Theory), and microscopy enhanced by deep learning (DeepTrack). He's now co-author of the book "Deep Learning Crash Course" (No Starch Press, 2025).