Multiplicative amplification in deep neural networks

by Bernhard Mehlig (University of Gothenburg)

Wednesday 15 Jan 2025, 15:00 → 17:00 Europe/Stockholm

Albano 2: C2207 - Auditorium 4 (80 seats) (Albano Building 2)

Auditorium 4 (below the Proviant canteen) in House 2. 

Please follow the Campus map or Google map to find the auditorium.

Or via Zoom link
 

Fika will be served at 14:30.

 

Abstract: 

After giving an introduction to deep learning, I will discuss how deep networks learn. This can be analysed and understood, in part, using concepts from dynamical-systems theory [1]. For deep neural networks, the maximal finite-time Lyapunov exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems such as turbulent flow. Ridges of large positive exponents divide input space into different regions that the network associates with different classes in a classification task. The ridges visualise the geometry that deep networks construct in input space, and help to quantify how the learning depends on the network depth and width [2].
[1] Bernhard Mehlig, Machine Learning with neural networks, Cambridge University Press (2021).
[2] Storm, Linander, Bec, Gustavsson & Mehlig, Finite-time Lyapunov exponents of deep neural networks, Phys. Rev. Lett. 132 (2024) 057301.