Toward a practical theory of deep learning: feature learning in deep neural networks and backpropagation-free algorithms that learn features - Stream

Remarkable recent advances in deep neural networks are rapidly changing science and society.
Never before had a technology been deployed so widely and so quickly with so little understanding of its fundamentals. I will argue that developing a fundamental mathematical theory of deep learning is necessary for a successful AI transition and, furthermore, that such a theory may well be within reach. I will discuss what a theory might look like and some of its ingredients that we already have available.

In particular, I will discuss how deep neural networks of various architectures learn features and how the lessons of deep learning can be incorporated in non-backpropagation-based algorithms that we call Recursive Feature Machines. I will provide a number of experimental results on different types of data, including texts and images, as well as some connections to classical statistical methods, such as Iteratively Reweighted Least Squares.

Speaker: Mikhail Belkin, UC San Diego

See weblink for streaming link, or attend stream presentation on campus