2024 CMS Winter Meeting
Vancouver/Richmond, Nov 29 - Dec 2, 2024
Mathematics of Machine Learning
Org: Ben Adcock (Simon Fraser University), Elina Robeva (UBC) and Giang Tran (University of Waterloo)
Org: Ben Adcock (Simon Fraser University), Elina Robeva (UBC) and Giang Tran (University of Waterloo)
- RICARDO BAPTISTA, California Institute of Technology
- BENJAMIN BLOEM-REDDY, University of British Columbia
- WUYANG CHEN, Simon Fraser University
- HANS DE STERCK, University of Waterloo
- NICK HARVEY, University of British Columbia
- MIRANDA HOLMES-CERFON, University of British Columbia
- NIKOLA KOVACHKI, Nvidia
- SAMUEL LANTHALER, California Institute of Technology
- MATHIAS LECUYER, University of British Columbia
- KE LI, Simon Fraser University
- WENLONG MOU, University of Toronto
- RAHUL PARHI, University of California, San Diego
Deep Learning Meets Sparse Regularization [PDF]
- BENJAMIN BLOEM-REDDY, University of British Columbia
-
Deep learning has been wildly successful in practice and most state-of-the-art artificial intelligence systems are based on neural networks. Lacking, however, is a rigorous mathematical theory that adequately explains the amazing performance of deep neural networks. In this talk, I present a new mathematical framework that provides the beginning of a deeper understanding of deep learning. This framework precisely characterizes the functional properties of trained neural networks. The key mathematical tools which support this framework include transform-domain sparse regularization, the Radon transform of computed tomography, and approximation theory. This framework explains the effect of weight decay regularization in neural network training, the importance of skip connections and low-rank weight matrices in network architectures, the role of sparsity in neural networks, and explains why neural networks can perform well in high-dimensional problems.
- DANICA SUTHERLAND, University of British Columbia
- CHRISTOS THRAMPOULIDIS, University of British Columbia
- SHARAN VASWANI, Simon Fraser University
- ANDREW WARREN, University of British Columbia
- YIMING XU, University of Waterloo
- OZGUR YILMAZ, University of British Columbia
- CHRISTOS THRAMPOULIDIS, University of British Columbia