Approximating and learning smooth functions by ReLU neural networks

Abstract

In this talk, we will discuss some recent progresses on the approximation and learning theory of ReLU neural networks. We divide the approximation theory of neural networks into two parts according to the methods. In the first method, we approximate smooth functions by piecewise polynomials and then construct neural networks to approximate these piecewise polynomials. Using constructive approximation, one can derive optimal approximation rates in terms of the width and depth. In the second method, we consider the function space of shallow neural networks, which is called Barron space, and use random approximation method to derive approximation bounds in this space. By studying the relation between the Barron space and the smooth function spaces, we can characterize the approximation error of shallow neural networks by the width and certain norm of the weights. As an application, we will discuss how these approximation results can be used in nonparametric regression problems. In particular, we will show that least squares estimations based on deep or shallow neural networks can achieve minimax optimal rates of convergence for learning smooth function classes in various settings.

Biography

杨云斐，中山大学数学学院（珠海）副教授，2022年博士毕业于香港科技大学。研究兴趣包括机器学习、应用调和分析和逼近论。目前的研究工作主要集中于神经网络的逼近与泛化理论。相关研究成果发表于Journal of Machine Learning Research、Applied and Computational Harmonic Analysis、Constructive Approximation、NeurIPS等期刊和会议。