Events for 12/06/2021 from all calendars

Colloquium - Jonathan Siegel

Time: 4:00PM - 5:00PM

Location: BLOC 117

Speaker: Jonathan Siegel, University of California, Los Angeles

Description:
Title: The Approximation Properties of Convex Hulls, Greedy Algorithms, and Applications to Neural Networks
Abstract: Given a collection of functions in a Banach space, typically called a dictionary in machine learning, we study the problem of determining the approximation properties of its convex hull. Specifically, we develop techniques for bounding the metric entropy and n-widths, which are fundamental quantities in approximation theory that control the limits of linear and non-linear approximation, of such a convex hull. Our results generalize existing methods by taking the smoothness of the dictionary into account, and in particular give sharp estimates for shallow neural networks. Consequences of these results include: the optimal approximation rates which can be attained for shallow neural networks, that shallow neural networks dramatically outperform linear methods of approximation, and indeed that shallow neural networks outperform all stable methods of approximation on the associated convex hull. Finally, we discuss greedy algorithms for non-linear approximation on such a convex hull. Specifically, we give optimal rates for the orthogonal greedy algorithm for dictionaries with small metric entropy, and for the pure greedy algorithm.