ETH-FDS seminar series

More information about ETH Foundations of Data Science can be found here

Modal title

Modal content

Please subscribe here if you would you like to be notified about these events via e-mail. Moreover you can also subscribe to the iCal/ics Calender.

Spring Semester 2022

Date / Time

Speaker

Title

Location

24 March 2022
16:15-17:15

Emmanuel Abbé
EPF Lausanne

Details

ETH-FDS seminar

Title	Towards a characterization of when neural networks can learn
Speaker, Affiliation	Emmanuel Abbé, EPF Lausanne
Date, Time	24 March 2022, 16:15-17:15
Location	HG D 1.1
Abstract	It is currently known how to characterize functions that neural networks can learn with SGD for two extremal parametrizations: neural networks in the linear/kernel regime, and neural networks with no structural constraints. However, for the main parametrization of interest ---non-linear but regular networks--- no tight characterization has yet been achieved, despite significant developments. In this talk, we take a step in this direction by considering depth-2 neural networks trained by SGD in the mean-field regime. We consider functions on binary inputs that depend on a latent low-dimensional subspace, since this provides a challenging framework for linear models (curse of dimensionality) but not for neural networks that routinely tackle high-dimensional data. Accordingly, we study learning of such functions with a linear sample complexity. In this setting, we establish a necessary and nearly sufficient condition for learning, i.e., the merged-staircase property (MSP). Joint work with E. Boix (MIT) and T. Misiakiewicz (Stanford)
Documents	Video E. Abbé - ETH-FDS talk on 24 March 2022file_download

Towards a characterization of when neural networks can learnread_more

HG D 1.1

31 March 2022
16:15-17:15

Tom Goldstein
University of Maryland

Details

ETH-FDS seminar

Title	End-to-end algorithm synthesis with "thinking" networks
Speaker, Affiliation	Tom Goldstein, University of Maryland
Date, Time	31 March 2022, 16:15-17:15
Location	HG D 1.2
Abstract	This talk will have two parts. In the first half of the talk, I'll survey the basics of adversarial machine learning, and discuss whether adversarial attacks and dataset poisoning can scale up to work on industrial systems. I'll also present applications where adversarial methods provide benefits for domain shift robustness, dataset privacy, and data augmentation. In the second half of the talk, I'll present my recent work on "thinking systems." These systems use recurrent networks to emulate a human-like thinking process, in which problems are represented in memory and then iteratively manipulated and simplified over time until a solution to a problem is found. When these models are trained only on "easy" problem instances, they can then solve "hard" problem instances without having ever seen one, provided the model is allowed the "think" for longer at test time. Bio: Tom Goldstein is the Perotto Associate Professor of Computer Science at the University of Maryland. His research lies at the intersection of machine learning and optimization, and targets applications in computer vision and signal processing. Before joining the faculty at Maryland, Tom completed his PhD in Mathematics at UCLA, and was a research scientist at Rice University and Stanford University. Professor Goldstein has been the recipient of several awards, including SIAM’s DiPrima Prize, a DARPA Young Faculty Award, a JP Morgan Faculty award, and a Sloan Fellowship.
Documents	Video T. Goldstein - ETH-FDS talk on 31 March 2022file_download

End-to-end algorithm synthesis with "thinking" networksread_more

HG D 1.2

12 May 2022
17:15-18:15

Song Mei
UC Berkeley

Details

ETH-FDS seminar

Title	A theoretical framework of convolutional kernels on image tasks
Speaker, Affiliation	Song Mei, UC Berkeley
Date, Time	12 May 2022, 17:15-18:15
Location	Zoom
Abstract	Recent empirical work has shown that hierarchical convolutional kernels inspired by convolutional neural networks (CNNs) significantly improve the performance of kernel methods in image classification tasks. A widely accepted explanation for the success of these architectures is that they encode hypothesis classes that are suitable for natural images. However, understanding the precise interplay between approximation and generalization in convolutional architectures remains a challenge. In this talk, we consider the stylized setting of covariates (image pixels), and fully characterize the RKHS of kernels composed of single layers of convolution, pooling, and downsampling operations. We then study the gain in sample efficiency of kernel methods using these kernels over standard inner-product kernels. In particular, we show that 1) the convolution layer breaks the curse of dimensionality by restricting the RKHS to `local' functions; 2) global average pooling enforces the learned function to be translation invariant; 3) local pooling biases learning towards low-frequency functions. Notably, our results quantify how choosing an architecture adapted to the target function leads to a large improvement in the sample complexity.
Documents	Video Song Mei - ETH-FDS talk on 12 May 2022file_download

A theoretical framework of convolutional kernels on image tasksread_more

Zoom

24 May 2022
16:15-17:15

Daniel Roy
University of Toronto

Details

ETH-FDS seminar

Title	Replacing assumptions in statistical analysis with adaptivity
Speaker, Affiliation	Daniel Roy, University of Toronto
Date, Time	24 May 2022, 16:15-17:15
Location	HG D 1.2
Abstract	In this talk, I will advocate for rethinking the role of key data assumptions in statistical analysis. In place of assumptions, I will suggest we aim for adaptivity, much like in nonparametric regression, where we seek methods that adapt to, say, the smoothness of the unknown regression function. Not all assumptions are created equal, however. I'll discuss two examples where dropping key assumptions forces us to reconsider also what promises we make to users about our statistical methods. In the first example, we drop the i.i.d. assumption when performing sequential prediction. In the second, we drop the no-unmeasured-confounders assumption when attempting to identify the best intervention. In both cases, we must redefine our goal to arrive at a well-defined problem. This talk will be based on results described in https://arxiv.org/abs/2007.06552, https://arxiv.org/abs/2110.14804, and https://arxiv.org/abs/2202.05100, joint work with Blair Bilodeau, Nicolò Campolongo, Jeffrey Negrea, Francesco Orabona, and Linbo Wang.
Documents	Video Daniel Roy - ETH-FDS talk on 24 May 2022file_download

Replacing assumptions in statistical analysis with adaptivityread_more

HG D 1.2

Archive: SS 25 AS 24 SS 24 AS 23 SS 23 AS 22 SS 22 AS 21 SS 21 AS 20 SS 20 AS 19 SS 19