Young Data Science Researcher Seminar Zurich

Modal title

Modal content

Please subscribe here if you would you like to be notified about these events via e-mail. Moreover you can also subscribe to the iCal/ics Calender.

Spring Semester 2024

Date / Time

Speaker

Title

Location

22 April 2024
17:00-18:30

Max Simchowitz
MIT
Mohammad Lotfollahi
Cambridge University
Zhijing Jin
MPI and ETH Zurich
Discussant: Nicolai Meinshausen
ETH Zurich

Details

Young Data Science Researcher Seminar Zurich

Title	Joint webinar of the IMS New Researchers Group, Young Data Science Researcher Seminar Zürich, and the YoungStatS Project: Extrapolation to unseen domains: from theory to applications
Speaker, Affiliation	Max Simchowitz, MIT Mohammad Lotfollahi, Cambridge University Zhijing Jin, MPI and ETH Zurich Discussant: Nicolai Meinshausen, ETH Zurich
Date, Time	22 April 2024, 17:00-18:30
Location	Zoom
Abstract	1) Statistical Learning under Heterogeneous Distribution Shift, Max Simchowitz, MIT Abstract: What makes a trained predictor, e.g. neural network, more or less susceptible to performance degradation under distribution shift? In this talk, we will investigate a less well-studied factor: that of the statistical complexity of the individual features themselves. We will show that, for a very general class of predictors with a certain additive structure, empirical risk minimization is less sensitive to distribution shifts in "simple features" than "complex" ones, where simplicity/complexity are measured in terms of natural statistical quantities. We demonstrate that this arises because standard ERM learns the dependence on the "simpler" feature more quickly, whilst avoiding the risk of overfitting to more "complex" features. We will conclude by drawing connections to the orthogonal machine learning literature, and validating our theory on various experimental domains (even those in which the additivity assumption fails to hold). 2) Generative Machine Learning to Model Cellular Perturbations, Mohammad Lotfollahi, Cambridge University Abstract: The field of cellular biology has long sought to understand the intricate mechanisms that govern cellular responses to various perturbations, be they chemical, physical, or biological. Traditional experimental approaches, while invaluable, often face limitations in scalability and throughput, especially when exploring the vast combinatorial space of potential cellular states. Enter generative machine learning that has shown exceptional promise in modeling complex biological systems. This talk will highlight recent successes, address the challenges and limitations of current models, and discuss the future direction of this exciting interdisciplinary field. Through examples of practical applications, we will illustrate the transformative potential of generative ML in advancing our understanding of cellular perturbations and in shaping the future of biomedical research. 3) A Paradigm Shift in Addressing Distribution Shifts: Insights from Large Language Models, Zhijing Jin, MPI and ETH Zurich Abstract: Traditionally, the challenge of distribution shifts—where the training data distribution differs from the test data distribution—has been a central concern in statistical learning and model generalization. Traditional methods have primarily focused on techniques such as domain adaptation, and transfer learning. However, the rise of large language models (LLMs) such as ChatGPT has ushered in a novel empirical success, triggering a significant "shift" in problem formulation and approach for traditional distribution shift problems. In this talk, I will start with two formulations for LLMs: (1) the engineering heuristics aimed at transforming "out-of-distribution" (OOD) problems into "in-distribution" scenarios, which is further accompanied by (2) the hypothesized "emergence of intelligence" through massive scaling of data and model parameters, which challenges our traditional views on distribution shifts. I will sequentially examine these aspects, first by presenting behavioral tests of these models' generalization capabilities across unseen data, and then by conducting intrinsic checks to uncover the mechanisms LLMs learned. This talk seeks to provoke thoughts on several questions: Do the strategies of "making OOD problem IID" and facilitating the "emergence of intelligence" by scaling, truly stand up to scientific scrutiny? Furthermore, what do these developments imply for the field of statistical learning and the broader evolution of AI? Discussant: Nicolai Meinshausen, ETH Zurich
Documents	Registration for the recordingfile_download

Joint webinar of the IMS New Researchers Group, Young Data Science Researcher Seminar Zürich, and the YoungStatS Project: Extrapolation to unseen domains: from theory to applicationsread_more

Zoom

Archive: SS 24 AS 23 SS 23 AS 22 SS 22 AS 21 SS 21 AS 20 SS 20