ETH-FDS Stiefel lectures

On this website you can find information about upcoming and past lectures.

Modal title

Modal content

Date / Time

Speaker

Title

Location

12 May 2025
17:15-18:15

Andrew Stuart
Caltech

Details

Title	Allowing Image And Text Data To Communicate (Attention Is Sometimes Useful)
Speaker, Affiliation	Andrew Stuart, Caltech
Date, Time	12 May 2025, 17:15-18:15
Location	HG F 30
Abstract	A fundamental problem in artificial intelligence is the question of how to simultaneously deploy data from different sources such as audio, image, text and video; such data is known as multimodal. In this talk I will focus on the canonical problem of aligning image and text data, and describe some of the mathematical ideas underlying the challenge of allowing them to communicate. I will describe the encoding of text and image in Euclidean spaces and describe contrastive learning methodology to identify and learn embeddings which align these two modalities. In so doing I will then describe the attention mechanism, a form of nonlinear transform that quantifies correlation in vector-valued sequences. Attention turns out to be useful beyond this specific context, and I will show how it may be used to design and learn maps between Banach spaces or between metric spaces of probability measures. The former is useful for accelerating MCMC, and the latter for nonlinear filtering.

Allowing Image And Text Data To Communicate (Attention Is Sometimes Useful)read_more

HG F 30

Notes: the highlighted event marks the next occurring event and if you want you can subscribe to the iCal/ics Calender.

Archive: SS 25 SS 24 AS 23 AS 22 SS 22 AS 19