Statlearn:
conferences on challenging problems
in statistical learning

Thursday 03/04

Welcome & registration - 8:30

Conf#1 (9:00-10:30) - Gilles Louppe, Professor at the University of Liège, Belgium: Inverting Scientific Images With Score-Based Generative Models

Inverse problems are at the core of many scientific imaging tasks, where the goal is to recover latent states from noisy observations. In this talk, we will explore how diffusion models can be repurposed to address these challenges. We will briefly recap the theory of score-based generative models and then explain how posterior sampling can be performed in zero-shot settings without requiring pre-wired neural networks. Applications of this approach will be demonstrated across various scientific imaging tasks, including galaxy reconstruction, trajectory recovery in dynamical systems, and accelerated MRI. We will also present a novel approach for learning diffusion priors directly from noisy observations using expectation-maximization, eliminating the need for paired data. The talk will conclude with a discussion of current challenges in scalability and quality assessment.

Conf#2 (11:00-12:30) -  Yingzhen Li - Associate Professor in Machine Learning at the Department of Computing at Imperial College, London, UK: Sequential Latent Variable Models

This talk is about latent variable models (LVMs) but applied to sequences where learning the underlying dynamics is of primal interest. In the first half of the talk, I will discuss sequential VAEs as a canonical example for sequential LVMs and show various possible approaches, including the concept of filtering and smoothing posteriors. Then in the second half of the talk, I will discuss some of our research in this area relevant to the basics in the first part. In particular, I will discuss (1) how filtering/smoothing can be used in Gaussian Process VAEs where the latent dynamics is continuous-time; (2) how to build a recurrent memory based on dynamical system for Gaussian processes; and if time permits, (3) an alternative view of diffusion model’s success story from sequence modelling viewpoint.

Conf#3 (14:00-14:45) - Frederic Précioso, professor at INRIA Nice (MAASAI), France, Biases in LLM

In this short presentation we will try to cover several aspects on the question of Fairness of LLMs. We will first expose the different kind of biases these models are submitted to, we will also see propositions to mesure biases in these new textual data representations, and we will question if these biases can be corrected and how.

Conf#4 (15:00-16:30) - Aude Sportisse, CNRS researcher, Laboratoire d'Informatique de Grenoble (APTIKAL team), Grenoble, France: Generative Methods for Missing Data: From PCA to Deep Learning

Handling missing data is a crucial skill in machine learning and data science, especially when working with real-world datasets. In this talk, we will explore the literature on missing data and discuss key methods for imputation, estimation, and prediction. Our main focus will be on generative approaches, ranging from PCA to deep learning-based techniques. To put theory into practice, a hands-on session will help you overcome your fear of missing values!

Conf#5 (16:45-17h30) - Benjamin Billot, researcher at INRIA Nice (EPIONE), France, Bridging generative models and Neural Networks for domain-agnostic segmentation of brain MRI

Despite advances in data augmentation and domain adaptation, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain MRI scans, CNNs are highly sensitive to changes in resolution and contrast. Here we introduce SynthSeg, the first CNN for domain-agnostic segmentation of brain MRI scans without any retraining or fine-tuning. Specifically, SynthSeg is trained with synthetic data sampled from a parametric generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the parameters of the generative model, such that we obtain synthetic brain images of random contrast and resolution. By being presented with those images of random aspect, the network cannot focus on a specific domain, and has to learn domain-agnostic features, such that it can then segment brain scans of any domain without having to be retrained or fine-tuned. We demonstrate SynthSeg on 15,000 brain scans of heterogeneous modalities and resolutions, where it exhibits unparalleled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation.

Friday 04/04

Conf#6 (9:00-10:30) - Johannes Brandstetter, Assistant Professor JKU Linz, Austria : Latent space (generative) models in recent scientific breakthroughs

In the era of scaling and generative modeling – more specifically latent diffusion models -- one gets notoriously confronted with the question of where we stand with applicability of such powerful techniques within scientific or engineering domains. The discussion starts by reiterating on recent triumphs in weather and climate modeling, making connections to computer vision, physics-informed learning and neural operators. Secondly, we discuss challenges and conceptual barriers which need to be overcome for the next wave of disruption in science and engineering. We showcase recent breakthroughs in multi-physics modeling, molecular dynamics, computational fluid dynamics, nuclear fusion and related fields.

Conf#7  (11:00-12:30) - Anna Korba, assistant professor at ENSAE/ CREST, Paris, France:  Implicit Diffusion: Efficient Optimization through Stochastic Sampling

We present a new algorithm to optimize distributions defined implicitly by parameterized stochastic diffusions. Doing so allows us to modify the outcome distribution of sampling processes by optimizing over their parameters. We introduce a general framework for first-order optimization of these processes, that performs jointly, in a single loop, optimization and sampling steps. This approach is inspired by recent advances in bilevel optimization and automatic implicit differentiation, leveraging the point of view of sampling as optimization over the space of probability distributions. We provide theoretical guarantees on the performance of our method, as well as experimental results demonstrating its effectiveness. We apply it to training energy-based models and finetuning denoising diffusions.

Conf#8  (14:00-15:30) - Antonio Vergari, Associate Professor University of Edinburgh, UK: Subtractive Mixture Models: Representation, Learning and Inference

Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enables us to represent tensorized mixtures and generalize several other subtractive models such as positive semi-definite kernel models and Born machines.  PCs also enable several applications of reliable neuro-symbolic AI, from controlled generation with LLMs to knowledge-compliant concept bottleneck models.  We theoretically prove that the class of sum of squared circuits allowing subtractions can be exponentially more expressive than traditional additive mixtures. We empirically show this increased expressiveness on a series of real-world distribution estimation tasks and discuss which inference scenarios are tractable with this new class of circuits. Finally, I will talk about how to use these subtractive mixtures for approximate inference when plugged in monte carlo and importance sampling estimators.

Closing words - 15h30