Prè-requis
Les bases d’un M1 de mathématiques appliquées en probabilités, statistiques et optimisation.
Objectif du cours
The goal of this course is to give an overview on existing methods for generative modeling with applications to images. The first part of the course will focus on modern image models such as Variational AutoEncoders (VAEs), Generative Adversarial Networks (GAN) and Normalizing Flows (NFs). Applications of these models for conditional image generation will be discussed. The rest of the course will be dedicated to the study of a new contender in generative modeling called diffusion models or Score-Based Generative Models (SGM). We will study the mathematical aspects (Time reversal of stochastic processes, links with Regularized Optimal Transport and control) and practical aspects of these algorithms. In particular we will introduce the core concepts of practical implementations like DDIM (Denoising Implicit Diffusion Models), LDM (Latent Diffusion Models) and their application to inverse problem and text-to-image synthesis through the lens of StableDiffusion.
Classes will be given in French but slides will be written in English.
Organisation des séances
9 séances de 3h
Mode de validation
1 Devoir maison obligatoire à mi-cours
1 Projet avec étude d’un article à la fin du cours
Références
Generative Adversarial Networks (Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, 2014)
Auto-Encoding Variational Bayes (Kingma and Welling, 2014)
An Introduction to Variational Autoencoders (Kingma and Welling, 2019)
Wasserstein GAN (Martin Arjovsky, Soumith Chintala, Léon Bottou, 2017)
A style-based generator architecture for generative adversarial networks (Karras, T., Laine, S., and Aila, T., 2019)
Image-To-Image Translation With Conditional Adversarial Networks (Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros, 2019)
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi, 2017)
Generative Modeling by Estimating Gradients of the Data Distribution (Yang Song, Stefano Ermon, 2019)
Score-Based Generative Modeling through Stochastic Differential Equations (Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole, 2021)
Denoising Diffusion Probabilistic Models (Jonathan Ho, Ajay Jain, Pieter Abbeel, 2020)
Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling (Valentin De Bortoli, James Thornton, Jeremy Heng, Arnaud Doucet, 2021)
Denoising Diffusion Implicit Models (Jiaming Song, Chenlin Meng, Stefano Ermon, 2020)
Score-based Generative Modeling in Latent Space (Arash Vahdat, Karsten Kreis, Jan Kautz, 2021)
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, Mohammad Norouzi, 2022)
High-Resolution Image Synthesis with Latent Diffusion Models (Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer, 2022)
Thèmes abordés
Variational Auto-Encoders
Generative Adversarial Networks
Conditional image generation
Score-based Generative Modelling
Regularized Optimal Transport
Stochastic Processes and their discretizations
Bruno Galerne
(Institut Denis Poisson, Université d'Orléans)
Valentin De Bortoli
(CNRS et DI ENS)