Imaging in Paris Seminar

Parisian Seminar on the Mathematics of Imaging

Welcome to the website of the Parisian Seminar on the Mathematics of Imaging !

The goal of this seminar is to cover the fields of the mathematics of imaging in a very wide sense (including for instance signal processing, image processing, computer graphics, computer vision, various applications and connexion with statistics and machine learning). It is open to everyone. It takes place in Room 314 (Pierre Grisvard) at IHP on the first Tuesday of each month (the second Tuesday in November and January), from 14:00 to 16:00. Each seminar is composed of two presentations.

You can subscribe or unsubscribe to the mailing list of the seminar and to the agenda of the seminar.

Upcoming seminars

Click on the title to read the abstract.

Anton François (Centre Borelli, ENS Paris-Saclay)
June 4th 2024, 15h, room 314 (Pierre Grisvard).
Title: Medical Image Registration for Glioblastoma MRI Using Constrained Metamorphosis
Abstract: Registering three-dimensional medical images presents inherent challenges, exacerbated by topological variations between the images. Even the latest state-of-the-art methods often struggle to achieve realistic matching under these conditions. My research addresses these challenges by focusing on the registration of glioblastoma brain MRI, encompassing configurations such as healthy to cancerous states and post-operative scenarios.To tackle this task, we implemented Metamorphosis and LDDMM for 2D and 3D images using an object-oriented approach in PyTorch, with GPU acceleration and a semi-Lagrangian scheme. However, the classical Metamorphosis framework did not yield satisfactory results. To address this, we extended the framework to incorporate prior knowledge, which we term Constrained Metamorphosis. This extension allows for the addition of constraints to the registration problem by matching given priors, specifically a growing mask from a given segmentation and a field guiding deformation in a desired direction. We demonstrate the effectiveness of our approach through experiments on glioblastomas using BraTS datasets, comparing our results with state-of-the-art methods. In conclusion, I will discuss ongoing projects.

Shizhe Chen (WILLOW, INRIA Paris)
June 4th 2024, 14h, room 314 (Pierre Grisvard).
Title: Vision and language pre-training for robot navigation and manipulation
Abstract: Pre-training on large-scale datasets has significantly accelerated progress in various domains. However, collecting real robot data for pre-training remains expensive and lacks scalability. In this talk, I will demonstrate how we can leverage large-scale Internet data to enhance robot learning. Specifically, I will first present pre-training for vision-and-language navigation, where we take advantage of in-domain web image-captions and unlabeled 3D houses to improve models’ generalization capabilities in unseen environments. Next, I will delve into pre-training approaches for more complex robot manipulation which requires fine-grained visual perception and precise control. I will introduce a versatile pre-training framework based on web 3D objects to improve visual perception for robots.

Previous seminars of 2023-2024

The list of seminars prior to summer 2023 is available here.

Click on the title to read the abstract.

Éloi Tanguy (MAP5, Université Paris Cité)
May 21st 2024, TBA, room 201 (Maryam Mirzakhani).
Title: Properties of Discrete Sliced Wasserstein Losses
Abstract: The Sliced Wasserstein (SW) distance has become a common alternative to the Wasserstein distance for the comparison of probability measures. Widespread applications include image processing, domain adaptation and generative modelling, where it is typical to optimise some parameters in order to minimise SW, which in practice serves as a loss function between discrete probability measures. These optimisation problems all bear the same sub-problem, which is minimising the SW distance between two uniform discrete measures with the same amount of points as a function of the support (i.e. a matrix of data points) of one of the measures. We study the regularity and optimisation properties of this energy, as well as its Monte-Carlo approximation (estimating the expectation in SW using projection samples), as well as asymptotical and non-asymptotical statistical properties of the Monte-Carlo approximation. Finally, we show that in a certain sense, Stochastic Gradient Descent methods minimising these energies converge towards (Clarke) critical points, with an extension to Generative Neural Network training.

Théophile Cantelobre (SIERRA, INRIA Paris)
Apr 2nd 2024, 14h, room 314 (Pierre Grisvard).
Title: Measuring dissimilarity with diffeomorphism invariance
Abstract: Measures of similarity (or dissimilarity) are a key ingredient to many machine learning algorithms. We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces, which leverages the data's internal structure to be invariant to diffeomorphisms. We prove that DID enjoys properties which make it relevant for theoretical study and practical use. By representing each datum as a function, DID is defined as the solution to an optimization problem in a Reproducing Kernel Hilbert Space and can be expressed in closed-form. In practice, it can be efficiently approximated via Nyström sampling. Empirical experiments support the merits of DID. Article : Code :

Nicolas Chérel (Télécom Paris)
Apr 2nd 2024, 15h, room 314 (Pierre Grisvard).
Title: Diffusion-based image and video inpainting with internal learning
Abstract: Diffusion models are now the undisputed state-of-the-art for image generation and image restoration. However, they require large amounts of computational power for training and inference. We propose lightweight diffusion models for image inpainting that can be trained on a single image, or a few images. We develop a special training and inference strategy which significantly improves the results over our baseline. On images, we show that our approach competes with large state-of-the-art models in specific cases. Training a model on a single image is particularly relevant for image acquisition modality that differ from the RGB images of standard learning databases; for which no trained model is available. We have results in three different contexts: texture images, line drawing images, and materials BRDF, for which we achieve state-of-the-art results in terms of realism, with a computational load that is greatly reduced compared to concurrent methods. On videos, we present the first diffusion-based video inpainting approach. We show that our method is superior to other existing techniques for difficult situations such as dynamic textures and complex motion; other methods require supporting elements such as optical flow estimation, which limits their performance in the case of dynamic textures for example.

Yann Traonmilin (IOP Team, Institut de Mathématiques de Bordeaux)
Ma 5th 2024, 14h, room 314 (Pierre Grisvard).
Title: A few years of non-convex off-the-grid estimation
Abstract: In this talk, we focus on non-convex approaches for off-the-grid spike estimation. Centered around the study of basins of attraction of a non-convex functional, we explain how the study of recovery guarantees can be generally linked with the number of available measurements. With a general result on non-convex estimation of low-dimensional models, we show that the size of basins of attraction explicitly increases with respect to the number of measurements, with tight bounds for spikes recovery. These results lead to the conception of a fast algorithm for the recovery of many off-the-grid spikes: over-parametrized projected gradient descent (OP-PGD), showing promising results on realistic datasets. We also are able to give a theoretical partial control of the quality of continuous orthogonal matching pursuit without sliding which is the initialization procedure of OP-PGD.

Arnaud Quillent (Télécom Paris)
Ma 5th 2024, 15h, room 314 (Pierre Grisvard).
Title: Reconstruction par apprentissage profond en tomosynthèse du sein et estimation d'incertitude
Abstract: La tomosynthèse numérique du sein (Digital Breast Tomosynthesis – DBT) est une modalité d'imagerie médicale à rayons X qui permet la reconstruction de volumes 3D utilisés dans le cadre du dépistage du cancer du sein. Toutefois, à cause de diverses contraintes géométriques du système d'acquisition (anglé limité et vue éparse), des artéfacts apparaissent dans les reconstructions, ce qui dégrade grandement leur qualité et réduit leur résolution suivant l'axe vertical (détecteur-source). De ce fait, seuls les plans axiaux (parallèles au détecteur) sont actuellement exploitables par les radiologues pour poser leur diagnostic. Nous proposons donc une méthode de reconstruction par apprentissage profond pour la DBT, composé de deux étapes. Tout d'abord, une reconstruction itérative conventionnelle est obtenue à partir des projections acquises par le système, puis un réseau de neurones convolutifs est appliqué de façon à réduire les artéfacts présents dans l'image. Du fait de l'inexistence de données cliniques qui ne seraient pas impactées par les contraintes géométriques présentées plus haut et qui pourraient être utilisées comme vérité, nous employons une base de données composée de fantômes synthétiques pour entraîner notre modèle. Nous obtenons ainsi des résultats visuellement intéressants et améliorant grandement la qualité des volumes reconstruits. Le problème inverse de la reconstruction DBT étant très mal-posé du fait de l'angle limité et du peu de projections acquises, la quantité d'informations à extrapoler est importante et le réseau de neurones pourrait halluciner certaines structures. Ainsi, les volumes reconstruits ne sont pas totalement fiables. Nous donnons donc à notre modèle la capacité d'évaluer sa propre incertitude de reconstruction, et démontrons que cette dernière peut être utilisée comme une estimation fidèle de l'erreur à la vérité.

Nicolas Chahine (DXOMark)
Feb 6th 2024, 15h, room 314 (Pierre Grisvard).
Title: Assessing Portrait Quality in Digital Photography: Methods, Challenges, and Innovations
Abstract: This talk focuses on the development of deep learning-based image quality assessment methods tailored for digital portrait photography. Emphasizing the estimation of portrait-specific quality attributes, it addresses the challenges in predicting various global and local aspects such as color balance, detail rendering, and facial features across a variety of scenarios. Additionally, the seminar introduces PIQ23, a comprehensive portrait-specific IQA dataset. This dataset includes images from a wide range of smartphone models, annotated for key quality attributes by expert evaluators. The discussion will highlight the dataset's role in understanding the consistency of quality assessments and the potential of integrating semantic information to improve IQA predictions in portrait photography.

Jean Prost (IMB, Université de Bordeaux)
Feb 6th 2024, 14h, room 314 (Pierre Grisvard).
Title: Inverse Problem Regularization with a Variational Autoencoder Prior
Abstract: In this presentation, I will introduce various strategies to use pretrained variational autoencoders (VAE) as a prior model to regularize ill-posed image inverse problems, such as deblurring or super-resolution. VAE can model complex data such as images, by defining a latent variable model paramaterized by a deep neural network. However, it is difficult to use the probabilistic model learned by a VAE as a prior for an inverse problem, because it is defined as an intractable integral. In order to circumvent the intractability of the VAE model, I will first present PnP-HVAE, an iterative optimization algorithm which maximizes a joint posterior distribution on an augmented (image-latent) space. PnP-HVAE is adapted to expressive hierarchical VAE models, and enables us to control the strength of the regularization. Additionally, we draw connection with Plug-and-play methods based on deep image denoisers, and we demonstrate the convergence of our algorithm. Next, I will introduce a strategy to sample from the posterior distribution of a super-resolution problem by using a hierarchical VAE (HVAE) as a prior model. To this end, we propose to train an additional encoder on degraded observation in order to condition the HVAE generative process on the degraded observation. We demonstrate that our approach provides sample quality on par with recent diffusion models while being significantly more computationally efficient.

Antoine Salmona (Centre Borelli, ENS Paris Saclay)
Jan 9th 2024, TBA, room 314 (Pierre Grisvard).
Title: Expressivité des modèles génératifs push-forward
Abstract: Les modèles génératifs sont aujourd'hui l'un des sujets de recherche les plus populaires en apprentissage automatique, notamment grâce à leur impressionnante capacité à générer des images synthétiques photo-réalistes. Cependant, il reste souvent difficile de savoir si ces modèles approchent correctement la distribution sous-jacente des données ou s'ils se contentent de générer des échantillons qui semblent similaires aux données. Dans cet exposé, nous nous concentrons sur la classe particulière des modèles génératifs push-forward, qui inclut notamment les Variational Autoencoders (VAEs) et les Generative Adversarial Networks (GANs), et nous mettons en évidence qu'il existe un compromis fondamental pour ces modèles entre leur capacité à approcher des distributions multimodales et la stabilité de leur entraînement. Nous montrons par ailleurs que les modèles à diffusion ne semblent pas souffrir de cette limitation.

Claire Launay (LMBA, Université Bretagne Sud)
Dec 5th 2023, 14h-15h, room 314 (Pierre Grisvard).
Title: Modélisation de textures : champs gaussiens autosimilaires et signal monogène
Abstract: Nos travaux s'intéressent à la représentation de champs gaussiens autosimilaires anisotropes à l’aide du signal monogène. Le signal monogène utilise la transformée de Riesz et permet d’extraire des informations locales d’orientation et de structure d’une image. Une analyse multi-échelle, développée en collaboration avec Hermine Biermé, Céline Lacaux et Philippe Carré, permet d’obtenir des estimateurs non biaisés et fortement consistants des paramètres d’anisotropie et d’autosimilarité de textures gaussiennes particulières, modélisées par des champs élémentaires.

Remy Abergel (MAP5, Université Paris Cité)
Dec 5th 2023, 15h-16h, room 314 (Pierre Grisvard).
Title: Méthodes variationnelles pour l'imagerie par résonance paramagnétique électronique Variational methods for Electron Paramagnetic Resonance Imaging
Abstract: L'imagerie par résonance paramagnétique électronique (RPE) est une méthode d'imagerie des molécules paramagnétiques. Celle-ci est basée sur la capacité des électrons libres à absorber puis réémettre l'énergie électromagnétique en présence d'un champ magnétique. Cet exposé sera consacré à la modélisation du problème direct reliant les mesures RPE à la cartographie des espèces paramagnétiques en présence dans l'échantillon étudié, ainsi qu'aux méthodes variationnelles proposées récemment pour effectuer son inversion. Travaux en collaboration avec Mehdi Boussâa (MAP5 & LCBPT), Sylvain Durand (MAP5) et Yves-Michel Frapart (LCBPT).

Carole Le Guyader (LMI, INSA Rouen)
Nov 14th 2023, 14h-15h, room 314 (Pierre Grisvard).
Title: A Multiscale Deformation Representation
Abstract: Motivated by Tadmor et al.’s work dedicated to multiscale image representation using hierarchical ($BV$, $L^2$) decompositions, we propose transposing their approach to the case of registration, task which consists in determining a smooth deformation aligning the salient constituents visible in an image into their counterpart in another. The underlying goal is to obtain a hierarchical decomposition of the deformation in the form of a composition of intermediate deformations: the coarser one, computed from versions of the two images capturing the essential features, encodes the main structural/geometrical deformation, while iterating the procedure and refining the versions of the two images yields more accurate deformations that map faithfully small-scale features. The proposed model falls within the framework of variational methods and hyperelasticity by viewing the shapes to be matched as Ogden materials. The material behaviour is described by means of a specifically tailored strain energy density function, complemented by $L^\infty$-penalisations ensuring that the computed deformation is a bi-Lipschitz homeomorphism. Theoretical results emphasising the mathematical soundness of the model are provided, among which the existence of minimisers/asymptotic results, and a suitable numerical algorithm is supplied, along with numerical simulations demonstrating the ability of the model to produce accurate hierarchical representations of deformations.

Romain Petit (MaLGa, University of Genoa)
Nov 14th 2023, 15h-16h, room 314 (Pierre Grisvard).
Title: Reconstruction of piecewise constant images via total (gradient) variation regularization
Abstract: In this talk, I will consider the reconstruction of some unknown image from noisy linear measurements using total (gradient) variation regularization. Empirical evidence and theoretical results suggest that this method is particularly well suited to recover piecewise constant images. It is therefore natural to study the case where the unknown image has precisely this structure. I will present two works on this topic, which are collaborations with Yohann De Castro and Vincent Duval. The first concerns a noise robustness result, stating that, in a low noise regime, the reconstruction is also piecewise constant, and one exactly recovers the number of shapes in the unknown image. The second is about introducing a new numerical method for solving the variational regularization problem. Its main feature is that it does not rely on the introduction of a fixed spatial discretization (e.g. a pixel grid), and builds a sequence of iterates that are linear combinations of indicator functions.

Valentin Penaud-Polge (CMM, Mines Paris, PSL)
Oct 3rd 2023, 14h-15h, room 314 (Pierre Grisvard).
Title: GenHarris-ResNet: A Rotation Invariant Neural Network Based on Elementary Symmetric Polynomials
Abstract: We propose a rotation invariant neural network based on Gaussian derivatives. The proposed network covers the main steps of the Harris corner detector in a generalized manner. More precisely, the Harris corner response function is a combination of the elementary symmetric polynomials of the integrated dyadic (outer) product of the gradient with itself. In the same way, we define matrices to be the self dyadic product of vectors composed with higher order partial derivatives and combine the elementary symmetric polynomials. A specific global pooling layer is used to mimic the local pooling used by Harris in his method. The proposed network is evaluated through three experiments. It first shows a quasi perfect invariance to rotations on Fashion-MNIST, it obtains competitive results compared to other rotation invariant networks on MNIST-Rot, and it obtains better performances classifying galaxies (EFIGI Dataset) than networks using up to a thousand times more trainable parameters.

Jonathan Vacher (MAP5, Université Paris Cité)
Oct 3rd 2023, 15h-16h, room 314 (Pierre Grisvard).
Title: Perceptual Measurements, Distances and Metrics [Slides]
Abstract: Perception is often viewed as a process that transforms physical variables, external to an observer, into internal psychological variables. Such a process can be modeled by a function coined perceptual scale. The perceptual scale can be deduced from psychophysical measurements that consist in comparing the relative differences between stimuli (i.e. difference scaling experiments). However, this approach is often overlooked by the modeling and experimentation communities. Here, we demonstrate the value of measuring the perceptual scale of classical (spatial frequency, orientation) and less classical physical variables (interpolation between textures) by embedding it in recent probabilistic modeling of perception. First, we show that the assumption that an observer has an internal representation of univariate parameters such as spatial frequency or orientation while stimuli are high-dimensional does not lead to contradictory predictions when following the theoretical framework. Second, we show that the measured perceptual scale corresponds to the transduction function hypothesized in this framework. In particular, we demonstrate that it is related to the Fisher information of the generative model that underlies perception and we test the predictions given by the generative model of different stimuli in a set a of difference scaling experiments. Our main conclusion is that the perceptual scale is mostly driven by the stimulus power spectrum. Finally, we propose that these measure of perceptual scales is a way to push further the notion of perceptual distances by estimating the perceptual geometry of images i.e. the path between images instead of simply the distance between those.



The seminar is hosted by IHP, and is labelled by the SIGMA group of the SMAI and the RT MIA. We gratefully acknowledge support from Télécom Paris.


En français

Bienvenue sur le site du Séminaire Parisien des Mathématiques Appliquées à l’Imagerie.

Le but de ce séminaire est de couvrir le domaine des mathématiques de l’imagerie. Il est ouvert à tous. Cette année, le séminaire a lieu en Salle 314 (Pierre Grisvard) à l’IHP le premier mardi de chaque mois (le deuxième mardi de novembre et janvier), de 14h00 à 16h00. Chaque séance est composée de deux exposés.

La liste des séminaires antérieurs à l’été 2023 est disponible ici.

Vous pouvez vous abonner ou désabonner à la liste de diffusion du séminaire.