**Bayesian inversion with deep learning-driven priors – Application to spectral imaging problems**
Ph.D. proposal in statistical signal/image processing – Diarra FALL 1, Aladine CHETOUANI 2 and Nicolas DOBIGEON 3
1 University of Orleans, Institut Denis Poisson, Orleans, France
2 Polytech’Orleans, PRISME, Orleans, France
3 University of Toulouse, IRIT/INP-ENSEEIHT, Toulouse, France
diarra.fall@univ-orleans.fr, aladine.chetouani@univ-orleans.fr, nicolas.dobigeon@irit.fr
Context
Spectral imaging finds applications in many different fields including remote sensing for Earth observation and in medicine. In Earth observation, multiband imaging provides a detailed characterization of the observed scene by sensing the reflected electromagnetic spectrum in tens nay hundreds of spectral bands. This characterization can be leveraged for ecosystem monitoring, environmental suveillance or land cover mapping. However, multiband images face an unsurpassable trade-off which limits the intrinsic spatial resolution as spectral resolution increases. Several techniques have been developped in the remote sensing literature to overcome this limitation, namely spectral unmixing, subpixel mapping or pansharpening. All these tasks can be
formulated as challenging inverse problems. On the other hand, in medicine, functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique used to
measure evoked changes in cerebral blood oxygenation. Because it is more portable and less restrictive than other popular brain imaging such as functional magnetic resonance imaging (fMRI), fNIRS is widely used with children and other special populations. However, fNIRS has a lower spatial resolution compared to fMRI. Furthermore, the signals are corrupted by physiological noise and motion artefacts, and isolating the desired signals from the unwanted noises is a challenging inverse problem task.
Objectives
Whatever the applicative contexts, the aforementioned restoration problems can be straighforwardly formulated in a Bayesian framework. Indeed the Bayesian paradigm provides a versatile statistical framework to formulate inverse problems. Formulating restoration problems within a Bayesian formalism allows the estimation to be endowed with an assessment of uncertainty, which is of great importance for several applications. However this formulation requires the definition of regularizations by introducing additional information to mitigate the lack of information brought by the observations. For ill-posed problems, the choice of the prior has a significant impact on the solution. Conventional approached generally use explicit priors designed to promote expected or desired properties of the signals and images to be restored. However, in practice, it can be difficult to explicitly define such a function that captures all the desired properties.
As an alternative, we propose to tackle these restoration problems in a Bayesian framework using implicitly priors specified by neural networks. For instance, implicit priors defined by the architecture of convolutional neural networks have been used in [1]. Variational auto-encoders proposed in [2] have been successfully used for learning priors in various imaging problems such as denoising and deblurring in [3]. Plug and play priors [4] appear also of great interest since they have have shown remarkably accurate results when combined with denoisers based on convolutional neural networks [5].
The proposed PhD thesis project aims at developing new Bayesian restoration methods for Earth observation and fNIRS data, using convolutional neural networks data-driven priors. The proposed methods will be applied on hyperspectral mineralogical data from BRGM and acquired in the SOLSA H2020 project for rock analysis; and FNIRS data available at Centre Hospitalier Re ́gional d’Orle ́ans for studying human brain activity during motor execution.
Scientific environment
The three labs involved in this Ph.D. thesis are the Institut Denis Poisson (CNRS and University of Orle ́ans), the PRISME Labo- ratory (Polytech’Orleans) and IRIT (CNRS and Toulouse INP). The Ph.D. student will therefore benefit from a scientifically rich environment and will be able to acquire a solid background on the most recent results and advances in Bayesian signal & image processing and machine learning. He/she will be mainly co-advised by
• Diarra Fall, Associate Professor within the Institut Denis Poisson, University of Orle ́ans,
• Aladine Chetouani, Associate Professor (HDR) within the PRISME Laboratory, Polytech’Orle ́ans, University of Orle ́ans,
• Nicolas Dobigeon, Professor within the SC group at IRIT laboratory (UMR CNRS 5505, Toulouse) and AI Research Chair
at the Artificial and Natural Intelligence Toulouse Institute (ANITI).
The student’s workplace will be the Institut Denis Poisson (campus of the University of Orle ́ans), with periodic visits in Poly-
tech’Orle ́ans (campus of the University of Orle ́ans). He/she may also have short-period visits in Toulouse.
Funding
This position will be co-funded by the ANR project AI.iO and the University of Orle ́ans.
Period
The Ph.D. shall start in September 2022, with a duration of 3 years. The precise starting date can be adjusted according to the availability of the selected candidate.
Profile & requirements
Master or Engineering school student in applied mathematics, computer science or electrical engineering. The knowledge needed for this work includes a strong background in signal & image processing, applied mathematics (probability & statistics, opti- mization, etc.) and/or machine learning. Good scientific programming skills (e.g., Python or Matlab) and good communication skills in English, both written and oral are also expected.
Contact & application procedure
Applicants are also invited to send (as pdf files)
• a detailed curriculum,
• official transcripts from each institution you have attended (in French or English), • up to 3 recommendation letters
to the co-advisors
• Diarra Fall, diarra.fall@univ-orleans.fr
• Aladine Chetouani, aladine.chetouani@univ-orleans.fr • Nicolas Dobigeon, nicolas.dobigeon@irit.fr
You will be contacted if your profile meets the expectations. Review of applications will be closed when the position is filled.
References
[1] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018, pp 9446–9454.
[2] D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in Proc. Int. Conf. Learning Representations (ICLR), 2014.
[3] M. Holden, M. Pereyra and K. C. Zygalakis “Bayesian Imaging With Data-Driven Priors Encoded by Neural Networks: Theory, Methods, and Algorithms”, arXiv:2103.10182
[4] S. V. Venkatakrishnan, C. A. Bouman and B. Wohlberg, “Plug-and-Play priors for model based reconstruction,” in Proc. IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2013, pp. 945–948.
[5] R. Laumont, V. De Bortoli, A. Almansa, J. Delon, A. Durmus and M. Pereyra, “On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent,” arXiv:2201.06133.