arXiv subjects

Ali Mohammad-Djafari

Publications and source records attributed to Ali Mohammad-Djafari.

At least 19 recordsLinked to original sources

arXiv2026

Bayesian PINNs for uncertainty-aware inverse problems (BPINN-IP)

The main contribution of this paper is to develop a hierarchical Bayesian formulation of PINNs for linear inverse problems, which is called BPINN-IP. The proposed methodology extends PINN to account for prior knowledge on the nature of the expected NN output, as well as its weights. Also, as we can have access to the posterior probability distributions, naturally uncertainties can be quantified. Also, variational inference and Monte Carlo dropout are employed to provide predictive means and variances for reconstructed images. Un example of applications to deconvolution and super-resolution is considered, details of the different steps of implementations are given, and some preliminary results are presented.

stat.ML↗

arXiv2025

Bayesian Physics-Informed Neural Networks for Inverse Problems (BPINN-IP): Application in Infrared Image Processing

Inverse problems arise across scientific and engineering domains, where the goal is to infer hidden parameters or physical fields from indirect and noisy observations. Classical approaches, such as variational regularization and Bayesian inference, provide well established theoretical foundations for handling ill posedness. However, these methods often become computationally restrictive in high dimensional settings or when the forward model is governed by complex physics. Physics Informed Neural Networks (PINNs) have recently emerged as a promising framework for solving inverse problems by embedding physical laws directly into the training process of neural networks. In this paper, we introduce a new perspective on the Bayesian Physics Informed Neural Network (BPINN) framework, extending classical PINNs by explicitly incorporating training data generation, modeling and measurement uncertainties through Bayesian prior modeling and doing inference with the posterior laws. Also, as we focus on the inverse problems, we call this method BPINN-IP, and we show that the standard PINN formulation naturally appears as its special case corresponding to the Maximum A Posteriori (MAP) estimate. This unified formulation allows simultaneous exploitation of physical constraints, prior knowledge, and data-driven inference, while enabling uncertainty quantification through posterior distributions. To demonstrate the effectiveness of the proposed framework, we consider inverse problems arising in infrared image processing, including deconvolution and super-resolution, and present results on both simulated and real industrial data.

stat.ML↗

arXiv2025

Temperature calibration of surface emissivities with an improved thermal image enhancement network

Infrared thermography faces persistent challenges in temperature accuracy due to material emissivity variations, where existing methods often neglect the joint optimization of radiometric calibration and image degradation. This study introduces a physically guided neural framework that unifies temperature correction and image enhancement through a symmetric skip-CNN architecture and an emissivity-aware attention module. The pre-processing stage segments the ROIs of the image and and initially corrected the firing rate. A novel dual-constrained loss function strengthens the statistical consistency between the target and reference regions through mean-variance alignment and histogram matching based on Kullback-Leibler dispersion. The method works by dynamically fusing thermal radiation features and spatial context, and the model suppresses emissivity artifacts while recovering structural details. After validating the industrial blower system under different conditions, the improved network realizes the dynamic fusion of thermal radiation characteristics and spatial background, with accurate calibration results in various industrial conditions.

eess.IV↗

arXiv2025

Physics-Informed Neural Networks with Unknown Partial Differential Equations: an Application in Multivariate Time Series

A significant advancement in Neural Network (NN) research is the integration of domain-specific knowledge through custom loss functions. This approach addresses a crucial challenge: how can models utilize physics or mathematical principles to enhance predictions when dealing with sparse, noisy, or incomplete data? Physics-Informed Neural Networks (PINNs) put this idea into practice by incorporating physical equations, such as Partial Differential Equations (PDEs), as soft constraints. This guidance helps the networks find solutions that align with established laws. Recently, researchers have expanded this framework to include Bayesian NNs (BNNs), which allow for uncertainty quantification while still adhering to physical principles. But what happens when the governing equations of a system are not known? In this work, we introduce methods to automatically extract PDEs from historical data. We then integrate these learned equations into three different modeling approaches: PINNs, Bayesian-PINNs (B-PINNs), and Bayesian Linear Regression (BLR). To assess these frameworks, we evaluate them on a real-world Multivariate Time Series (MTS) dataset. We compare their effectiveness in forecasting future states under different scenarios: with and without PDE constraints and accuracy considerations. This research aims to bridge the gap between data-driven discovery and physics-guided learning, providing valuable insights for practical applications.

cs.LG↗

arXiv2025

Digital Twins in Industrial Applications: Concepts, Mathematical Modeling, and Use Cases

Digital Twins (DTs) are virtual representations of physical systems synchronized in real time through Internet of Things (IoT) sensors and computational models. In industrial applications, DTs enable predictive maintenance, fault diagnosis, and process optimization. This paper explores the mathematical foundations of DTs, hybrid modeling techniques, including Physics Informed Neural Networks (PINNs), and their implementation in industrial scenarios. We present key applications, computational tools, and future research directions.

cs.OH↗

arXiv2025

Bayesian Physics Informed Neural Networks for Linear Inverse problems

Inverse problems arise almost everywhere in science and engineering where we need to infer on a quantity from indirect observation. The cases of medical, biomedical, and industrial imaging systems are the typical examples. A very high overview of classification of the inverse problems method can be: i) Analytical, ii) Regularization, and iii) Bayesian inference methods. Even if there are straight links between them, we can say that the Bayesian inference based methods are the most powerful, as they give the possibility of accounting for prior knowledge and can account for errors and uncertainties in general. One of the main limitations stay in computational costs in particular for high dimensional imaging systems. Neural Networks (NN), and in particular Deep NNs (DNN), have been considered as a way to push farther this limit. Physics Informed Neural Networks (PINN) concept integrates physical laws with deep learning techniques to enhance the speed, accuracy and efficiency of the above mentioned problems. In this work, a new Bayesian framework for the concept of PINN (BPINN) is presented and discussed which includes the deterministic one if we use the Maximum A Posteriori (MAP) estimation framework. We consider two cases of supervised and unsupervised for training step, obtain the expressions of the posterior probability of the unknown variables, and deduce the posterior laws of the NN's parameters. We also discuss about the challenges of implementation of these methods in real applications.

cs.LG↗

arXiv2024

Quantum states and quantum computing

In classical theory, the physical systems are elucidated through the concepts of particles and waves, which aim to describe the reality of the physical system with certainty. In this framework, particles are mathematically represented by position vectors as functions of time, $\vec{x}(t)$, while waves are modeled by tensor fields in space-time, $Φ(t, \vec{x})$. These functions are embedded in, and evolve within space-time. All information about the physical system are coded in these mathematical functions, upon which the classical technologies are developed. In contrast, quantum theory models the physical system using a quantum state $\vert α,t\rangle$, situated in an evolving within Hilbert space, portraying the system's reality with inherent uncertainty. Despite the probabilistic nature of reality observation, the quantum state $\vert α,t\rangle$ can be precisely determined due to the unitary principle, provided we know the initial state. Therefore, it can serve as a foundation for developing quantum technologies, which we call quantum state-tronics similar to electronics. This discussion focuses on quantum computation, given its expansive scope. One of the paramount challenges in quantum computing is the scarcity of individuals equipped with the requisite knowledge of quantum field theory and the training necessary for this field. This article aims to elucidate the fundamental concepts of quantum field theory and their interconnections with quantum computing, striving to simplify them for those engaged in quantum computing.

quant-ph↗

arXiv2024

Model Based and Physics Informed Deep Learning Neural Network Structures

Neural Networks (NN) has been used in many areas with great success. When a NN's structure (Model) is given, during the training steps, the parameters of the model are determined using an appropriate criterion and an optimization algorithm (Training). Then, the trained model can be used for the prediction or inference step (Testing). As there are also many hyperparameters, related to the optimization criteria and optimization algorithms, a validation step is necessary before its final use. One of the great difficulties is the choice of the NN's structure. Even if there are many "on the shelf" networks, selecting or proposing a new appropriate network for a given data, signal or image processing, is still an open problem. In this work, we consider this problem using model based signal and image processing and inverse problems methods. We classify the methods in five classes, based on: i) Explicit analytical solutions, ii) Transform domain decomposition, iii) Operator Decomposition, iv) Optimization algorithms unfolding, and v) Physics Informed NN methods (PINN). Few examples in each category are explained.

cs.LG↗

arXiv2024

Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference

This study presents a Bayesian maximum \textit{a~posteriori} (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint maximum \textit{a~posteriori} (JMAP) and variational Bayesian approximation (VBA), are compared to the {LASSO, ridge regression and SINDy algorithms for sparse} regression, by application to several dynamical systems with added {Gaussian or Laplace} noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or ``Gaussian norm'' $||\vec{y}-\hat{\vec{y}}||^2_{M^{-1}} = (\vec{y}-\hat{\vec{y}})^\top {M^{-1}} (\vec{y}-\hat{\vec{y}})$, where $\vec{y}$ is a vector variable, $\hat{\vec{y}}$ is its estimator and $M$ is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection {for the different systems and noise models examined.}

stat.ME↗

arXiv2023

Deep Learning and Inverse Problems

Machine Learning (ML) methods and tools have gained great success in many data, signal, image and video processing tasks, such as classification, clustering, object detection, semantic segmentation, language processing, Human-Machine interface, etc. In computer vision, image and video processing, these methods are mainly based on Neural Networks (NN) and in particular Convolutional NN (CNN), and more generally Deep NN. Inverse problems arise anywhere we have indirect measurement. As, in general, those inverse problems are ill-posed, to obtain satisfactory solutions for them needs prior information. Different regularization methods have been proposed, where the problem becomes the optimization of a criterion with a likelihood term and a regularization term. The main difficulty, however, in great dimensional real applications, remains the computational cost. Using NN, and in particular Deep Learning (DL) surrogate models and approximate computation, can become very helpful. In this work, we focus on NN and DL particularly adapted for inverse problems. We consider two cases: First the case where the forward operator is known and used as physics constraint, the second more general data driven DL methods.

cs.LG↗

arXiv2023

Deep Learning and Bayesian inference for Inverse Problems

Inverse problems arise anywhere we have indirect measurement. As, in general they are ill-posed, to obtain satisfactory solutions for them needs prior knowledge. Classically, different regularization methods and Bayesian inference based methods have been proposed. As these methods need a great number of forward and backward computations, they become costly in computation, in particular, when the forward or generative models are complex and the evaluation of the likelihood becomes very costly. Using Deep Neural Network surrogate models and approximate computation can become very helpful. However, accounting for the uncertainties, we need first understand the Bayesian Deep Learning and then, we can see how we can use them for inverse problems. In this work, we focus on NN, DL and more specifically the Bayesian DL particularly adapted for inverse problems. We first give details of Bayesian DL approximate computations with exponential families, then we will see how we can use them for inverse problems. We consider two cases: First the case where the forward operator is known and used as physics constraint, the second more general data driven DL methods. keyword: Neural Network, Variational Bayesian inference, Bayesian Deep Learning (DL), Inverse problems, Physics based DL.

stat.ML↗

arXiv2017

A generalized multivariate Student-t mixture model for Bayesian classification and clustering of radar waveforms

In this paper, a generalized multivariate Student-t mixture model is developed for classification and clustering of Low Probability of Intercept radar waveforms. A Low Probability of Intercept radar signal is characterized by a pulse compression waveform which is either frequency-modulated or phase-modulated. The proposed model can classify and cluster different modulation types such as linear frequency modulation, non linear frequency modulation, polyphase Barker, polyphase P1, P2, P3, P4, Frank and Zadoff codes. The classification method focuses on the introduction of a new prior distribution for the model hyper-parameters that gives us the possibility to handle sensitivity of mixture models to initialization and to allow a less restrictive modeling of data. Inference is processed through a Variational Bayes method and a Bayesian treatment is adopted for model learning, supervised classification and clustering. Moreover, the novel prior distribution is not a well-known probability distribution and both deterministic and stochastic methods are employed to estimate its expectations. Some numerical experiments show that the proposed method is less sensitive to initialization and provides more accurate results than the previous state of the art mixture models.

stat.ML↗

arXiv2010

Utilisation de la notion de copule en tomographie

Un problème important en statistique est la détermination d'une loi de probabilité jointe à partir de ses lois marginales. Dans le cas bidimensionnel, les lois de probabilité marginales f1 (x) et f2(y) sont reliées à la loi jointe f(x,y) par les intégrales suivant les lignes horizontale et verticale (les deux axes x et y). Ainsi, le problème de la détermination de f(x,y) connaissant f1 (x) et f2(y) est un problème inverse mal posé. En statistique la notion de copule est introduite pour obtenir une solution à ce problème. Un problème similaire en tomographie à rayon X est la reconstruction d'une image f(x,y) représentant la répartition de la densité d'une quantité à l'intérieur de l'objet à partir de ses deux projections horizontale et verticale, f1 (x) et f2(y). Il existe aussi un grand nombre de méthodes pour de tels problèmes fondées sur la transformée de Radon. Dans cet article, nous montrons les liens entre la notion de copule et celle de la tomographie à rayon X et voyons si on peut utiliser les méthodes d'un domaine à l'autre.

physics.data-an↗

arXiv2009

Joint Image Restoration and Segmentation using Gauss-Markov-Potts Prior Models and Variational Bayesian Computation: Technical Details

We propose a method to restore and to segment simultaneously images degraded by a known point spread function (PSF) and additive white noise. For this purpose, we propose a joint Bayesian estimation framework, where a family of non-homogeneous Gauss-Markov fields with Potts region labels models are chosen to serve as priors for images. Since neither the joint maximum a posteriori estimator nor posterior mean one are tractable, the joint posterior law of the image, its segmentation and all the hyper-parameters, is approximated by a separable probability laws using the Variational Bayes technique. This yields a known probability laws of the posterior with mutually dependent shaping parameter, which aims to enhance the convergence speed of the estimator compared to stochastic sampling based estimator. The main work is description is given in [1], while technical details of the variational calculations are presented in the current paper.

physics.data-an↗

arXiv2009

Approche variationnelle pour le calcul bayésien dans les problèmes inverses en imagerie

In a non supervised Bayesian estimation approach for inverse problems in imaging systems, one tries to estimate jointly the unknown image pixels $f$ and the hyperparameters $θ$ given the observed data $g$ and a model $M$ linking these quantities. This is, in general, done through the joint posterior law $p(f,θ|g;M)$. The expression of this joint law is often very complex and its exploration through sampling and computation of the point estimators such as MAP and posterior means need either optimization of or integration of multivariate probability laws. In any of these cases, we need to do approximations. Laplace approximation and sampling by MCMC are two approximation methods, respectively analytical and numerical, which have been used before with success for this task. In this paper, we explore the possibility of approximating this joint law by a separable one in $f$ and in $θ$. This gives the possibility of developing iterative algorithms with more reasonable computational cost, in particular, if the approximating laws are choosed in the exponential conjugate families. The main objective of this paper is to give details of different algorithms we obtain with different choices of these families. To illustrate more in detail this approach, we consider the case of image restoration by simple or myopic deconvolution with separable, simple markovian or hidden markovian models.

math-ph↗

arXiv2007

On the estimation of a parameter with incomplete knowledge on a nuisance parameter

In this paper we consider the problem of estimating a parameter of a probability distribution when we have some prior information on a nuisance parameter. We start by the very simple case where we know perfectly the value of the nuisance parameter. The complete likelihood is the classical tool in this case. Then, progressively, we consider the case where we are given a prior probability distribution on this nuisance parameter. The marginal likelihood is then the classical tool in this case. Then, we consider the case where we only have a fixed number of its moments. Here, we may use the maximum entropy (ME) principle to assign a prior law and thus go back to the previous case. Finally, we consider the case where we know only its median. In our knowledge, there is not any classical tool for this case. We propose then a new tool for this case based on a recently proposed alternative distribution to the marginal probability distribution. This new criterion is obtained by first remarking that the marginal distribution can be considered as the mean value of the original distribution over the prior probability law of the nuisance parameter, and then, by using the median in place of the mean. In this paper, we first summarize the classical tools used for the three first cases, then we give the precise definition of this new criterion and its properties and, finally, present a few examples to show the differences of these cases. Key Words: Nuisance parameter, Bayesian inference, Maximum Entropy, Marginalization, Incomplete knowledge, Mean and Median of the Likelihood over the prior distribution

physics.data-an↗

arXiv2007

Bayesian segmentation of hyperspectral images

In this paper we consider the problem of joint segmentation of hyperspectral images in the Bayesian framework. The proposed approach is based on a Hidden Markov Modeling (HMM) of the images with common segmentation, or equivalently with common hidden classification label variables which is modeled by a Potts Markov Random Field. We introduce an appropriate Markov Chain Monte Carlo (MCMC) algorithm to implement the method and show some simulation results.

physics.data-an↗

arXiv2007

Approche variationnelle pour le calcul bayésien dans les problèmes inverses en imagerie

In a non supervised Bayesian estimation approach for inverse problems in imaging systems, one tries to estimate jointly the unknown image pixels $\fb$ and the hyperparameters $\thetab$. This is, in general, done through the joint posterior law $p(\fb,\thetab|\gb)$. The expression of this joint law is often very complex and its exploration through sampling and computation of the point estimators such as MAP and posterior means need either optimization of non convex criteria or intégration of non Gaussian and multi variate probability laws. In any of these cases, we need to do approximations. We had explored before the possibilities of Laplace approximation and sampling by MCMC. In this paper, we explore the possibility of approximating this joint law by a separable one in $\fb$ and in $\thetab$. This gives the possibility of developing iterative algorithms with more reasonable computational cost, in particular, if the approximating laws are choosed in the exponential conjugate families. The main objective of this paper is to give details of different algorithms we obtain with different choices of these families.

physics.data-an↗