arXiv subjects

Ahmed Attia

Publications and source records attributed to Ahmed Attia.

At least 19 recordsLinked to original sources

arXiv2026

Distributional Sensitivity Analysis: Enabling Differentiability in Sample-Based Inference

This work introduces a mathematical framework for estimating the space-parameter sensitivity of random samples in arbitrary dimensions. Such sensitivity effectively acts as gradients of random samples with respect to distributional parameters, which are essential in sample-based inverse problems in nuclear physics, such as inferring quantum correlation functions. We present two analytical formulae for sensitivity and gradient estimation. The first interprets sensitivity as the partial derivatives of the inverse mapping of 1-D conditional distributions. The second, suited for optimization methods that tolerate inexact gradients, applies a diagonal approximation that reduces computational cost with minimal accuracy loss. When closed forms are unavailable, four second-order numerical algorithms are provided to approximate both expressions. Verification and validation studies confirm the correctness of these algorithms and the effectiveness of the proposed formulae. A nuclear physics application demonstrates how the framework enables uncertainty quantification and parameter inference for quantum correlation functions. Unlike existing approaches, our method requires neither model fitting nor knowledge of sampling algorithms or high-dimensional integrals, making it suitable for black-box or simulation-based samplers. Moreover, it renders arbitrary sampling subroutines differentiable, facilitating integration into deep learning and automatic differentiation frameworks. Algorithmic details and open-source implementations are provided to ensure reproducibility and promote further development.

stat.ML↗

arXiv2026

Probabilistic Approach to Black-Box Binary Optimization with Budget Constraints: Application to Sensor Placement

This paper presents a fully probabilistic approach for solving optimal experimental design problems under budget constraints. The experimental design is viewed as a random variable and is associated with a parametric conditional distribution that inherently models the budget constraints. The original optimization problem is replaced with an optimization over the expected value of the original objective, which is then optimized over the distribution parameters. The resulting optimal parameter (policy) is used to sample the feasible region of binary space to produce estimates of the optimal solution(s) of the original optimization problem. In this work we extend the family of conditional Bernoulli models to model the random variable conditioned by the total number of nonzero entries, that is, the budget constraint. This approach (a) is generally applicable to binary optimization problems with nonstochastic black-box objective functions and budget constraints; (b) employs conditional probabilities to model and sample only the feasible region and thus considerably reduces the computational cost compared with employing soft constraints; and (c) does not employ soft constraints and thus does not require tuning of a regularization parameter, for example to promote sparsity, which is generally challenging. The proposed approach is verified numerically using an optimal sensor placement experiment based on an advection-diffusion forward model in a parameter identification setup.

math.OC↗

arXiv2026

A Probabilistic Approach to Trajectory-Based Optimal Experimental Design

We present a novel probabilistic approach for optimal experimental path design. In this approach a discrete path optimization problem is defined on a static navigation mesh, and trajectories are modeled as random variables governed by a parametric Markov policy. The discrete path optimization problem is then replaced with an equivalent stochastic optimization problem over the policy parameters, resulting in an optimal probability model that samples estimates of the optimal discrete path. This approach enables exploration of the utility function's distribution tail and treats the utility function of the design as a black box, making it applicable to linear and nonlinear inverse problems and beyond experimental design. Numerical verification and analysis are carried out by using a parameter identification problem widely used in model-based optimal experimental design, namely a two-dimensional time-dependent advection diffusion problem in which the initial condition is the inference target. Experiments use both coarse and fine navigation meshes, with either a single moving sensor or a group of seven coordinated sensors, and the proposed approach is evaluated under D-, A-, and E-optimality criteria.

math.OC↗

arXiv2026

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning

Low-resource machine translation (MT) has gained increasing attention as parallel data from low-resource language communities is collected, but many approaches for improving low-resource MT remain underexplored. We investigate a self-supervised reinforcement learning fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models. Our approach translates English into a target low-resource language and then back into English, using a combination of chrF++ and BLEU as the reward function on the reconstructed English sentences. Using the NLLB-MD dataset, we evaluate both the 600M and 1.3B parameter NLLB models and observe consistent improvements for the following languages: Central Aymara, Friulian, Wolof, Dyula, Bhojpuri and Russian. Qualitative inspection of translation outputs indicates increased fluency and semantic fidelity. We argue that our method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving. Code available at: https://github.com/Copticoder/MT-via-Round-Trip-RL

cs.CL↗

arXiv2026

Path-OED for infinite-dimensional Bayesian linear inverse problems governed by PDEs

We consider infinite-dimensional Bayesian linear inverse problems governed by time-dependent partial differential equations (PDEs) and develop a mathematical and computational framework for optimal design of mobile sensor paths in this setting. The proposed path optimal experimental design (path-OED) framework is established rigorously in a function space setting and elaborated for the case of Bayesian c-optimality, which quantifies the posterior variance in a linear functional of the inverse parameter. The latter is motivated by goal-oriented formulations, where we seek to minimize the uncertainty in a scalar prediction of interest. To facilitate computations, we complement the proposed infinite-dimensional framework with discretized formulations, in suitably weighted finite-dimensional inner product spaces, and derive efficient methods for finding optimal sensor paths. The resulting computational framework is flexible, scalable, and can be adapted to a broad range of linear inverse problems and design criteria. We also present extensive computational experiments, for a model inverse problem constrained by an advection-diffusion equation, to demonstrate the effectiveness of the proposed approach.

math.OC↗

arXiv2025

Optimal experimental design with k-space data: application to inverse hemodynamics

Subject-specific cardiovascular models rely on parameter estimation using measurements such as 4D Flow MRI data. However, acquiring high-resolution, high-fidelity functional flow data is costly and taxing for the patient. As a result, there is growing interest in using highly undersampled MRI data to reduce acquisition time and thus the cost, while maximizing the information gain from the data. Examples of such recent work include inverse problems to estimate boundary conditions of aortic blood flow from highly undersampled k-space data. The undersampled data is selected based on a predefined sampling mask which can significantly influences the performance and the quality of the solution of the inverse problem. While there are many established sampling patterns to collect undersampled data, it remains unclear how to select the best sampling pattern for a given set of inference parameters. In this paper we propose an Optimal Experimental Design (OED) framework for MRI measurements in k-space, aiming to find optimal masks for estimating specific parameters directly from k-space. As OED is typically applied to sensor placement problems in spatial locations, this is, to our knowledge, the first time the technique is used in this context. We demonstrate that the masks optimized by employing OED consistently outperform conventional sampling patterns in terms of parameter estimation accuracy and variance, facilitating a speed-up of 10x of the acquisition time while maintaining accuracy.

physics.med-ph↗

arXiv2025

Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNets

Achieving both accuracy and diverse reasoning remains challenging for Large Language Models (LLMs) in complex domains like mathematics. A key bottleneck is evaluating intermediate reasoning steps to guide generation without costly human annotations. To address this, we first introduce a novel Process Reward Model (PRM) trained automatically using Monte Carlo Tree Search coupled with a similarity-based data augmentation technique, effectively capturing step-level reasoning quality. Leveraging this PRM, we then adapt Generative Flow Networks (GFlowNets) to operate at the reasoning step level. Unlike traditional reinforcement learning focused on maximizing a single reward, GFlowNets naturally sample diverse, high-quality solutions proportional to their rewards, as measured by our PRM. Empirical evaluation shows strong improvements in both accuracy and solution diversity on challenging mathematical benchmarks (e.g., +2.59% absolute accuracy on MATH Level 5 for Llama3.2-3B), with effective generalization to unseen datasets (+9.4\% absolute on SAT MATH). Furthermore, we benchmark our PRM against existing open-source reward models, demonstrating superior alignment with reasoning quality and more consistent guidance for downstream generation. Our work demonstrates the potential of PRM-guided, step-level GFlowNets for developing more robust and versatile mathematical reasoning in LLMs.

cs.LG↗

arXiv2025

Robust optimal design of large-scale Bayesian nonlinear inverse problems

We consider robust optimal experimental design (ROED) for nonlinear Bayesian inverse problems governed by partial differential equations (PDEs). An optimal design is one that maximizes some utility quantifying the quality of the solution of an inverse problem. However, the optimal design is dependent on elements of the inverse problem such as the simulation model, the prior, or the measurement error model. ROED aims to produce an optimal design that is aware of the additional uncertainties encoded in the inverse problem and remains optimal even after variations in them. We follow a worst-case scenario approach to develop a new framework for robust optimal design of nonlinear Bayesian inverse problems. The proposed framework a) is scalable and designed for infinite-dimensional Bayesian nonlinear inverse problems constrained by PDEs; b) develops efficient approximations of the utility, namely, the expected information gain; c) employs eigenvalue sensitivity techniques to develop analytical forms and efficient evaluation methods of the gradient of the utility with respect to the uncertainties we wish to be robust against; and d) employs a probabilistic optimization paradigm that properly defines and efficiently solves the resulting combinatorial max-min optimization problem. The effectiveness of the proposed approach is illustrated for optimal sensor placement problem in an inverse problem governed by an elliptic PDE.

math.NA↗

arXiv2023

PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments

This paper describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization. This paper provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.

cs.MS↗

arXiv2023

Centralized calibration of power system dynamic models using variational data assimilation

This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based resources, assessing power systems' dynamic performance under disturbances has become challenging, necessitating robust model calibration methods. The proposed approach expands on previous Bayesian frameworks by establishing a posterior distribution of parameters using an approximation around the maximum a posteriori value. We illustrate the efficacy of our method by generating events of varying intensity, highlighting its ability to capture the systems' evolution accurately and with associated uncertainty estimates. This research improves the precision of dynamic performance assessments in modern power systems, with potential applications in managing uncertainties and optimizing system operations.

math.OC↗

arXiv2023

Heuristic Algorithms for Placing Geomagnetically Induced Current Blocking Devices

We propose a new heuristic approach for solving the challenge of determining optimal placements for geomagnetically induced current blocking devices on electrical grids. Traditionally, these determinations are approached by formulating the problem as mixed-integer nonlinear programming models and solving them using optimization solvers based on the spatial branch-and-bound algorithm.However, computing an optimal solution using the solvers often demands substantial computational time due to their inability to leverage the inherent problem structure. Therefore, in this work we propose a new heuristic approach based on a three-block alternating direction method of multipliers algorithm, and we compare it with an existing stochastic learning algorithm. Both heuristics exploit the structure of the problem of interest. We test these heuristic approaches through extensive numerical experiments conducted on the EPRI-21 and UIUC-150 test systems. The outcomes showcase the superior performance of our methodologies in terms of both solution quality and computational speed when compared with conventional solvers.

math.OC↗

arXiv2023

Robust A-Optimal Experimental Design for Bayesian Inverse Problems

Optimal design of experiments for Bayesian inverse problems has recently gained wide popularity and attracted much attention, especially in the computational science and Bayesian inversion communities. An optimal design maximizes a predefined utility function that is formulated in terms of the elements of an inverse problem, an example being optimal sensor placement for parameter identification. The state-of-the-art algorithmic approaches following this simple formulation generally overlook misspecification of the elements of the inverse problem, such as the prior or the measurement uncertainties. This work presents an efficient algorithmic approach for designing optimal experimental design schemes for Bayesian inverse problems such that the optimal design is robust to misspecification of elements of the inverse problem. Specifically, we consider a worst-case scenario approach for the uncertain or misspecified parameters, formulate robust objectives, and propose an algorithmic approach for optimizing such objectives. Both relaxation and stochastic solution approaches are discussed with detailed analysis and insight into the interpretation of the problem and the proposed algorithmic approach. Extensive numerical experiments to validate and analyze the proposed approach are carried out for sensor placement in a parameter identification problem.

math.OC↗

arXiv2022

Optimal Experimental Design for Inverse Problems in the Presence of Observation Correlations

Optimal experimental design (OED) is the general formalism of sensor placement and decisions about the data collection strategy for engineered or natural experiments. This approach is prevalent in many critical fields such as battery design, numerical weather prediction, geosciences, and environmental and urban studies. State-of-the-art computational methods for experimental design, however, do not accommodate correlation structure in observational errors produced by many expensive-to-operate devices such as X-ray machines or radar and satellite retrievals. Discarding evident data correlations leads to biased results, poor data collection decisions, and waste of valuable resources. We present a general formulation of the OED formalism for model-constrained large-scale Bayesian linear inverse problems, where measurement errors are generally correlated. The proposed approach utilizes the Hadamard product of matrices to formulate the weighted likelihood and is valid for both finite- and infinite- dimensional Bayesian inverse problems. We also discuss widely used approaches for relaxation of the binary OED problem, in light of the proposed pointwise weighting approach, and present a clear interpretation of the relaxed design and its effect on the observational error covariance. Extensive numerical experiments are carried out for empirical verification of the proposed approach by using an advection-diffusion model, where the objective is to optimally place a small set of sensors, under a limited budget, to predict the concentration of a contaminant in a bounded domain.

math.OC↗

arXiv2021

Stochastic Learning Approach to Binary Optimization for Optimal Design of Experiments

We present a novel stochastic approach to binary optimization for optimal experimental design (OED) for Bayesian inverse problems governed by mathematical models such as partial differential equations. The OED utility function, namely, the regularized optimality criterion, is cast into a stochastic objective function in the form of an expectation over a multivariate Bernoulli distribution. The probabilistic objective is then solved by using a stochastic optimization routine to find an optimal observational policy. The proposed approach is analyzed from an optimization perspective and also from a machine learning perspective with correspondence to policy gradient reinforcement learning. The approach is demonstrated numerically by using an idealized two-dimensional Bayesian linear inverse problem, and validated by extensive numerical experiments carried out for sensor placement in a parameter identification setup.

math.OC↗

arXiv2019

An Optimal Experimental Design Framework for Adaptive Inflation and Covariance Localization for Ensemble Filters

We develop an optimal experimental design framework for adapting the covariance inflation and localization in data assimilation problems. Covariance inflation and localization are ubiquitously employed to alleviate the effect of using ensembles of finite sizes in all practical data assimilation systems. The choice of both the inflation factor and the localization radius can have a significant impact on the performance of the assimilation scheme. These parameters are generally tuned by trial and error, rendering them expensive to optimize in practice. Spatially and temporally varying inflation parameter and localization radii have been recently proposed and have been empirically proven to enhance the performance of the employed assimilation filter. In this study, we present a variational framework for adaptive tuning of the inflation and localization parameters. Each of these parameters is optimized independently, with an objective to minimize the uncertainty in the posterior state. The proposed framework does not assume uncorrelated observations or prior errors and can in principle be applied without expert knowledge about the model and the observations. Thus, it is adequate for handling dense as well as sparse observational networks. We present the mathematical formulation, algorithmic description of the approach, and numerical experiments using the two-layer Lorenz-96 model.

cs.CE↗

arXiv2018

DATeS: A Highly-Extensible Data Assimilation Testing Suite v1.0

A flexible and highly-extensible data assimilation testing suite, named DATeS, is described in this paper. DATeS aims to offer a unified testing environment that allows researchers to compare different data assimilation methodologies and understand their performance in various settings. The core of DATeS is implemented in Python and takes advantage of its object-oriented capabilities. The main components of the package (the numerical models, the data assimilation algorithms, the linear algebra solvers, and the time discretization routines) are independent of each other, which offers great flexibility to configure data assimilation applications. DATeS can interface easily with large third-party numerical models written in Fortran or in C, and with a plethora of external solvers.

cs.MS↗

arXiv2018

Goal-Oriented Optimal Design of Experiments for Large-Scale Bayesian Linear Inverse Problems

We develop a framework for goal-oriented optimal design of experiments (GOODE) for large-scale Bayesian linear inverse problems governed by PDEs. This framework differs from classical Bayesian optimal design of experiments (ODE) in the following sense: we seek experimental designs that minimize the posterior uncertainty in the experiment end-goal, e.g., a quantity of interest (QoI), rather than the estimated parameter itself. This is suitable for scenarios in which the solution of an inverse problem is an intermediate step and the estimated parameter is then used to compute a QoI. In such problems, a GOODE approach has two benefits: the designs can avoid wastage of experimental resources by a targeted collection of data, and the resulting design criteria are computationally easier to evaluate due to the often low-dimensionality of the QoIs. We present two modified design criteria, A-GOODE and D-GOODE, which are natural analogues of classical Bayesian A- and D-optimal criteria. We analyze the connections to other ODE criteria, and provide interpretations for the GOODE criteria by using tools from information theory. Then, we develop an efficient gradient-based optimization framework for solving the GOODE optimization problems. Additionally, we present comprehensive numerical experiments testing the various aspects of the presented approach. The driving application is the optimal placement of sensors to identify the source of contaminants in a diffusion and transport problem. We enforce sparsity of the sensor placements using an $\ell_1$-norm penalty approach, and propose a practical strategy for specifying the associated penalty parameter.

cs.CE↗

arXiv2018

A Machine Learning Approach to Adaptive Covariance Localization

Data assimilation plays a key role in large-scale atmospheric weather forecasting, where the state of the physical system is estimated from model outputs and observations, and is then used as initial condition to produce accurate future forecasts. The Ensemble Kalman Filter (EnKF) provides a practical implementation of the statistical solution of the data assimilation problem and has gained wide popularity as. This success can be attributed to its simple formulation and ease of implementation. EnKF is a Monte-Carlo algorithm that solves the data assimilation problem by sampling the probability distributions involved in Bayes theorem. Because of this, all flavors of EnKF are fundamentally prone to sampling errors when the ensemble size is small. In typical weather forecasting applications, the model state space has dimension $10^{9}-10^{12}$, while the ensemble size typically ranges between $30-100$ members. Sampling errors manifest themselves as long-range spurious correlations and have been shown to cause filter divergence. To alleviate this effect covariance localization dampens spurious correlations between state variables located at a large distance in the physical space, via an empirical distance-dependent function. The quality of the resulting analysis and forecast is greatly influenced by the choice of the localization function parameters, e.g., the radius of influence. The localization radius is generally tuned empirically to yield desirable results.This work, proposes two adaptive algorithms for covariance localization in the EnKF framework, both based on a machine learning approach. The first algorithm adapts the localization radius in time, while the second algorithm tunes the localization radius in both time and space. Numerical experiments carried out with the Lorenz-96 model, and a quasi-geostrophic model, reveal the potential of the proposed machine learning approaches.

stat.ME↗