arXiv subjects

Hossein Gorji

Publications and source records attributed to Hossein Gorji.

7 recordsLinked to original sources

arXiv2026

Variance Reduction in the Fokker-Planck Particle Method for Rarefied Gases using Quasi-Random Numbers

The Fokker-Planck (FP) particle method accelerates rarefied-gas simulations by replacing the binary collisions of the commonly used Direct Simulation Monte Carlo (DSMC) method with a drift=diffusion process. Like all particle methods, the FP method is inherently stochastic, which leads to statistical fluctuations in macroscopic quantities and necessitates large particle numbers for accurate results. In this work, we investigate the use of quasi-random numbers, which sample distributions more evenly and thereby reduce the variance. To preserve the low-discrepancy structure across time steps, we employ the Array Randomized Quasi-Monte Carlo (Array-RQMC) technique. We combine the FP method with Array-RQMC and compare it in homogeneous and inhomogeneous problems with other commonly used variance-reduction techniques. The proposed FP-Array-RQMC approach achieves improved convergence rates compared with pseudo-random sampling and yields smaller estimator errors for sufficiently large particle numbers.

math.NA↗

arXiv2025

Surrogate Trajectories Along Probability Flows: Pseudo Markovian Alternative to Mori Zwanzig

Model reduction techniques have emerged as a powerful paradigm across different fronts of scientific computing. Despite their success, the provided tools and methodologies remain limited if high-dimensional dynamical systems subject to initial uncertainty and/or stochastic noise are encountered; in particular if rare events are of interest. We address this open challenge by borrowing ideas from Mori-Zwanzig formalism and Chorin's optimal prediction method. The novelty of our work lies on employing time-dependent optimal projection of the dynamic on a desired set of resolved variables. We show several theoretical and numerical properties of our model reduction approach. In particular, we show that the devised surrogate trajectories are consistent with the probability flow of the full-order system. Furthermore, we identify the measure underlying the projection through polynomial chaos expansion technique. This allows us to efficiently compute the projection even for trajectories that are initiated on low probability events. Moreover, we investigate the introduced model-reduction error of the surrogate trajectories on a standard setup, characterizing the convergence behaviour of the scheme. Several numerical results highlight the computational advantages of the proposed scheme in comparison to Monte-Carlo and optimal prediction method. Through this framework, we demonstrate that by tracking the measure along with the consistent projection of the dynamic we are able to access accurate estimates of different statistics including observables conditional on a given initial configuration.

stat.CO↗

arXiv2025

Optimal Transportation by Orthogonal Coupling Dynamics

Many numerical and learning algorithms rely on the solution of the Monge-Kantorovich problem and Wasserstein distances, which provide appropriate distributional metrics. While the natural approach is to treat the problem as an infinite-dimensional linear programming, such a methodology limits the computational performance due to the polynomial scaling with respect to the sample size along with intensive memory requirements. We propose a novel alternative framework to address the Monge-Kantorovich problem based on a projection type gradient descent scheme. The dynamics builds on the notion of the conditional expectation, where the connection with the opinion dynamics is leveraged to devise efficient numerical schemes. We demonstrate that the resulting dynamics recovers random maps with favourable computational performance. Along with the theoretical insight, the proposed dynamics paves the way for innovative approaches to construct numerical schemes for computing optimal transport maps as well as Wasserstein distances.

math.OC↗

arXiv2025

Why Cannot Neural Networks Master Extrapolation? Insights from Physical Laws

Motivated by the remarkable success of Foundation Models (FMs) in language modeling, there has been growing interest in developing FMs for time series prediction, given the transformative power such models hold for science and engineering. This culminated in significant success of FMs in short-range forecasting settings. However, extrapolation or long-range forecasting remains elusive for FMs, which struggle to outperform even simple baselines. This contrasts with physical laws which have strong extrapolation properties, and raises the question of the fundamental difference between the structure of neural networks and physical laws. In this work, we identify and formalize a fundamental property characterizing the ability of statistical learning models to predict more accurately outside of their training domain, hence explaining performance deterioration for deep learning models in extrapolation settings. In addition to a theoretical analysis, we present empirical results showcasing the implications of this property on current deep learning architectures. Our results not only clarify the root causes of the extrapolation gap but also suggest directions for designing next-generation forecasting models capable of mastering extrapolation.

cs.LG↗

arXiv2025

Can Linear Probes Measure LLM Uncertainty?

Effective Uncertainty Quantification (UQ) represents a key aspect for reliable deployment of Large Language Models (LLMs) in automated decision-making and beyond. Yet, for LLM generation with multiple choice structure, the state-of-the-art in UQ is still dominated by the naive baseline given by the maximum softmax score. To address this shortcoming, we demonstrate that taking a principled approach via Bayesian statistics leads to improved performance despite leveraging the simplest possible model, namely linear regression. More precisely, we propose to train multiple Bayesian linear models, each predicting the output of a layer given the output of the previous one. Based on the obtained layer-level posterior distributions, we infer the global uncertainty level of the LLM by identifying a sparse combination of distributional features, leading to an efficient UQ scheme. Numerical experiments on various LLMs show consistent improvement over state-of-the-art baselines.

cs.LG↗

arXiv2025

Fisher entropic Fokker-Planck model of monatomic rarefied gases

Particle-based stochastic approximations of the Boltzmann equation are popular tools for simulations of non-equilibrium gas flows, for which the Navier-Stokes-Fourier equations fail to provide accurate description. However, these numerical methods are computationally demanding, especially in the near-continuum regime, where the collisions become overwhelming. On the other hand, the Fokker-Planck kinetic models offer an efficient alternative, as the binary collisions are described by a diffusive process. Despite the intuitive advantage, rigorous and efficient Fokker-Planck approximations of the Boltzmann equation remain an open problem. On one hand, the moment projection of the Fokker-Planck operator should be consistent with that of the Boltzmann operator. On the other hand, the Fokker-Planck model should be constructed in such a way that the H-theorem is satisfied. The central aim of this study is fulfilling these two categorically different constraints, i.e. moment matching and entropy dissipation, within a flexible and tractable Fokker-Planck framework. To this end, we introduce a Fisher information-based entropic constraint and demonstrate that, with a suitable polynomial expansion of the drift term, it is possible to simultaneously achieve weak moment matching while honouring the H-theorem. We support our theoretical result by numerical experiments on the shock problem, validating our Fisher Entropic Fokker-Planck framework.

math.NA↗

arXiv2025

Collision-based Dynamics for Multi-Marginal Optimal Transport

Inspired by the Boltzmann kinetics, we propose a collision-based dynamics with a Monte Carlo solution algorithm that approximates the solution of the multi-marginal optimal transport problem via randomized pairwise swapping of sample indices. The computational complexity and memory usage of the proposed method scale linearly with the number of samples, making it highly attractive for high-dimensional settings. In several examples, we demonstrate the efficiency of the proposed method compared to the state-of-the-art methods.

cs.AI↗