arXiv subjects

Sergey Dolgov

Publications and source records attributed to Sergey Dolgov.

At least 19 recordsLinked to original sources

arXiv2025

An Inexact Tensor-Train Primal-Dual Interior-Point Method for Semidefinite Programs

In this work, we introduce an interior-point method that employs tensor decompositions to efficiently represent and manipulate the variables and constraints of semidefinite programs, targeting problems where the solutions may not be low-rank but admit low-tensor-train rank approximations. Our method leverages a primal-dual infeasible interior-point framework with global and local convergence estimates. In experiments on Maximum Cut, Maximum Stable Set, and Correlation Clustering, the tensor-train interior-point method handles problems up to size $2^{12}$ with certified primal-dual accuracy, reported gaps of approximate order $10^{-4}$, and remains the only primal-dual SDP solver reported at the largest dimensions across all three benchmarks. Moreover, numerical evidence indicates that tensor-train ranks of the iterates typically remain structured and moderate along the interior-point trajectory, with instance-dependent growth for harder high-rank cases, supporting the scalability of the approach. Tensor-train interior-point methods offer a promising avenue for problems that lack traditional sparsity or low-rank structure, exploiting tensor-train structures instead.

math.OC↗

arXiv2025

Alternating steepest descent methods for tensor completion with applications to spectromicroscopy

In this paper we develop two new Tensor Alternating Steepest Descent algorithms for tensor completion in the low-rank $\star_{M}$-product format, whereby we aim to reconstruct an entire low-rank tensor from a small number of measurements thereof. Both algorithms are rooted in the Alternating Steepest Descent (ASD) method for matrix completion, first proposed in [J. Tanner and K. Wei, Appl. Comput. Harmon. Anal., 40 (2016), pp. 417-429]. In deriving the new methods we target the X-ray spectromicroscopy undersampling problem, whereby data are collected by scanning a specimen on a rectangular viewpoint with X-ray beams of different energies. The recorded absorptions coefficients of the mixed specimen materials are naturally stored in a third-order tensor, with spatial horizontal and vertical axes, and an energy axis. To speed the X-ray spectromicroscopy measurement process up, only a fraction of tubes from (a reshaped version of) this tensor are fully scanned, leading to a tensor completion problem. In this framework we can apply any transform (such as the Fourier transform) to the tensor tube by tube, providing a natural way to work with the $\star_{M}$-tensor algebra, and propose: (1) a tensor completion algorithm that is essentially ASD reformulated in the $\star_{M}$-induced metric space and (2) a tensor completion algorithm that solves a set of (readily parallelizable) independent matrix completion problems for the frontal slices of the transformed tensor. The two new methods are tested on real X-ray spectromicroscopy data, demonstrating that they achieve the same reconstruction error with fewer samples from the tensor compared to the matrix completion algorithms applied to a flattened tensor.

math.NA↗

arXiv2025

Tensor cross interpolation for global discrete optimization with application to Bayesian network inference

Global discrete optimization is notoriously difficult due to the lack of gradient information and the curse of dimensionality, making exhaustive search infeasible. Tensor cross approximation is an efficient technique to approximate multivariate tensors (and discretized functions) by tensor product decompositions based on a small number of tensor elements, evaluated on adaptively selected fibers of the tensor, that intersect on submatrices of (nearly) maximum volume. The submatrices of maximum volume are empirically known to contain large elements, hence the entries selected for cross interpolation can also be good candidates for the globally maximal element within the tensor. In this paper we consider evolution of epidemics on networks, and infer the contact network from observations of network nodal states over time. By numerical experiments we demonstrate that the contact network can be inferred accurately by finding the global maximum of the likelihood using tensor cross interpolation. The proposed tensor product approach is flexible and can be applied to global discrete optimization for other problems, e.g. discrete hyperparameter tuning.

stat.CO↗

arXiv2024

Tensor train solution to uncertain optimization problems with shared sparsity penalty

We develop both first and second order numerical optimization methods to solve non-smooth optimization problems featuring a shared sparsity penalty, constrained by differential equations with uncertainty. To alleviate the curse of dimensionality we use tensor product approximations. To handle the non-smoothness of the objective function we employ a smoothed version of the shared sparsity objective. We consider both a benchmark elliptic PDE constraint, and a more realistic topology optimization problem in engineering. We demonstrate that the error converges linearly in iterations and the smoothing parameter, and faster than algebraically in the number of degrees of freedom, consisting of the number of quadrature points in one variable and tensor ranks. Moreover, in the topology optimization problem, the smoothed shared sparsity penalty actually reduces the tensor ranks compared to the unpenalised solution. This enables us to find a sparse high-resolution design under a high-dimensional uncertainty.

math.OC↗

arXiv2024

Tensor product algorithms for inference of contact network from epidemiological data

We consider a problem of inferring contact network from nodal states observed during an epidemiological process. In a black--box Bayesian optimisation framework this problem reduces to a discrete likelihood optimisation over the set of possible networks. The cardinality of this set grows combinatorially with the number of network nodes, which makes this optimisation computationally challenging. For each network, its likelihood is the probability for the observed data to appear during the evolution of the epidemiological process on this network. This probability can be very small, particularly if the network is significantly different from the ground truth network, from which the observed data actually appear. A commonly used stochastic simulation algorithm struggles to recover rare events and hence to estimate small probabilities and likelihoods. In this paper we replace the stochastic simulation with solving the chemical master equation for the probabilities of all network states. Since this equation also suffers from the curse of dimensionality, we apply tensor train approximations to overcome it and enable fast and accurate computations. Numerical simulations demonstrate efficient black--box Bayesian inference of the network.

stat.CO↗

arXiv2024

Smoothed Moreau-Yosida Tensor Train Approximation of State-constrained Optimization Problems under Uncertainty

We propose an algorithm to solve optimization problems constrained by partial (ordinary) differential equations under uncertainty, with almost sure constraints on the state variable. To alleviate the computational burden of high-dimensional random variables, we approximate all random fields by the tensor-train decomposition. To enable efficient tensor-train approximation of the state constraints, the latter are handled using the Moreau-Yosida penalty, with an additional smoothing of the positive part (plus/ReLU) function by a softplus function. In a special case of a quadratic cost minimization constrained by linear elliptic partial differential equations, and some additional constraint qualification, we prove strong convergence of the regularized solution to the optimal control. This result also proposes a practical recipe for selecting the smoothing parameter as a function of the penalty parameter. We develop a second order Newton type method with a fast matrix-free action of the approximate Hessian to solve the smoothed Moreau-Yosida problem. This algorithm is tested on benchmark elliptic problems with random coefficients, optimization problems constrained by random elliptic variational inequalities, and a real-world epidemiological model with 20 random variables. These examples demonstrate mild (at most polynomial) scaling with respect to the dimension and regularization parameters.

math.OC↗

arXiv2023

Statistical Proper Orthogonal Decomposition for model reduction in feedback control

Feedback control synthesis for nonlinear, parameter-dependent fluid flow control problems is considered. The optimal feedback law requires the solution of the Hamilton-Jacobi-Bellman (HJB) PDE suffering the curse of dimensionality. This is mitigated by Model Order Reduction (MOR) techniques, where the system is projected onto a lower-dimensional subspace, over which the feedback synthesis becomes feasible. However, existing MOR methods assume at least one relaxation of generality, that is, the system should be linear, or stable, or deterministic. We propose a MOR method called Statistical POD (SPOD), which is inspired by the Proper Orthogonal Decomposition (POD), but extends to more general systems. Random samples of the original dynamical system are drawn, treating time and initial condition as random variables similarly to possible parameters in the model, and employing a stabilizing closed-loop control. The reduced subspace is chosen to minimize the empirical risk, which is shown to estimate the expected risk of the MOR solution with respect to the distribution of all possible outcomes of the controlled system. This reduced model is then used to compute a surrogate of the feedback control function in the Tensor Train (TT) format that is computationally fast to evaluate online. Using unstable Burgers' and Navier-Stokes equations, it is shown that the SPOD control is more accurate than Linear Quadratic Regulator or optimal control derived from a model reduced onto the standard POD basis, and faster than the direct optimal control of the original system.

math.OC↗

arXiv2023

Undersampling Raster Scans in Spectromicroscopy for reduced dose and faster measurements

Combinations of spectroscopic analysis and microscopic techniques are used across many disciplines of scientific research, including material science, chemistry and biology. X-ray spectromicroscopy, in particular, is a powerful tool used for studying chemical state distributions at the micro and nano scales. With the beam fixed, a specimen is typically rastered through the probe with continuous motion and a range of multimodal data is collected at fixed time intervals. The application of this technique is limited in some areas due to: long scanning times to collect the data, either because of the area/volume under study or the compositional properties of the specimen; and material degradation due to the dose absorbed during the measurement. In this work, we propose a novel approach for reducing the dose and scanning times by undersampling the raster data. This is achieved by skipping rows within scans and reconstructing the x-ray spectromicroscopic measurements using low-rank matrix completion. The new method is robust and allows for x 5-6 reduction in sampling. Experimental results obtained on real data are illustrated.

physics.med-ph↗

arXiv2023

Deep importance sampling using tensor trains with application to a priori and a posteriori rare event estimation

We propose a deep importance sampling method that is suitable for estimating rare event probabilities in high-dimensional problems. We approximate the optimal importance distribution in a general importance sampling problem as the pushforward of a reference distribution under a composition of order-preserving transformations, in which each transformation is formed by a squared tensor-train decomposition. The squared tensor-train decomposition provides a scalable ansatz for building order-preserving high-dimensional transformations via density approximations. The use of composition of maps moving along a sequence of bridging densities alleviates the difficulty of directly approximating concentrated density functions. To compute expectations over unnormalized probability distributions, we design a ratio estimator that estimates the normalizing constant using a separate importance distribution, again constructed via a composition of transformations in tensor-train format. This offers better theoretical variance reduction compared with self-normalized importance sampling, and thus opens the door to efficient computation of rare event probabilities in Bayesian inference problems. Numerical experiments on problems constrained by differential equations show little to no increase in the computational complexity with the event probability going to zero, and allow to compute hitherto unattainable estimates of rare event probabilities for complex, high-dimensional posterior densities.

stat.ML↗

arXiv2023

Numerical solution of the incompressible Navier-Stokes equations for chemical mixers via quantum-inspired Tensor Train Finite Element Method

The solution of computational fluid dynamics problems is one of the most computationally hard tasks, especially in the case of complex geometries and turbulent flow regimes. We propose to use Tensor Train (TT) methods, which possess logarithmic complexity in problem size and have great similarities with quantum algorithms in the structure of data representation. We develop the Tensor train Finite Element Method -- TetraFEM -- and the explicit numerical scheme for the solution of the incompressible Navier-Stokes equation via Tensor Trains. We test this approach on the simulation of liquids mixing in a T-shape mixer, which, to our knowledge, was done for the first time using tensor methods in such non-trivial geometries. As expected, we achieve exponential compression in memory of all FEM matrices and demonstrate an exponential speed-up compared to the conventional FEM implementation on dense meshes. In addition, we discuss the possibility of extending this method to a quantum computer to solve more complex problems. This paper is based on work we conducted for Evonik Industries AG.

physics.flu-dyn↗

arXiv2023

Self-reinforced polynomial approximation methods for concentrated probability densities

Transport map methods offer a powerful statistical learning tool that can couple a target high-dimensional random variable with some reference random variable using invertible transformations. This paper presents new computational techniques for building the Knothe--Rosenblatt (KR) rearrangement based on general separable functions. We first introduce a new construction of the KR rearrangement -- with guaranteed invertibility in its numerical implementation -- based on approximating the density of the target random variable using tensor-product spectral polynomials and downward closed sparse index sets. Compared to other constructions of KR arrangements based on either multi-linear approximations or nonlinear optimizations, our new construction only relies on a weighted least square approximation procedure. Then, inspired by the recently developed deep tensor trains (Cui and Dolgov, Found. Comput. Math. 22:1863--1922, 2022), we enhance the approximation power of sparse polynomials by preconditioning the density approximation problem using compositions of maps. This is particularly suitable for high-dimensional and concentrated probability densities commonly seen in many applications. We approximate the complicated target density by a composition of self-reinforced KR rearrangements, in which previously constructed KR rearrangements -- based on the same approximation ansatz -- are used to precondition the density approximation problem for building each new KR rearrangement. We demonstrate the efficiency of our proposed methods and the importance of using the composite map on several inverse problems governed by ordinary differential equations (ODEs) and partial differential equations (PDEs).

math.NA↗

arXiv2023

Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction

We present a novel offline-online method to mitigate the computational burden of the characterization of posterior random variables in statistical learning. In the offline phase, the proposed method learns the joint law of the parameter random variables and the observable random variables in the tensor-train (TT) format. In the online phase, the resulting order-preserving conditional transport can characterize the posterior random variables given newly observed data in real time. Compared with the state-of-the-art normalizing flow techniques, the proposed method relies on function approximation and is equipped with a thorough performance analysis. The function approximation perspective also allows us to further extend the capability of transport maps in challenging problems with high-dimensional observations and high-dimensional parameters. On the one hand, we present novel heuristics to reorder and/or reparametrize the variables to enhance the approximation power of TT. On the other hand, we integrate the TT-based transport maps and the parameter reordering/reparametrization into layered compositions to further improve the performance of the resulting transport maps. We demonstrate the efficiency of the proposed method on various statistical learning tasks in ordinary differential equations (ODEs) and partial differential equations (PDEs).

stat.ML↗

arXiv2023

Data-driven Tensor Train Gradient Cross Approximation for Hamilton-Jacobi-Bellman Equations

A gradient-enhanced functional tensor train cross approximation method for the resolution of the Hamilton-Jacobi-Bellman (HJB) equations associated to optimal feedback control of nonlinear dynamics is presented. The procedure uses samples of both the solution of the HJB equation and its gradient to obtain a tensor train approximation of the value function. The collection of the data for the algorithm is based on two possible techniques: Pontryagin Maximum Principle and State Dependent Riccati Equations. Several numerical tests are presented in low and high dimension showing the effectiveness of the proposed method and its robustness with respect to inexact data evaluations, provided by the gradient information. The resulting tensor train approximation paves the way towards fast synthesis of the control signal in real-time applications.

math.NA↗

arXiv2023

A weighted subspace exponential kernel for support tensor machines

High-dimensional data in the form of tensors are challenging for kernel classification methods. To both reduce the computational complexity and extract informative features, kernels based on low-rank tensor decompositions have been proposed. However, what decisive features of the tensors are exploited by these kernels is often unclear. In this paper we propose a novel kernel that is based on the Tucker decomposition. For this kernel the Tucker factors are computed based on re-weighting of the Tucker matrices with tuneable powers of singular values from the HOSVD decomposition. This provides a mechanism to balance the contribution of the Tucker core and factors of the data. We benchmark support tensor machines with this new kernel on several datasets. First we generate synthetic data where two classes differ in either Tucker factors or core, and compare our novel and previously existing kernels. We show robustness of the new kernel with respect to both classification scenarios. We further test the new method on real-world datasets. The proposed kernel has demonstrated a higher test accuracy than the state-of-the-art tensor train multi-way multi-level kernel, and a significantly lower computational time.

stat.ML↗

arXiv2023

Functional Tucker approximation using Chebyshev interpolation

This work is concerned with approximating a trivariate function defined on a tensor-product domain via function evaluations. Combining tensorized Chebyshev interpolation with a Tucker decomposition of low multilinear rank yields function approximations that can be computed and stored very efficiently. The existing Chebfun3 algorithm [Hashemi and Trefethen, SIAM J. Sci. Comput., 39 (2017)]uses a similar format but the construction of the approximation proceeds indirectly, via a so called slice-Tucker decomposition. As a consequence, Chebfun3 sometimes uses unnecessarily many function evaluations and does not fully benefit from the potential of the Tucker decomposition to reduce, sometimes dramatically, the computational cost. We propose a novel algorithm Chebfun3F that utilizes univariate fibers instead of bivariate slices to construct the Tucker decomposition. Chebfun3F reduces the cost for the approximation in terms of the number of function evaluations for nearly all functions considered, typically by 75%, and sometimes by over 98%.

math.NA↗

arXiv2022

TTRISK: Tensor Train Decomposition Algorithm for Risk Averse Optimization

This article develops a new algorithm named TTRISK to solve high-dimensional risk-averse optimization problems governed by differential equations (ODEs and/or PDEs) under uncertainty. As an example, we focus on the so-called Conditional Value at Risk (CVaR), but the approach is equally applicable to other coherent risk measures. Both the full and reduced space formulations are considered. The algorithm is based on low rank tensor approximations of random fields discretized using stochastic collocation. To avoid non-smoothness of the objective function underpinning the CVaR, we propose an adaptive strategy to select the width parameter of the smoothed CVaR to balance the smoothing and tensor approximation errors. Moreover, unbiased Monte Carlo CVaR estimate can be computed by using the smoothed CVaR as a control variate. To accelerate the computations, we introduce an efficient preconditioner for the KKT system in the full space formulation.The numerical experiments demonstrate that the proposed method enables accurate CVaR optimization constrained by large-scale discretized systems. In particular, the first example consists of an elliptic PDE with random coefficients as constraints. The second example is motivated by a realistic application to devise a lockdown plan for United Kingdom under COVID-19. The results indicate that the risk-averse framework is feasible with the tensor approximations under tens of random variables.

math.NA↗

arXiv2022

Optimizing semilinear representations for State-dependent Riccati Equation-based feedback control

An optimized variant of the State Dependent Riccati Equations (SDREs) approach for nonlinear optimal feedback stabilization is presented. The proposed method is based on the construction of equivalent semilinear representations associated to the dynamics and their affine combination. The optimal combination is chosen to minimize the discrepancy between the SDRE control and the optimal feedback law stemming from the solution of the corresponding Hamilton Jacobi Bellman (HJB) equation. Numerical experiments assess effectiveness of the method in terms of stability of the closed-loop with near-to-optimal performance.

math.OC↗

arXiv2022

A Quantum Inspired Approach to Exploit Turbulence Structures

Understanding turbulence is the key to our comprehension of many natural and technological flow processes. At the heart of this phenomenon lies its intricate multi-scale nature, describing the coupling between different-sized eddies in space and time. Here we introduce a new paradigm for analyzing the structure of turbulent flows by quantifying correlations between different length scales using methods inspired from quantum many-body physics. We present results for interscale correlations of two paradigmatic flow examples, and use these insights along with tensor network theory to design a structure-resolving algorithm for simulating turbulent flows. With this algorithm, we find that the incompressible Navier-Stokes equations can be accurately solved within a computational space reduced by over an order of magnitude compared to direct numerical simulation. Our quantum-inspired approach provides a pathway towards conducting computational fluid dynamics on quantum computers.

physics.flu-dyn↗