arXiv subjects

David Sommer

Publications and source records attributed to David Sommer.

12 recordsLinked to original sources

arXiv2026

Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Sampling

Diffusion models offer a powerful framework for sampling from complex probability densities by learning to reverse a noising process. A common approach involves solving for the time-reversed stochastic differential equation (SDE), which requires the score function of the evolving sample distribution. The logarithm of this distribution's density is governed by a Hamilton-Jacobi-Bellman (HJB) type partial differential equation (PDE). However, current methods for solving this PDE, such as PINNs or trajectory-based techniques, often suffer from long training times and significant sensitivity to hyperparameter tuning. In this work, we introduce a novel and efficient solver for the underlying HJB equation based on the functional tensor train (FTT) format. The FTT representation leverages latent low-rank structures to efficiently approximate high-dimensional functions, enabling both model compression and rapid computation. By integrating this efficient representation with a backward-in-time iterative scheme derived from backward stochastic differential equations (BSDEs), we develop a fast, robust and accurate sampling method. Our approach overcomes primary bottlenecks of existing techniques, enabling high-fidelity sampling from challenging target distributions with improved efficiency.

stat.ML↗

arXiv2025

Approximation and learning with compositional tensor trains

We introduce compositional tensor trains (CTTs) for the approximation of multivariate functions, a class of models obtained by composing low-rank functions in the tensor-train format. This format can encode standard approximation tools, such as (sparse) polynomials, deep neural networks (DNNs) with fixed width, or tensor networks with arbitrary permutation of the inputs, or more general affine coordinate transformations, with similar complexities. This format can be viewed as a DNN with width exponential in the input dimension and structured weights matrices. Compared to DNNs, this format enables controlled compression at the layer level using efficient tensor algebra. On the optimization side, we derive a layerwise algorithm inspired by natural gradient descent, allowing to exploit efficient low-rank tensor algebra. This relies on low-rank estimations of Gram matrices, and tensor structured random sketching. Viewing the format as a discrete dynamical system, we also derive an optimization algorithm inspired by numerical methods in optimal control. Numerical experiments on regression tasks demonstrate the expressivity of the new format and the relevance of the proposed optimization algorithms. Overall, CTTs combine the expressivity of compositional models with the algorithmic efficiency of tensor algebra, offering a scalable alternative to standard deep neural networks.

math.NA↗

arXiv2024

Sampling from Boltzmann densities with physics informed low-rank formats

Our method proposes the efficient generation of samples from an unnormalized Boltzmann density by solving the underlying continuity equation in the low-rank tensor train (TT) format. It is based on the annealing path commonly used in MCMC literature, which is given by the linear interpolation in the space of energies. Inspired by Sequential Monte Carlo, we alternate between deterministic time steps from the TT representation of the flow field and stochastic steps, which include Langevin and resampling steps. These adjust the relative weights of the different modes of the target distribution and anneal to the correct path distribution. We showcase the efficiency of our method on multiple numerical examples.

cs.LG↗

arXiv2024

Generative Modelling with Tensor Train approximations of Hamilton--Jacobi--Bellman equations

Sampling from probability densities is a common challenge in fields such as Uncertainty Quantification (UQ) and Generative Modelling (GM). In GM in particular, the use of reverse-time diffusion processes depending on the log-densities of Ornstein-Uhlenbeck forward processes are a popular sampling tool. In Berner et al. [2022] the authors point out that these log-densities can be obtained by solution of a \textit{Hamilton-Jacobi-Bellman} (HJB) equation known from stochastic optimal control. While this HJB equation is usually treated with indirect methods such as policy iteration and unsupervised training of black-box architectures like Neural Networks, we propose instead to solve the HJB equation by direct time integration, using compressed polynomials represented in the Tensor Train (TT) format for spatial discretization. Crucially, this method is sample-free, agnostic to normalization constants and can avoid the curse of dimensionality due to the TT compression. We provide a complete derivation of the HJB equation's action on Tensor Train polynomials and demonstrate the performance of the proposed time-step-, rank- and degree-adaptive integration method on a nonlinear sampling task in 20 dimensions.

stat.ML↗

arXiv2024

Machine Learning, Density Functional Theory, and Experiments to Understand the Photocatalytic Reduction of CO$_2$ by CuPt/TiO$_2$

The photoconversion of CO$_2$ to hydrocarbons is a sustainable route to its transformation into value-added compounds and, thereby, crucial to mitigating the energy and climate crises. CuPt nanoparticles on TiO$_2$ surfaces have been reported to show promising photoconversion efficiency. For further progress, a mechanistic understanding of the catalytic properties of these CuPt/TiO$_2$ systems is vital. Here, we employ $\textit{ab-initio}$ calculations, machine learning, and photocatalysis experiments to explore their configurational space and examine their reactivity and find that the interface plays a key role in stabilizing *CO$_2$, *CO, and other CH-containing intermediates, facilitating higher activity and selectivity for methane. A bias-corrected machine-learning interatomic potential trained on density functional theory data enables efficient exploration of the potential energy surfaces of numerous CO$_2$@CuPt/TiO$_2$ configurations via basin-hopping Monte Carlo simulations, greatly accelerating the study of these photocatalyst systems. Our simulations show that CO$_2$ preferentially adsorbs at the interface, with C atom bonded to a Pt site and one O atom occupying an O-vacancy site. The interface also promotes the formation of *CH and *CH$_2$ intermediates. For confirmation, we synthesize CuPt/TiO$_2$ samples with a variety of compositions and analyze their morphologies and compositions using scanning electron microscopy and energy-dispersive X-ray spectroscopy, and measure their photocatalytic activity. Our computational and experimental findings qualitatively agree and highlight the importance of interface design for selective conversion of CO$_2$ to hydrocarbons.

cond-mat.mtrl-sci↗

arXiv2023

Approximating Langevin Monte Carlo with ResNet-like Neural Network architectures

We sample from a given target distribution by constructing a neural network which maps samples from a simple reference, e.g. the standard normal distribution, to samples from the target. To that end, we propose using a neural network architecture inspired by the Langevin Monte Carlo (LMC) algorithm. Based on LMC perturbation results, we show approximation rates of the proposed architecture for smooth, log-concave target distributions measured in the Wasserstein-$2$ distance. The analysis heavily relies on the notion of sub-Gaussianity of the intermediate measures of the perturbed LMC process. In particular, we derive bounds on the growth of the intermediate variance proxies under different assumptions on the perturbations. Moreover, we propose an architecture similar to deep residual neural networks and derive expressivity results for approximating the sample to target distribution map.

cs.LG↗

arXiv2022

Less interaction with forward models in Langevin dynamics

Ensemble methods have become ubiquitous for the solution of Bayesian inference problems. State-of-the-art Langevin samplers such as the Ensemble Kalman Sampler (EKS), Affine Invariant Langevin Dynamics (ALDI) or its extension using weighted covariance estimates rely on successive evaluations of the forward model or its gradient. A main drawback of these methods hence is their vast number of required forward calls as well as their possible lack of convergence in the case of more involved posterior measures such as multimodal distributions. The goal of this paper is to address these challenges to some extend. First, several possible adaptive ensemble enrichment strategies that successively enlarge the number of particles in the underlying Langevin dynamics are discusses that in turn lead to a significant reduction of the total number of forward calls. Second, analytical consistency guarantees of the ensemble enrichment method are provided for linear forward models. Third, to address more involved target distributions, the method is extended by applying adapted Langevin dynamics based on a homotopy formalism for which convergence is proved. Finally, numerical investigations of several benchmark problems illustrates the possible gain of the proposed method, comparing it to state-of-the-art Langevin samplers.

math.NA↗

arXiv2022

Decomposition of Industrial Systems for Energy Efficiency Optimization with OptTopo

The operation of industrial facilities is a broad field for optimization. Industrial plants are often a) composed of several components, b) linked using network technology, c) physically interconnected and d) complex regarding the effect of set-points and operating points in every entity. This leads to the possibility of overall optimization but also to a high complexity of the emerging optimization problems. The decomposition of complex systems allows the modeling of individual models which can be structured according to the physical topology. A method for energy performance indicators (EnPI) helps to formulate an optimization problem. The optimization algorithm OptTopo achieves efficient set-points by traversing a graph representation of the overall system.

eess.SY↗

arXiv2021

Dynamical low-rank approximations of solutions to the Hamilton-Jacobi-Bellman equation

We present a novel method to approximate optimal feedback laws for nonlinear optimal control based on low-rank tensor train (TT) decompositions. The approach is based on the Dirac-Frenkel variational principle with the modification that the optimisation uses an empirical risk. Compared to current state-of-the-art TT methods, our approach exhibits a greatly reduced computational burden while achieving comparable results. A rigorous description of the numerical scheme and demonstrations of its performance are provided.

math.OC↗

arXiv2020

System identification of a hysteresis-controlled pump system using SINDy

Hysteresis-controlled devices are widely used in industrial applications. For example, cooling devices usually contain a two-point controller, resulting in a nonlinear hybrid system with two discrete states. Dynamic models of systems are essential for optimizing such industrial supply technology. However, conventional system identification approaches can hardly handle hysteresis-controlled devices. Thus, the new identification method Sparse Identification of Nonlinear Dynamics (SINDy) is extended to consider hybrid systems. SINDy composes models from basis functions out of a customized library in a data-driven manner. For modeling systems that behave dependent on their own past as in the case of natural hysteresis, Ferenc Preisach introduced the relay hysteron as an elementary mathematical description. In this new method (SINDyHybrid), tailored basis functions in form of relay hysterons are added to the library which is used by SINDy. Experiments with a hysteresis controlled water basin show that this approach correctly identifies state transitions of hybrid systems and also succeeds in modeling the dynamics of the discrete system states. A novel proximity hysteron achieves the robustness of this method. The impacts of the sampling rate and the signal noise ratio of the measurement data are examined accordingly.

eess.SY↗

arXiv2016

Hacking in the Blind: (Almost) Invisible Runtime UI Attacks on Safety-Critical Terminals

Many terminals are used in safety-critical operations in which humans, through terminal user interfaces, become a part of the system control loop (e.g., medical and industrial systems). These terminals are typically embedded, single-purpose devices with restricted functionality, sometimes air-gapped and increasingly hardened. We describe a new way of attacking such terminals in which an adversary has only temporary, non-invasive, physical access to the terminal. In this attack, the adversary attaches a small device to the interface that connects user input peripherals to the terminal. The device executes the attack when the authorized user is performing safety-critical operations, by modifying or blocking user input, or injecting new input events. Given that the attacker has access to user input, the execution of this attack might seem trivial. However, to succeed, the attacker needs to overcome a number of challenges including the inability to directly observe the user interface and avoid being detected by the users. We present techniques that allow user interface state and input tracking. We evaluate these techniques and show that they can be implemented efficiently. We further evaluate the effectiveness of our attack through an online user study and find input modification attacks that are hard for the users to detect and would therefore lead to serious violations of the input integrity.

cs.CR↗

arXiv2014

Charged black holes in Horava gravity

We explore static spherically symmetric black hole solutions allowing a bulk U(1) vector field in the khronometric formulation of Horava gravity by way of Einstein-Aether. We examine analytic solutions and study numerical results in the limit that the khronon does not backreact on the metric.

hep-th↗