Searcharxiv⌕ Search

arXiv subjects

Hans-Andrea Loeliger

Publications and source records attributed to Hans-Andrea Loeliger.

At least 19 recordsLinked to original sources
arXiv2026

L1 Regularization Paths in Linear Models by Parametric Gaussian Message Passing

The paper considers the computation of L1 regularization paths in a state space setting, which includes L1 regularized Kalman smoothing, linear SVM, LASSO, and more. The paper proposes two new algorithms, which are duals of each other; the first algorithm applies to L1 regularization of independent variables while the second applies to L1 regularization of dependent variables. The heart of the proposed algorithms is parametric Gaussian message passing (i.e., Kalman-type forward-backward recursions) in the pertinent factor graphs. The proposed methods are broadly applicable, they (usually) require only matrix multiplications, and their complexity can be competitive with prior methods in some cases.

cs.LG↗
arXiv2026

A Single-Parameter Factor-Graph Image Prior

We propose a novel piecewise smooth image model with piecewise constant local parameters that are automatically adapted to each image. Technically, the model is formulated in terms of factor graphs with NUP (normal with unknown parameters) priors, and the pertinent computations amount to iterations of conjugate-gradient steps and Gaussian message passing. The proposed model and algorithms are demonstrated with applications to denoising and contrast enhancement.

eess.IV↗
arXiv2025

Continuous-Time Neural Networks Can Stably Memorize Random Spike Trains

The paper explores the capability of continuous-time recurrent neural networks to store and recall precisely timed scores of spike trains. We show (by numerical experiments) that this is indeed possible: within some range of parameters, any random score of spike trains (for all neurons in the network) can be robustly memorized and autonomously reproduced with stable accurate relative timing of all spikes, with probability close to one. We also demonstrate associative recall under noisy conditions. In these experiments, the required synaptic weights are computed offline, to satisfy a template that encourages temporal stability.

cs.NE↗
arXiv2025

Dual NUP Representations and Min-Maximization in Factor Graphs

Normals with unknown parameters (NUP) can be used to convert nontrivial model-based estimation problems into iterations of linear least-squares or Gaussian estimation problems. In this paper, we extend this approach by augmenting factor graphs with convex-dual variables and pertinent NUP representations. In particular, in a state space setting, we propose a new iterative forward-backward algorithm that is dual to a recently proposed backward-forward algorithm.

stat.ML↗
arXiv2025

Model-Predictive Control with NUP Priors

Normals with unknown variance (NUV) and, more generally, normals with unknown parameters (NUP) can represent many useful priors including L_p norms and other sparsifying priors, and they blend well with linear-Gaussian models and Gaussian message passing algorithms. In this paper, we elaborate on recently proposed NUP representations of half-space constraints, box constraints, and finite-level constraints. We then demonstrate the use of such NUP representations for exemplary applications in model predictive control with a variety of constraints on the input, the output, or the internal state of the controlled system. In such applications, the computations boil down to iterations of Kalman-type forward-backward recursions, with a complexity (per iteration) that is linear in the planning horizon. In consequence, this approach can handle long planning horizons, which distinguishes it from the prior art. For nonconvex constraints, this approach has no claim to optimality, but it is empirically very effective.

math.OC↗
arXiv2023

Calibrating Control-Bounded ADCs

The paper considers the calibration of control-bounded analog-to-digital converters. It is demonstrated that variations of the analog frontend can be addressed by calibrating the digital estimation filter. In simulations (both behavioral and transistor level) of a leapfrog analog frontend, the proposed calibration method restores essentially the nominal performance. Moreover, with digital-filter calibration in mind, the paper reformulates the design problem of control-bounded converters and thereby clarifies the role of sampling, desired filter shape, and nominal conversion error.

eess.SY↗
arXiv2021

Half-Space and Box Constraints as NUV Priors: First Results

Normals with unknown variance (NUV) can represent many useful priors and blend well with Gaussian models and message passing algorithms. NUV representations of sparsifying priors have long been known, and NUV representations of binary (and M-level) priors have been proposed very recently. In this document, we propose NUV representations of half-space constraints and box constraints, which allows to add such constraints to any linear Gaussian model with any of the previously known NUV priors without affecting the computational tractability.

stat.ML↗
arXiv2021

A Binarizing NUV Prior and its Use for M-Level Control and Digital-to-Analog Conversion

Priors with a NUV representation (normal with unknown variance) have mostly been used for sparsity. In this paper, a novel NUV prior is proposed that effectively binarizes. While such a prior may have many uses, in this paper, we explore its use for discrete-level control (with M $\geq$ 2 levels) including, in particular, a practical scheme for digital-to-analog conversion. The resulting computations, for each planning period, amount to iterating forward-backward Gaussian message passing recursions (similar to Kalman smoothing), with a complexity (per iteration) that is linear in the planning horizon. In consequence, the proposed method is not limited to a short planning horizon and can therefore outperform "optimal" methods. A preference for sparse level switches can easily be incorporated.

eess.SP↗
arXiv2021

Binary Control and Digital-to-Analog Conversion Using Composite NUV Priors and Iterative Gaussian Message Passing

The paper proposes a new method to determine a binary control signal for an analog linear system such that the state, or some output, of the system follows a given target trajectory. The method can also be used for digital-to-analog conversion. The heart of the proposed method is a new binary-enforcing NUV prior (normal with unknown variance). The resulting computations, for each planning period, amount to iterating forward-backward Gaussian message passing recursions (similar to Kalman smoothing), with a complexity (per iteration) that is linear in the planning horizon. In consequence, the proposed method is not limited to a short planning horizon.

eess.SP↗
arXiv2020

Online Memorization of Random Firing Sequences by a Recurrent Neural Network

This paper studies the capability of a recurrent neural network model to memorize random dynamical firing patterns by a simple local learning rule. Two modes of learning/memorization are considered: The first mode is strictly online, with a single pass through the data, while the second mode uses multiple passes through the data. In both modes, the learning is strictly local (quasi-Hebbian): At any given time step, only the weights between the neurons firing (or supposed to be firing) at the previous time step and those firing (or supposed to be firing) at the present time step are modified. The main result of the paper is an upper bound on the probability that the single-pass memorization is not perfect. It follows that the memorization capacity in this mode asymptotically scales like that of the classical Hopfield model (which, in contrast, memorizes static patterns). However, multiple-rounds memorization is shown to achieve a higher capacity (with a nonvanishing number of bits per connection/synapse). These mathematical findings may be helpful for understanding the functions of short-term memory and long-term memory in neuroscience.

cs.IT↗
arXiv2019

Quantum Measurement as Marginalization and Nested Quantum Systems

In prior work, we have shown how the basic concepts and terms of quantum mechanics relate to factorizations and marginals of complex-valued quantum mass functions, which are generalizations of joint probability mass functions. In this paper, using quantum mass functions, we discuss the realization of measurements in terms of unitary interactions and marginalizations. It follows that classical measurement results strictly belong to local models, i.e., marginals of more detailed models. Classical variables that are created by marginalization do not exist in the unmarginalized model, and different marginalizations may yield incompatible classical variables. These observations are illustrated by the Frauchiger-Renner paradox, which is analyzed (and resolved) in terms of quantum mass functions. Throughout, the paper uses factor graphs to represent quantum systems/models with multiple measurements at different points in time.

quant-ph↗
arXiv2018

Simultaneous Partial Inverses and Decoding Interleaved Reed-Solomon Codes

The paper introduces the simultaneous partial-inverse problem (SPI) for polynomials and develops its application to decoding interleaved Reed--Solomon codes beyond half the minimum distance. While closely related both to standard key equations and to well-known Padé approximation problems, the SPI problem stands out in several respects. First, the SPI problem has a unique solution (up to a scale factor), which satisfies a natural degree bound. Second, the SPI problem can be transformed (monomialized) into an equivalent SPI problem where all moduli are monomials. Third, the SPI problem can be solved by an efficient algorithm of the Berlekamp--Massey type. Fourth, decoding interleaved Reed--Solomon codes (or subfield-evaluation codes) beyond half the minimum distance can be analyzed in terms of a partial-inverse condition for the error pattern: if that condition is satisfied, then the (true) error locator polynomial is the unique solution of a standard key equation and can be computed in many different ways, including the well-known multi-sequence Berlekamp--Massey algorithm and the SPI algorithm of this paper. Two of the best performance bounds from the literature (the Schmidt--Sidorenko--Bossert bound and the Roth--Vontobel bound) are generalized to hold for the partial-inverse condition and thus to apply to several different decoding algorithms.

cs.IT↗
arXiv2017

Factor Graphs for Quantum Probabilities

A factor-graph representation of quantum-mechanical probabilities (involving any number of measurements) is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables. All joint probability distributions are marginals of some complex-valued function $q$, and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of $q$.

cs.IT↗
arXiv2016

On Sparsity by NUV-EM, Gaussian Message Passing, and Kalman Smoothing

Normal priors with unknown variance (NUV) have long been known to promote sparsity and to blend well with parameter learning by expectation maximization (EM). In this paper, we advocate this approach for linear state space models for applications such as the estimation of impulsive signals, the detection of localized events, smoothing with occasional jumps in the state space, and the detection and removal of outliers. The actual computations boil down to multivariate-Gaussian message passing algorithms that are closely related to Kalman smoothing. We give improved tables of Gaussian-message computations from which such algorithms are easily synthesized, and we point out two preferred such algorithms.

cs.IT↗
arXiv2013

Partition Function of the Ising Model via Factor Graph Duality

The partition function of a factor graph and the partition function of the dual factor graph are related to each other by the normal factor graph duality theorem. We apply this result to the classical problem of computing the partition function of the Ising model. In the one-dimensional case, we thus obtain an alternative derivation of the (well-known) analytical solution. In the two-dimensional case, we find that Monte Carlo methods are much more efficient on the dual graph than on the original graph, especially at low temperature.

cs.IT↗
arXiv2013

Reverse Berlekamp-Massey Decoding

We propose a new algorithm for decoding Reed-Solomon codes (up to half the minimum distance) and for computing inverses in $F[x]/m(x)$. The proposed algorithm is similar in spirit and structure to the Berlekamp-Massey algorithm, but it works naturally for general $m(x)$.

cs.IT↗
arXiv2013

LMMSE Estimation and Interpolation of Continuous-Time Signals from Discrete-Time Samples Using Factor Graphs

The factor graph approach to discrete-time linear Gaussian state space models is well developed. The paper extends this approach to continuous-time linear systems/filters that are driven by white Gaussian noise. By Gaussian message passing, we then obtain MAP/MMSE/LMMSE estimates of the input signal, or of the state, or of the output signal from noisy observations of the output signal. These estimates may be obtained with arbitrary temporal resolution. The proposed input signal estimation does not seem to have appeared in the prior Kalman filtering literature.

cs.IT↗