arXiv subjects

David Reeb

Publications and source records attributed to David Reeb.

At least 19 recordsLinked to original sources

arXiv2024

Tighter Confidence Bounds for Sequential Kernel Regression

Confidence bounds are an essential tool for rigorously quantifying the uncertainty of predictions. They are a core component in many sequential learning and decision-making algorithms, with tighter confidence bounds giving rise to algorithms with better empirical performance and better performance guarantees. In this work, we use martingale tail inequalities to establish new confidence bounds for sequential kernel regression. Our confidence bounds can be computed by solving a conic program, although this bare version quickly becomes impractical, because the number of variables grows with the sample size. However, we show that the dual of this conic program allows us to efficiently compute tight confidence bounds. We prove that our new confidence bounds are always tighter than existing ones in this setting. We apply our confidence bounds to kernel bandit problems, and we find that when our confidence bounds replace existing ones, the KernelUCB (GP-UCB) algorithm has better empirical performance, a matching worst-case performance guarantee and comparable computational cost.

stat.ML↗

arXiv2023

Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures

We present improved algorithms with worst-case regret guarantees for the stochastic linear bandit problem. The widely used "optimism in the face of uncertainty" principle reduces a stochastic bandit problem to the construction of a confidence sequence for the unknown reward function. The performance of the resulting bandit algorithm depends on the size of the confidence sequence, with smaller confidence sets yielding better empirical performance and stronger regret guarantees. In this work, we use a novel tail bound for adaptive martingale mixtures to construct confidence sequences which are suitable for stochastic bandits. These confidence sequences allow for efficient action selection via convex programming. We prove that a linear bandit algorithm based on our confidence sequences is guaranteed to achieve competitive worst-case regret. We show that our confidence sequences are tighter than competitors, both empirically and theoretically. Finally, we demonstrate that our tighter confidence sequences give improved performance in several hyperparameter tuning tasks.

stat.ML↗

arXiv2022

PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison

PAC-Bayes has recently re-emerged as an effective theory with which one can derive principled learning algorithms with tight performance guarantees. However, applications of PAC-Bayes to bandit problems are relatively rare, which is a great misfortune. Many decision-making problems in healthcare, finance and natural sciences can be modelled as bandit problems. In many of these applications, principled algorithms with strong performance guarantees would be very much appreciated. This survey provides an overview of PAC-Bayes bounds for bandit problems and an experimental comparison of these bounds. On the one hand, we found that PAC-Bayes bounds are a useful tool for designing offline bandit algorithms with performance guarantees. In our experiments, a PAC-Bayesian offline contextual bandit algorithm was able to learn randomised neural network polices with competitive expected reward and non-vacuous performance guarantees. On the other hand, the PAC-Bayesian online bandit algorithms that we tested had loose cumulative regret bounds. We conclude by discussing some topics for future work on PAC-Bayesian bandit algorithms.

cs.LG↗

arXiv2022

Validation of Composite Systems by Discrepancy Propagation

Assessing the validity of a real-world system with respect to given quality criteria is a common yet costly task in industrial applications due to the vast number of required real-world tests. Validating such systems by means of simulation offers a promising and less expensive alternative, but requires an assessment of the simulation accuracy and therefore end-to-end measurements. Additionally, covariate shifts between simulations and actual usage can cause difficulties for estimating the reliability of such systems. In this work, we present a validation method that propagates bounds on distributional discrepancy measures through a composite system, thereby allowing us to derive an upper bound on the failure probability of the real system from potentially inaccurate simulations. Each propagation step entails an optimization problem, where -- for measures such as maximum mean discrepancy (MMD) -- we develop tight convex relaxations based on semidefinite programs. We demonstrate that our propagation method yields valid and useful bounds for composite systems exhibiting a variety of realistic effects. In particular, we show that the proposed method can successfully account for data shifts within the experimental design as well as model inaccuracies within the simulation.

cs.LG↗

arXiv2022

Utilizing Expert Features for Contrastive Learning of Time-Series Representations

We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.

cs.LG↗

arXiv2022

PAC-Bayesian Lifelong Learning For Multi-Armed Bandits

We present a PAC-Bayesian analysis of lifelong learning. In the lifelong learning problem, a sequence of learning tasks is observed one-at-a-time, and the goal is to transfer information acquired from previous tasks to new learning tasks. We consider the case when each learning task is a multi-armed bandit problem. We derive lower bounds on the expected average reward that would be obtained if a given multi-armed bandit algorithm was run in a new task with a particular prior and for a set number of steps. We propose lifelong learning algorithms that use our new bounds as learning objectives. Our proposed algorithms are evaluated in several lifelong multi-armed bandit problems and are found to perform better than a baseline method that does not use generalisation bounds.

cs.LG↗

arXiv2021

SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning

Pruning neural networks reduces inference time and memory costs. On standard hardware, these benefits will be especially prominent if coarse-grained structures, like feature maps, are pruned. We devise two novel saliency-based methods for second-order structured pruning (SOSP) which include correlations among all structures and layers. Our main method SOSP-H employs an innovative second-order approximation, which enables saliency evaluations by fast Hessian-vector products. SOSP-H thereby scales like a first-order method despite taking into account the full Hessian. We validate SOSP-H by comparing it to our second method SOSP-I that uses a well-established Hessian approximation, and to numerous state-of-the-art methods. While SOSP-H performs on par or better in terms of accuracy, it has clear advantages in terms of scalability and efficiency. This allowed us to scale SOSP-H to large-scale vision tasks, even though it captures correlations across all layers of the network. To underscore the global nature of our pruning methods, we evaluate their performance not only by removing structures from a pretrained network, but also by detecting architectural bottlenecks. We show that our algorithms allow to systematically reveal architectural bottlenecks, which we then remove to further increase the accuracy of the networks.

cs.LG↗

arXiv2020

Which Minimizer Does My Neural Network Converge To?

The loss surface of an overparameterized neural network (NN) possesses many global minima of zero training error. We explain how common variants of the standard NN training procedure change the minimizer obtained. First, we make explicit how the size of the initialization of a strongly overparameterized NN affects the minimizer and can deteriorate its final test performance. We propose a strategy to limit this effect. Then, we demonstrate that for adaptive optimization such as AdaGrad, the obtained minimizer generally differs from the gradient descent (GD) minimizer. This adaptive minimizer is changed further by stochastic mini-batch training, even though in the non-adaptive case, GD and stochastic GD result in essentially the same minimizer. Lastly, we explain that these effects remain relevant for less overparameterized NNs. While overparameterization has its benefits, our work highlights that it induces sources of error absent from underparameterized models.

stat.ML↗

arXiv2020

Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties

Deep Gaussian Processes learn probabilistic data representations for supervised learning by cascading multiple Gaussian Processes. While this model family promises flexible predictive distributions, exact inference is not tractable. Approximate inference techniques trade off the ability to closely resemble the posterior distribution against speed of convergence and computational efficiency. We propose a novel Gaussian variational family that allows for retaining covariances between latent processes while achieving fast convergence by marginalising out all global latent variables. After providing a proof of how this marginalisation can be done for general covariances, we restrict them to the ones we empirically found to be most important in order to also achieve computational efficiency. We provide an efficient implementation of our new approach and apply it to several benchmark datasets. It yields excellent results and strikes a better balance between accuracy and calibrated uncertainty estimates than its state-of-the-art alternatives.

stat.ML↗

arXiv2019

A refinement of Reznick's Positivstellensatz with applications to quantum information theory

In his solution of Hilbert's 17th problem Artin showed that any positive definite polynomial in several variables can be written as the quotient of two sums of squares. Later Reznick showed that the denominator in Artin's result can always be chosen as an $N$-th power of the squared norm of the variables and gave explicit bounds on $N$. By using concepts from quantum information theory (such as partial traces, optimal cloning maps, and an identity due to Chiribella) we give simpler proofs and minor improvements of both real and complex versions of this result. Moreover, we discuss constructions of Hilbert identities using Gaussian integrals and we review an elementary method to construct complex spherical designs. Finally, we apply our results to give improved bounds for exponential quantum de Finetti theorems in the real and in the complex setting.

quant-ph↗

arXiv2018

Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds

Gaussian Processes (GPs) are a generic modelling tool for supervised learning. While they have been successfully applied on large datasets, their use in safety-critical applications is hindered by the lack of good performance guarantees. To this end, we propose a method to learn GPs and their sparse approximations by directly optimizing a PAC-Bayesian bound on their generalization performance, instead of maximizing the marginal likelihood. Besides its theoretical appeal, we find in our evaluation that our learning method is robust and yields significantly better generalization guarantees than other common GP approaches on several regression benchmark datasets.

stat.ML↗

arXiv2017

Bounds on Information Combining With Quantum Side Information

"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in classical information theory, particularly in coding and Shannon theory; entropy power inequalities are special instances of them. The arguably most elementary kind of information combining is the addition of two binary random variables (a CNOT gate), and the resulting quantities play an important role in Belief propagation and Polar coding. We investigate this problem in the setting where quantum side information is available, which has been recognized as a hard setting for entropy power inequalities. Our main technical result is a non-trivial, and close to optimal, lower bound on the combined entropy, which can be seen as an almost optimal "quantum Mrs. Gerber's Lemma". Our proof uses three main ingredients: (1) a new bound on the concavity of von Neumann entropy, which is tight in the regime of low pairwise state fidelities; (2) the quantitative improvement of strong subadditivity due to Fawzi-Renner, in which we manage to handle the minimization over recovery maps; (3) recent duality results on classical-quantum-channels due to Renes et al. We furthermore present conjectures on the optimal lower and upper bounds under quantum side information, supported by interesting analytical observations and strong numerical evidence. We finally apply our bounds to Polar coding for binary-input classical-quantum channels, and show the following three results: (A) Even non-stationary channels polarize under the polar transform. (B) The blocklength required to approach the symmetric capacity scales at most sub-exponentially in the gap to capacity. (C) Under the aforementioned lower bound conjecture, a blocklength polynomial in the gap suffices.

quant-ph↗

arXiv2016

Fundamental energy cost for quantum measurement

Measurements and feedback are essential in the control of any device operating at the quantum scale and exploiting the features of quantum physics. As the number of quantum components grows, it becomes imperative to consider the energetic expense of such elementary operations. Here, we derive energy requirements for general quantum measurement, extending previous models and obtaining stronger bounds in relevant situations, and then study two important classes of measurements in detail. One is the projective measurement, where we obtain the exact cost rather than a lower bound, and the other is the so-called inefficient measurement, in which we explicitly show that energy extraction is possible. As applications, we derive the energy-precision trade-off in quantum Zeno stabilisation schemes and the exact energy expense for quantum error correction. Our results constitute fundamental energetic limitations against which to benchmark implementations of future quantum devices as they grow in complexity.

quant-ph↗

arXiv2016

Measurement Uncertainty for Finite Quantum Observables

Measurement uncertainty relations are lower bounds on the errors of any approximate joint measurement of two or more quantum observables. The aim of this paper is to provide methods to compute optimal bounds of this type. The basic method is semidefinite programming, which we apply to arbitrary finite collections of projective observables on a finite dimensional Hilbert space. The quantification of errors is based on an arbitrary cost function, which assigns a penalty to getting result $x$ rather than y, for any pair (x,y). This induces a notion of optimal transport cost for a pair of probability distributions, and we include an appendix with a short summary of optimal transport theory as needed in our context. There are then different ways to form an overall figure of merit from the comparison of distributions. We consider three, which are related to different physical testing scenarios. The most thorough test compares the transport distances between the marginals of a joint measurement and the reference observables for every input state. Less demanding is a test just on the states for which a "true value" is known in the sense that the reference observable yields a definite outcome. Finally, we can measure a deviation as a single expectation value by comparing the two observables on the two parts of a maximally entangled state. All three error quantities have the property that they vanish if and only if the tested observable is equal to the reference. The theory is illustrated with some characteristic examples.

quant-ph↗

arXiv2015

Monotonicity of the Quantum Relative Entropy Under Positive Maps

We prove that the quantum relative entropy decreases monotonically under the application of any positive trace-preserving linear map, for underlying separable Hilbert spaces. This answers in the affirmative a natural question that has been open for a long time, as monotonicity had previously only been shown to hold under additional assumptions, such as complete positivity or Schwarz-positivity of the adjoint map. The first step in our proof is to show monotonicity of the sandwiched Renyi divergences under positive trace-preserving maps, extending a proof of the data processing inequality by Beigi [J. Math. Phys. 54, 122202 (2013)] that is based on complex interpolation techniques. Our result calls into question several measures of non-Markovianity that have been proposed, as these would assess all positive trace-preserving time evolutions as Markovian.

quant-ph↗

arXiv2015

Positivity of linear maps under tensor powers

We investigate linear maps between matrix algebras that remain positive under tensor powers, i.e., under tensoring with $n$ copies of themselves. Completely positive and completely co-positive maps are trivial examples of this kind. We show that for every $n\in\mathbb{N}$ there exist non-trivial maps with this property and that for two-dimensional Hilbert spaces there is no non-trivial map for which this holds for all $n$. For higher dimensions we reduce the existence question of such non-trivial "tensor-stable positive maps" to a one-parameter family of maps and show that an affirmative answer would imply the existence of NPPT bound entanglement. As an application we show that any tensor-stable positive map that is not completely positive yields an upper bound on the quantum channel capacity, which for the transposition map gives the well-known cb-norm bound. We furthermore show that the latter is an upper bound even for the LOCC-assisted quantum capacity, and that moreover it is a strong converse rate for this task.

quant-ph↗

arXiv2015

Scalable Dissipative Preparation of Many-Body Entanglement

We present a technique for the dissipative preparation of highly entangled multiparticle states of atoms coupled to common oscillator modes. By combining local spontaneous emission with coherent couplings we engineer many-body dissipation that drives the system from an arbitrary initial state into a Greenberger-Horne-Zeilinger state. We demonstrate that using our technique, highly entangled steady states can be prepared efficiently in a time that scales polynomially with the system size. Our protocol assumes generic couplings and will thus enable the dissipative production of multiparticle entanglement in a wide range of physical systems. As an example, we demonstrate the feasibility of our scheme in state-of-the-art trapped-ion systems.

quant-ph↗

arXiv2014

Positive reduction from spectra

We study the problem of whether all bipartite quantum states having a prescribed spectrum remain positive under the reduction map applied to one subsystem. We provide necessary and sufficient conditions, in the form of a family of linear inequalities, which the spectrum has to verify. Our conditions become explicit when one of the two subsystems is a qubit, as well as for further sets of states. Finally, we introduce a family of simple entanglement criteria for spectra, closely related to the reduction and positive partial transpose criteria, which also provide new insight into the set of spectra that guarantee separability or positivity of the partial transpose.

quant-ph↗