arXiv subjects

Antti Koskela

Publications and source records attributed to Antti Koskela.

33 records · Page 2Linked to original sources

arXiv2021

Differentially Private Bayesian Inference for Generalized Linear Models

Generalized linear models (GLMs) such as logistic regression are among the most widely used arms in data analyst's repertoire and often used on sensitive datasets. A large body of prior works that investigate GLMs under differential privacy (DP) constraints provide only private point estimates of the regression coefficients, and are not able to quantify parameter uncertainty. In this work, with logistic and Poisson regression as running examples, we introduce a generic noise-aware DP Bayesian inference method for a GLM at hand, given a noisy sum of summary statistics. Quantifying uncertainty allows us to determine which of the regression coefficients are statistically significantly different from zero. We provide a previously unknown tight privacy analysis and experimentally demonstrate that the posteriors obtained from our model, while adhering to strong privacy guarantees, are close to the non-private posteriors.

cs.LG↗

arXiv2020

Computing low-rank approximations of the Fréchet derivative of a matrix function using Krylov subspace methods

The Fréchet derivative $L_f(A,E)$ of the matrix function $f(A)$ plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of $L_f(A,E)$ when the direction term $E$ is of rank one (which can easily be extended to general low-rank). We analyze the convergence of the resulting method for the important special case that $A$ is Hermitian and $f$ is either the exponential, the logarithm or a Stieltjes function. In a number of numerical tests, both including matrices from benchmark collections and from real-world applications, we demonstrate and compare the accuracy and efficiency of the proposed methods.

math.NA↗

arXiv2020

Differentially private cross-silo federated learning

Strict privacy is of paramount importance in distributed machine learning. Federated learning, with the main idea of communicating only what is needed for learning, has been recently introduced as a general approach for distributed learning to enhance learning and improve security. However, federated learning by itself does not guarantee any privacy for data subjects. To quantify and control how much privacy is compromised in the worst-case, we can use differential privacy. In this paper we combine additively homomorphic secure summation protocols with differential privacy in the so-called cross-silo federated learning setting. The goal is to learn complex models like neural networks while guaranteeing strict privacy for the individual data subjects. We demonstrate that our proposed solutions give prediction accuracy that is comparable to the non-distributed setting, and are fast enough to enable learning models with millions of parameters in a reasonable time. To enable learning under strict privacy guarantees that need privacy amplification by subsampling, we present a general algorithm for oblivious distributed subsampling. However, we also argue that when malicious parties are present, a simple approach using distributed Poisson subsampling gives better privacy. Finally, we show that by leveraging random projections we can further scale-up our approach to larger models while suffering only a modest performance loss.

cs.CR↗

arXiv2020

Sampling of Stochastic Differential Equations using the Karhunen-Loève Expansion and Matrix Functions

We consider linearizations of stochastic differential equations with additive noise using the Karhunen-Loève expansion. We obtain our linearizations by truncating the expansion and writing the solution as a series of matrix-vector products using the theory of matrix functions. Moreover, we restate the solution as the solution of a system of linear differential equations. We obtain strong and weak error bounds for the truncation procedure and show that, under suitable conditions, the mean square error has order of convergence $\mathcal{O}(\frac{1}{m})$ and the second moment has a weak order of convergence $\mathcal{O}(\frac{1}{m})$, where $m$ denotes the size of the expansion. We also discuss efficient numerical linear algebraic techniques to approximate the series of matrix functions and the linearized system of differential equations. These theoretical results are supported by experiments showing the effectiveness of our algorithms when compared to standard methods such as the Euler-Maruyama scheme.

math.NA↗

arXiv2019

Computing Tight Differential Privacy Guarantees Using FFT

Differentially private (DP) machine learning has recently become popular. The privacy loss of DP algorithms is commonly reported using $(\varepsilon,δ)$-DP. In this paper, we propose a numerical accountant for evaluating the privacy loss for algorithms with continuous one dimensional output. This accountant can be applied to the subsampled multidimensional Gaussian mechanism which underlies the popular DP stochastic gradient descent. The proposed method is based on a numerical approximation of an integral formula which gives the exact $(\varepsilon,δ)$-values. The approximation is carried out by discretising the integral and by evaluating discrete convolutions using the fast Fourier transform algorithm. We give both theoretical error bounds and numerical error estimates for the approximation. Experimental comparisons with state-of-the-art techniques demonstrate significant improvements in bound tightness and/or computation time. Python code for the method can be found in Github (https://github.com/DPBayes/PLD-Accountant/).

stat.ML↗

arXiv2019

Learning Rate Adaptation for Federated and Differentially Private Learning

We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the result obtained by one full step and two half-steps. The algorithm is applied in two separate frameworks: federated and differentially private learning. Using examples of deep neural networks we empirically show that the adaptive algorithm is competitive with manually tuned commonly used optimisation methods for differentially privately training. We also show that it works robustly in the case of federated learning unlike commonly used optimisation methods.

stat.ML↗

arXiv2018

Analysis of Krylov Subspace Approximation to Large Scale Differential Riccati Equations

We consider a Krylov subspace approximation method for the symmetric differential Riccati equation $\dot{X} = AX + XA^T + Q - XSX$, $X(0)=X_0$. The method we consider is based on projecting the large scale equation onto a Krylov subspace spanned by the matrix $A$ and the low rank factors of $X_0$ and $Q$. We prove that the method is structure preserving in the sense that it preserves two important properties of the exact flow, namely the positivity of the exact flow, and also the property of monotonicity. We also provide a theoretical a priori error analysis which shows a superlinear convergence of the method. This behavior is illustrated in the numerical experiments. Moreover, we derive an efficient a posteriori error estimate as well as discuss multiple time stepping combined with a cut of the rank of the numerical solution.

math.NA↗

arXiv2017

Krylov integrators for Hamiltonian systems

We consider Arnoldi like processes to obtain symplectic subspaces for Hamiltonian systems. Large systems are locally approximated by ones living in low dimensional subspaces; we especially consider Krylov subspaces and some extensions. This will be utilized in two ways: solve numerically local small dimensional systems or in a given numerical, e.g. exponential, integrator, use the subspace for approximations of necessary functions. In the former case one can expect an excellent energy preservation. For the latter this is so for linear systems. For some second order exponential integrators we consider these two approaches are shown to be equivalent. In numerical experiments with nonlinear Hamiltonian problems their behaviour seems promising.

math.NA↗

arXiv2017

On a generalization of the Bessel function Neumann expansion

The Bessel-Neumann expansion (of integer order) of a function $g:\mathbb{C}\rightarrow\mathbb{C}$ corresponds to representing $g$ as a linear combination of basis functions $ϕ_0,ϕ_1,\ldots$, i.e., $g(z)=\sum_{\ell = 0}^\infty w_\ell ϕ_\ell(s)$, where $ϕ_i(z)=J_i(z)$, $i=0,\ldots$, are the Bessel functions. In this work, we study an expansion for a more general class of basis functions. More precisely, we assume that the basis functions satisfy an infinite dimensional linear ordinary differential equation associated with a Hessenberg matrix, motivated by the fact that these basis functions occur in certain iterative methods. A procedure to compute the basis functions as well as the coefficients is proposed. Theoretical properties of the expansion are studied. We illustrate that non-standard basis functions can give faster convergence than the Bessel functions.

math.NA↗

arXiv2017

Disguised and new Quasi-Newton methods for nonlinear eigenvalue problems

In this paper we take a quasi-Newton approach to nonlinear eigenvalue problems (NEPs) of the type $M(λ)v=0$, where $M:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a holomorphic function. We investigate which types of approximations of the Jacobian matrix lead to competitive algorithms, and provide convergence theory. The convergence analysis is based on theory for quasi-Newton methods and Keldysh's theorem for NEPs. We derive new algorithms and also show that several well-established methods for NEPs can be interpreted as quasi-Newton methods, and thereby provide insight to their convergence behavior. In particular, we establish quasi-Newton interpretations of Neumaier's residual inverse iteration and Ruhe's method of successive linear problems.

math.NA↗

arXiv2015

Splitting methods for time integration of trajectories in combined electric and magnetic fields

The equations of motion of a single particle subject to an arbitrary electric and a static magnetic field form a Poisson system. We present a second-order time integration method which preserves well the Poisson structure and compare it to commonly used algorithms, such as the Boris scheme. All the methods are represented in a general framework of splitting methods. We use the so-called $ϕ$ functions, which give efficient ways for both analyzing and implementing the algorithms. Numerical experiments show an excellent long term stability for the new method considered.

math.NA↗

arXiv2015

Krylov approximation of ODEs with polynomial parameterization

We propose a new numerical method to solve linear ordinary differential equations of the type $\frac{\partial u}{\partial t}(t,\varepsilon) = A(\varepsilon) \, u(t,\varepsilon)$, where $A:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a matrix polynomial with large and sparse matrix coefficients. The algorithm computes an explicit parameterization of approximations of $u(t,\varepsilon)$ such that approximations for many different values of $\varepsilon$ and $t$ can be obtained with a very small additional computational effort. The derivation of the algorithm is based on a reformulation of the parameterization as a linear parameter-free ordinary differential equation and on approximating the product of the matrix exponential and a vector with a Krylov method. The Krylov approximation is generated with Arnoldi's method and the structure of the coefficient matrix turns out to have an independence on the truncation parameter so that it can also be interpreted as Arnoldi's method applied to an infinite dimensional matrix. We prove the superlinear convergence of the algorithm and provide a posteriori error estimates to be used as termination criteria. The behavior of the algorithm is illustrated with examples stemming from spatial discretizations of partial differential equations.

math.NA↗

arXiv2015

The infinite Arnoldi exponential integrator for linear inhomogeneous ODEs

Exponential integrators that use Krylov approximations of matrix functions have turned out to be efficient for the time-integration of certain ordinary differential equations (ODEs). This holds in particular for linear homogeneous ODEs, where the exponential integrator is equivalent to approximating the product of the matrix exponential and a vector. In this paper, we consider linear inhomogeneous ODEs, $y'(t)=Ay(t)+g(t)$, where the function $g(t)$ is assumed to satisfy certain regularity conditions. We derive an algorithm for this problem which is equivalent to approximating the product of the matrix exponential and a vector using Arnoldi's method. The construction is based on expressing the function $g(t)$ as a linear combination of given basis functions $[ϕ_i]_{i=0}^\infty$ with particular properties. The properties are such that the inhomogeneous ODE can be restated as an infinite-dimensional linear homogeneous ODE. Moreover, the linear homogeneous infinite-dimensional ODE has properties that directly allow us to extend a Krylov method for finite-dimensional linear ODEs. Although the construction is based on an infinite-dimensional operator, the algorithm can be carried out with operations involving matrices and vectors of finite size. This type of construction resembles in many ways the infinite Arnoldi method for nonlinear eigenvalue problems. We prove convergence of the algorithm under certain natural conditions, and illustrate properties of the algorithm with examples stemming from the discretization of partial differential equations.

math.NA↗

arXiv2013

Differential Geometrically Consistent Artificial Viscosity in Comoving Curvilinear Coordinates

Context. High-resolution numerical methods have been developed for nonlinear, discontinuous problems as they appear in simulations of astrophysical objects. One of the strategies applied is the concept of artificial viscosity. Aims. Grid-based numerical simulations ideally utilize problem-oriented grids in order to minimize the necessary number of cells at a given (desired) spatial resolution. We want to propose a modified tensor of artificial viscosity which is employable for generally comoving, curvilinear grids. Methods. We study a differential geometrically consistent artificial viscosity analytically and visualize a comparison of our result to previous implementations by applying it to a simple self-similar velocity field. We give a general introduction to artificial viscosity first and motivate its application in numerical analysis. Then we present how a tensor of artificial viscosity has to be designed when going beyond common static Eulerian or Lagrangian comoving rectangular grids. Results. We find that in comoving, curvilinear coordinates the isotropic (pressure) part of the tensor of artificial viscosity has to be modified metrically in order for it to fulfill all its desired properties.

astro-ph.IM↗

arXiv2011

Investigation of the recombination of the retarded shell of "born-again" CSPNe by time-dependent radiative transfer models

A standard planetary nebula stays more than 10 000 years in the state of a photoionized nebula. As long as the timescales of the most important ionizing processes are much smaller, the ionization state can be characterized by a static photoionization model and simulated with codes like CLOUDY (Ferland et al. 1998). When the star exhibits a late Helium flash, however, its ionizing flux stops within a very short period. The star then re-appears from itsopaque shell after a few years (or centuries) as a cold giant star without any hard ionizing photons. Describing the physics of such behavior requires a fully time-dependent radiative transfer model. Pollacco (1999), Kerber et al. (1999) and Lechner & Kimeswenger (2004) used data of the old nebulae around V605 Aql and V4334 Sgr to derive a model of the pre-outburst state of the CSPN in a static model. Their argument was the long recombination time scale for such thin media. With regard to these models Schoenberner (2008) critically raised the question whether a significant change in the ionization state (and thus the spectrum) has to be expected after a time of up to 80 years, and whether static models are applicable at all.

astro-ph.GA↗