arXiv subjects

Omri Lev

Publications and source records attributed to Omri Lev.

9 recordsLinked to original sources

arXiv2026

The Fast Mixing Mechanism for Differential Privacy

Randomized sketching is a central tool for compressing large-scale optimization problems while preserving accuracy. In particular, sketches that are based on structured matrices, such as the Hadamard matrix, can be applied efficiently and often yield solutions that approximate those of the original problem at much lower computational cost. In differential privacy (DP), Gaussian sketching has been used to solve DP linear regression, beginning with \citet{sheffet2017differentially, sheffet2019old} and later refined by \citet{lev2025gaussianmix, lev2026near}. However, although these methods achieve strong utility guarantees, they usually do not improve runtime over classical DP approaches. In this work, we introduce a new DP sketching mechanism based on fast transforms, which, in certain cases, matches the runtime of classical fast sketching methods. We prove state-of-the-art privacy guarantees for this mechanism and show that, in favorable regimes, they match those of the Gaussian sketch up to a constant factor. As an application, we combine this mechanism with recent sketch-based methods for DP linear regression to obtain a new algorithm with strong utility and improved runtime. We establish privacy and accuracy guarantees for this algorithm, yielding, to the best of our knowledge, the first fast method for DP ordinary least squares.

cs.LG↗

arXiv2026

Near-Optimal Private Linear Regression via Iterative Hessian Mixing

We study differentially private ordinary least squares (DP-OLS) with bounded data $(X,Y)$ via sketching-based mechanisms. While Gaussian sketching approaches have been explored for DP-OLS \citep{sheffet2017differentially}, they are typically viewed as less competitive than the Adaptive Sufficient Statistics Perturbation (AdaSSP) method \citep{wang_adassp}, which directly perturbs the sufficient statistics $(X^{\top}X, X^{\top}Y)$. This method was shown to be close to information-theoretically optimal, while also exhibiting strong empirical performance. In this work, we propose the \emph{Iterative Hessian Mixing} (IHM), an algorithm that builds on Gaussian sketching approaches to DP-OLS and is inspired by the Iterative Hessian Sketch of \citet{pilanci_hessiansketch}. We prove that IHM is differentially private and provide utility guarantees in the form of excess empirical risk bounds. These bounds improve upon those of AdaSSP by removing a multiplicative factor that can be as large as the square root of the data dimension. The design of the IHM is based on new accuracy guarantees that we present for prior Gaussian sketching approaches for DP-OLS, which clarify when these methods are expected to perform well and how IHM circumvents their inherent limitations. We also conduct a rigorous empirical evaluation on a large suite of datasets, demonstrating that IHM consistently outperforms prior baselines, including AdaSSP.

cs.LG↗

arXiv2025

The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches

Gaussian sketching, which consists of pre-multiplying the data with a random Gaussian matrix, is a widely used technique for multiple problems in data science and machine learning, with applications spanning computationally efficient optimization, coded computing, and federated learning. This operation also provides differential privacy guarantees due to its inherent randomness. In this work, we revisit this operation through the lens of Renyi Differential Privacy (RDP), providing a refined privacy analysis that yields significantly tighter bounds than prior results. We then demonstrate how this improved analysis leads to performance improvement in different linear regression settings, establishing theoretical utility guarantees. Empirically, our methods improve performance across multiple datasets and, in several cases, reduce runtime.

cs.LG↗

arXiv2024

The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models

Quantifying the influence of infinitesimal changes in training data on model performance is crucial for understanding and improving machine learning models. In this work, we reformulate this problem as a weighted empirical risk minimization and enhance existing influence function-based methods by using information geometry to derive a new algorithm to estimate influence. Our formulation proves versatile across various applications, and we further demonstrate in simulations how it remains informative even in non-convex cases. Furthermore, we show that our method offers significant computational advantages over current Newton step-based methods.

cs.LG↗

arXiv2021

Universal Joint Source-Channel Coding Under an Input Energy Constraint

We consider the problem of transmitting a source over an infinite-bandwidth additive white Gaussian noise channel with unknown noise level under an input energy constraint. We construct a universal scheme that uses modulo-lattice modulation with multiple layers; for each layer, we employ either analog linear modulation or analog pulse position modulation (PPM). We show that the designed scheme with linear layers requires less energy compared to existing solutions to achieve the same quadratically increasing distortion profile with the noise level; replacing the linear layers with PPM layers offers an additional improvement.

cs.IT↗

arXiv2021

Energy-limited Joint Source--Channel Coding via Analog Pulse Position Modulation

We study the problem of transmitting a source sample with minimum distortion over an infinite-bandwidth additive white Gaussian noise channel under an energy constraint. To that end, we construct a joint source--channel coding scheme using analog pulse position modulation (PPM) and bound its quadratic distortion. We show that this scheme outperforms existing techniques since its quadratic distortion attains both the exponential and polynomial decay orders of Burnashev's outer bound. We supplement our theoretical results with numerical simulations and comparisons to existing schemes.

cs.IT↗

arXiv2020

Gauss-Markov Source Tracking with Side Information: Lower Bounds

We consider the problem of causal source coding and causal decoding of a Gauss--Markov source, where the decoder has causal access to a side-information signal. We define the information causal rate-distortion function with causal decoder side information and prove that it bounds from below its operational counterpart. We further explain how to adapt the result to the setting of control over communication channels.

cs.IT↗

arXiv2020

Schemes for LQG Control over Gaussian Channels with Side Information

We consider the problem of controlling an unstable scalar linear plant over a power-constrained additive white Gaussian noise (AWGN) channel, where the controller/receiver has access to an additional noisy measurement of the state of the control system. To that end, we view the noisy measurement as side information and recast the problem to that of joint source-channel coding with side information at the receiver. We argue that judicious modulo-based schemes improve over their linear counterparts and allow to avoid a large increase in the transmit power due to the ignorance of the side information at the sensor/transmitter. We demonstrate the usefulness of our technique for the settings where i) the sensor is oblivious of the control objectives, control actions and previous controller state estimates, ii) the system output tracks a desired reference signal that is available only at the controller via integral control action.

eess.SY↗

arXiv2016

WINDOW: Wideband Demodulator for Optical Waveforms

Optical communication systems, which operate at very high rates, are often limited by the sampling rate bottleneck. The optical wideband regime may exceed analog to digital converters (ADCs) front-end bandwidth. Multi-channel sampling approaches, such as multicoset or interleaved ADCs, have been proposed to sample the wideband signal using several channels. Each channel samples below the Nyquist rate such that the overall sampling rate is preserved. However, this scheme suffers from two practical limitations that make its implementation difficult. First, the inherent anti-aliasing filter of the samplers distorts the wideband signal. Second, it requires accurate time shifts on the order of the signal's Nyquist rate, which are challenging to maintain. In this work, we propose an alternative multi-channel sampling scheme, the wideband demodulator for optical waveforms (WINDOW), based on analog RF demodulation, where each channel aliases the spectrum using a periodic mixing function before integration and sampling. We show that intentionally using the inherent ADC filter to perform integration increases the signal to noise ratio (SNR). We demonstrate both theoretically and through numerical experiments that our system outperforms multicoset in terms of signal recovery and symbol estimation in the presence of both thermal and quantization noise but is slightly less robust to timing jitter.

cs.IT↗