arXiv subjects

Ittai Rubinstein

Publications and source records attributed to Ittai Rubinstein.

12 recordsLinked to original sources

arXiv2025

On the Accuracy of Newton Step and Influence Function Data Attributions

Data attribution aims to explain model predictions by estimating how they would change if certain training points were removed, and is used in a wide range of applications, from interpretability and credit assignment to unlearning and privacy. Even in the relatively simple case of logistic regressions, existing mathematical analyses of leading data attribution methods such as Influence Functions (IF) and single Newton Step (NS) remain limited in two key ways. First, they rely on global strong convexity assumptions which are often not satisfied in practice. Second, the resulting bounds scale very poorly with the number of parameters ($d$) and the number of samples removed ($k$). As a result, these analyses are not tight enough to answer fundamental questions such as "what is the asymptotic scaling of the errors of each method?" or "which of these methods is more accurate for a given dataset?" In this paper, we introduce a new analysis of the NS and IF data attribution methods for convex learning problems. To the best of our knowledge, this is the first analysis of these questions that does not assume global strong convexity and also the first explanation of [KATL19] and [RH25a]'s observation that NS data attribution is often more accurate than IF. We prove that for sufficiently well-behaved logistic regressions, our bounds are asymptotically tight up to poly-logarithmic factors, yielding scaling laws for the errors in the average-case sample removals. \[ \mathbb{E}_{T \subseteq [n],\, |T| = k} \bigl[ \|\hat{\theta}_T - \hat{\theta}_T^{\mathrm{NS}}\|_2 \bigr] = \widetilde{\Theta}\!\left(\frac{k d}{n^2}\right), \qquad \mathbb{E}_{T \subseteq [n],\, |T| = k} \bigl[ \|\hat{\theta}_T^{\mathrm{NS}} - \hat{\theta}_T^{\mathrm{IF}}\|_2 \bigr] = \widetilde{\Theta}\!\left( \frac{(k + d)\sqrt{k d}}{n^2} \right). \]

cs.LG↗

arXiv2025

Reliable high-accuracy error mitigation for utility-scale quantum circuits

Error mitigation is essential for unlocking the full potential of quantum algorithms and accelerating the timeline toward quantum advantage. As quantum hardware progresses to push the boundaries of classical simulation, efficient and robust error mitigation methods are becoming increasingly important for producing accurate and reliable outputs. However, existing error-mitigation approaches face a fundamental tradeoff between practical performance and reliability: heuristic methods such as zero-noise extrapolation (ZNE) enjoy faster runtime but lack accuracy guarantees, while rigorous techniques such as probabilistic error cancellation (PEC) provide unbiased estimates at prohibitive computational cost. We introduce a characterization-based, rigorously-grounded quantum error mitigation and error suppression framework (QESEM) that resolves this tradeoff by leveraging the accuracy guarantees of quasi-probabilistic mitigation with dramatically reduced overhead. We explain the innovative methods underlying QESEM and demonstrate its capabilities in the largest utility-scale error mitigation experiment based on an unbiased method. This experiment simulates the kicked transverse field Ising model with far-from-Clifford parameters on an IBM Heron device. We further validate QESEM's versatility across arbitrary quantum circuits and devices through high-accuracy error-mitigated molecular VQE circuits executed on IBM Heron and IonQ trapped-ion devices. Compared with multiple variants of the widely used zero-noise extrapolation method, QESEM consistently achieves higher accuracy while avoiding the prohibitive runtime overhead associated with PEC. These results mark a significant step forward in accuracy and reliability for running quantum circuits on current devices across diverse applications. Finally, we provide projections of QESEM's performance on near-term devices toward quantum advantage.

quant-ph↗

arXiv2025

Rescaled Influence Functions: Accurate Data Attribution in High Dimension

How does the training data affect a model's behavior? This is the question we seek to answer with data attribution. The leading practical approaches to data attribution are based on influence functions (IF). IFs utilize a first-order Taylor approximation to efficiently predict the effect of removing a set of samples from the training set without retraining the model, and are used in a wide variety of machine learning applications. However, especially in the high-dimensional regime (# params $\geq \Omega($# samples$)$), they are often imprecise and tend to underestimate the effect of sample removals, even for simple models such as logistic regression. We present rescaled influence functions (RIF), a new tool for data attribution which can be used as a drop-in replacement for influence functions, with little computational overhead but significant improvement in accuracy. We compare IF and RIF on a range of real-world datasets, showing that RIFs offer significantly better predictions in practice, and present a theoretical analysis explaining this improvement. Finally, we present a simple class of data poisoning attacks that would fool IF-based detections but would be detected by RIF.

cs.LG↗

arXiv2024

The Quasi-probability Method and Applications for Trace Reconstruction

In the trace reconstruction problem, one attempts to reconstruct a fixed but unknown string $x$ of length $n$ from a given number of traces $\tilde{x}$ drawn iid from the application of a noisy process (such as the deletion channel) to $x$. The best known algorithm for the trace reconstruction from the deletion channel is due to Chase, and recovers the input string whp given $\exp(\tilde{O}(n^{1/5}))$ traces [Cha21b]. The main component in Chase's algorithm is a procedure for k-mer estimation, which, for any marker $w$ in $\{0, 1\}^k$ of length $k$, computes a "smoothed" distribution of its appearances in the input string $x$ [CGL+23, MS24]. Current k-mer estimation algorithms fail when the deletion probability is above $1/2$, requiring a more complex analysis for Chase's algorithm. Moreover, the only known extension of these approaches beyond the deletion channels is based on numerically estimating high-order differentials of a multivariate polynomial, making it highly impractical [Rub23]. In this paper, we construct a simple Monte Carlo method for k-mer estimation which can be easily applied to a much wider variety of channels. In particular, we solve k-mer estimation for any combination of insertion, deletion, and bit-flip channels, even in the high deletion probability regime, allowing us to directly apply Chase's algorithm for this wider class of channels. To accomplish this, we utilize an approach from the field of quantum error mitigation (the process of using many measurements from noisy quantum computers to simulate a clean quantum computer), called the quasi-probability method (also known as probabilistic error cancellation) [TBG17, PSW22]. We derive a completely classical version of this technique, and use it to construct a k-mer estimation algorithm. No background in quantum computing is needed to understand this paper.

cs.DS↗

arXiv2024

Robustness Auditing for Linear Regression: To Singularity and Beyond

It has recently been discovered that the conclusions of many highly influential econometrics studies can be overturned by removing a very small fraction of their samples (often less than $0.5\%$). These conclusions are typically based on the results of one or more Ordinary Least Squares (OLS) regressions, raising the question: given a dataset, can we certify the robustness of an OLS fit on this dataset to the removal of a given number of samples? Brute-force techniques quickly break down even on small datasets. Existing approaches which go beyond brute force either can only find candidate small subsets to remove (but cannot certify their non-existence) [BGM20, KZC21], are computationally intractable beyond low dimensional settings [MR22], or require very strong assumptions on the data distribution and too many samples to give reasonable bounds in practice [BP21, FH23]. We present an efficient algorithm for certifying the robustness of linear regressions to removals of samples. We implement our algorithm and run it on several landmark econometrics datasets with hundreds of dimensions and tens of thousands of samples, giving the first non-trivial certificates of robustness to sample removal for datasets of dimension $4$ or greater. We prove that under distributional assumptions on a dataset, the bounds produced by our algorithm are tight up to a $1 + o(1)$ multiplicative factor.

cs.LG↗

arXiv2023

Improved Upper and Lower Bounds on the Capacity of the Binary Deletion Channel

The {\em binary deletion channel} with deletion probability $d$ ($\text{BDC}_d$) is a random channel that deletes each bit of the input message i.i.d with probability $d$. It has been studied extensively as a canonical example of a channel with synchronization errors. Perhaps the most important question regarding the BDC is determining its capacity. Mitzenmacher and Drinea (ITIT 2006) and Kirsch and Drinea (ITIT 2009) show a method by which distributions on run lengths can be converted to codes for the BDC, yielding a lower bound of $\mathcal{C}(\text{BDC}_d) > 0.1185 \cdot (1-d)$. Fertonani and Duman (ITIT 2010), Dalai (ISIT 2011) and Rahmati and Duman (ITIT 2014) use computer aided analyses based on the Blahut-Arimoto algorithm to prove an upper bound of $\mathcal{C}(\text{BDC}_d) < 0.4143\cdot(1-d)$ in the high deletion probability regime ($d > 0.65$). In this paper, we show that the Blahut-Arimoto algorithm can be implemented with a lower space complexity, allowing us to extend the upper bound analyses, and prove an upper bound of $\mathcal{C}(\text{BDC}_d) < 0.3745 \cdot(1-d)$ for all $d \geq 0.68$. Furthermore, we show that an extension of the Blahut-Arimoto algorithm can also be used to select better run length distributions for Mitzenmacher and Drinea's construction, yielding a lower bound of $\mathcal{C}(\text{BDC}_d) > 0.1221 \cdot (1 - d)$.

cs.IT↗

arXiv2022

Average-Case to (shifted) Worst-Case Reduction for the Trace Reconstruction Problem

The {\em insertion-deletion channel} takes as input a binary string $x \in\{0, 1\}^n$, and outputs a string $\widetilde{x}$ where some of the bits have been deleted and others inserted independently at random. In the {\em trace reconstruction problem}, one is given many outputs (called {\em traces}) of the insertion-deletion channel on the same input message $x$, and asked to recover the input message. Nazarov and Peres (STOC 2017), and De, O'Donnell and Servedio (STOC 2017) showed that any string $x$ can be reconstructed from $\exp(O(n^{1/3}))$ traces. Holden, Pemantle, Peres and Zhai (COLT 2018) adapt the techniques used to prove this upper bound, to an algorithm for the average-case trace reconstruction with a sample complexity of $\exp(O(\log^{1/3} n))$. However, it is not clear how to apply their techniques more generally and in particular for the recent worst-case upper bound of $\exp(\widetilde{O}(n^{1/5}))$ shown by Chase (STOC 2021) for the deletion-channel. We prove a general reduction from the average-case to smaller instances of a problem similar to worst-case. Using this reduction and a generalization of Chase's bound, we construct an improved average-case algorithm with a sample complexity of $\exp(\widetilde{O}(\log^{1/5} n))$. Additionally, we show that Chase's upper-bound holds for the insertion-deletion channel as well.

cs.IT↗

arXiv2022

Explicit and Efficient Construction of (nearly) Optimal Rate Codes for Binary Deletion Channel and the Poisson Repeat Channel

Two of the most common models for channels with synchronisation errors are the Binary Deletion Channel with parameter $p$ ($\text{BDC}_p$) -- a channel where every bit of the codeword is deleted i.i.d with probability $p$, and the Poisson Repeat Channel with parameter $λ$ ($\text{PRC}_λ$) -- a channel where every bit of the codeword is repeated $\text{Poisson}(λ)$ times. Previous constructions based on synchronisation strings yielded codes with rates far lower than the capacities of these channels [CS19, GL18], and the only efficient construction to achieve capacity on the BDC at the time of writing this paper is based on the far more advanced methods of polar codes [TPFV21]. In this work, we present a new method for concatenating synchronisation codes and use it to construct simple and efficient encoding and decoding algorithms for both channels with nearly optimal rates.

cs.IT↗

arXiv2022

Multivariate Generating Functions for Information Spread on Multi-Type Random Graphs

We study the spread of information on multi-type directed random graphs. In such graphs the vertices are partitioned into distinct types (communities) that have different transmission rates between themselves and with other types. We construct multivariate generating functions and use multi-type branching processes to derive an equation for the size of the large out-components in multi-type random graphs with a general class of degree distributions. We use our methods to analyse the spread of epidemics and verify the results with population based simulations

cond-mat.stat-mech↗

arXiv2021

Heterogeneity and Superspreading Effect on Herd Immunity

We model and calculate the fraction of infected population necessary to reach herd immunity, taking into account the heterogeneity in infectiousness and susceptibility, as well as the correlation between those two parameters. We show that these cause the effective reproduction number to decrease more rapidly, and consequently have a drastic effect on the estimate of the necessary percentage of the population that has to contract the disease for herd immunity to be reached. We quantify the difference between the size of the infected population when the effective reproduction number decreases below 1 vs. the ultimate fraction of population that had contracted the disease. This sheds light on an important distinction between herd immunity and the end of the disease and highlights the importance of limiting the spread of the disease even if we plan to naturally reach herd immunity. We analyze the effect of various lock-down scenarios on the resulting final fraction of infected population. We discuss implications to COVID-19 and other pandemics and compare our theoretical results to population-based simulations. We consider the dependence of the disease spread on the architecture of the infectiousness graph and analyze different graph architectures and the limitations of the graph models.

q-bio.PE↗

arXiv2020

Superspreaders and High Variance Infectious Diseases

A well-known characteristic of pandemics such as COVID-19 is the high level of transmission heterogeneity in the infection spread: not all infected individuals spread the disease at the same rate and some individuals (superspreaders) are responsible for most of the infections. To quantify this phenomenon requires the analysis of the effect of the variance and higher moments of the infection distribution. Working in the framework of stochastic branching processes, we derive an approximate analytical formula for the probability of an outbreak in the high variance regime of the infection distribution, verify it numerically and analyze its regime of validity in various examples. We show that it is possible for an outbreak not to occur in the high variance regime even when the basic reproduction number $R_0$ is larger than one and discuss the implications of our results for COVID-19 and other pandemics.

q-bio.PE↗

arXiv2019

Deep learning reconstruction of ultrashort pulses from 2D spatial intensity patterns recorded by an all-in-line system in a single-shot

We propose a simple all-in-line single-shot scheme for diagnostics of ultrashort laser pulses, consisting of a multi-mode fiber, a nonlinear crystal and a CCD camera. The system records a 2D spatial intensity pattern, from which the pulse shape (amplitude and phase) are recovered, through a fast Deep Learning algorithm. We explore this scheme in simulations and demonstrate the recovery of ultrashort pulses, robustness to noise in measurements and to inaccuracies in the parameters of the system components. Our technique mitigates the need for commonly used iterative optimization reconstruction methods, which are usually slow and hampered by the presence of noise. These features make our concept system advantageous for real time probing of ultrafast processes and noisy conditions. Moreover, this work exemplifies that using deep learning we can unlock new types of systems for pulse recovery.

physics.optics↗