arXiv subjects

Sarel Cohen

Publications and source records attributed to Sarel Cohen.

30 records · Page 2Linked to original sources

arXiv2023

Fast Feature Selection with Fairness Constraints

We study the fundamental problem of selecting optimal features for model construction. This problem is computationally challenging on large datasets, even with the use of greedy algorithm variants. To address this challenge, we extend the adaptive query model, recently proposed for the greedy forward selection for submodular functions, to the faster paradigm of Orthogonal Matching Pursuit for non-submodular functions. The proposed algorithm achieves exponentially fast parallel run time in the adaptive query model, scaling much better than prior work. Furthermore, our extension allows the use of downward-closed constraints, which can be used to encode certain fairness criteria into the feature selection process. We prove strong approximation guarantees for the algorithm based on standard assumptions. These guarantees are applicable to many parametric models, including Generalized Linear Models. Finally, we demonstrate empirically that the proposed algorithm competes favorably with state-of-the-art techniques for feature selection, on real-world and synthetic datasets.

cs.LG↗

arXiv2022

Deterministic Sensitivity Oracles for Diameter, Eccentricities and All Pairs Distances

We construct data structures for extremal and pairwise distances in directed graphs in the presence of transient edge failures. Henzinger et al. [ITCS 2017] initiated the study of fault-tolerant (sensitivity) oracles for the diameter and vertex eccentricities. We extend this with a special focus on space efficiency. We present several new data structures, among them the first fault-tolerant eccentricity oracle for dual failures in subcubic space. We further prove lower bounds that show limits to approximation vs. space and diameter vs. space trade-offs for fault-tolerant oracles. They highlight key differences between data structures for undirected and directed graphs. Initially, our oracles are randomized leaning on a sampling technique frequently used in sensitivity analysis. Building on the work of Alon, Chechik, and Cohen [ICALP 2019] as well as Karthik and Parter [SODA 2021], we develop a hierarchical framework to derandomize fault-tolerant data structures. We first apply it to our own diameter and eccentricity oracles and then show its versatility by derandomizing algorithms from the literature: the distance sensitivity oracle of Ren [JCSS 2022] and the Single-Source Replacement Path algorithm of Chechik and Magen [ICALP 2020]. This way, we obtain the first deterministic distance sensitivity oracle with subcubic preprocessing time.

cs.DS↗

arXiv2022

Non-Volatile Memory Accelerated Posterior Estimation

Bayesian inference allows machine learning models to express uncertainty. Current machine learning models use only a single learnable parameter combination when making predictions, and as a result are highly overconfident when their predictions are wrong. To use more learnable parameter combinations efficiently, these samples must be drawn from the posterior distribution. Unfortunately computing the posterior directly is infeasible, so often researchers approximate it with a well known distribution such as a Gaussian. In this paper, we show that through the use of high-capacity persistent storage, models whose posterior distribution was too big to approximate are now feasible, leading to improved predictions in downstream tasks.

cs.LG↗

arXiv2022

Non-Volatile Memory Accelerated Geometric Multi-Scale Resolution Analysis

Dimensionality reduction algorithms are standard tools in a researcher's toolbox. Dimensionality reduction algorithms are frequently used to augment downstream tasks such as machine learning, data science, and also are exploratory methods for understanding complex phenomena. For instance, dimensionality reduction is commonly used in Biology as well as Neuroscience to understand data collected from biological subjects. However, dimensionality reduction techniques are limited by the von-Neumann architectures that they execute on. Specifically, data intensive algorithms such as dimensionality reduction techniques often require fast, high capacity, persistent memory which historically hardware has been unable to provide at the same time. In this paper, we present a re-implementation of an existing dimensionality reduction technique called Geometric Multi-Scale Resolution Analysis (GMRA) which has been accelerated via novel persistent memory technology called Memory Centric Active Storage (MCAS). Our implementation uses a specialized version of MCAS called PyMM that provides native support for Python datatypes including NumPy arrays and PyTorch tensors. We compare our PyMM implementation against a DRAM implementation, and show that when data fits in DRAM, PyMM offers competitive runtimes. When data does not fit in DRAM, our PyMM implementation is still able to process the data.

cs.LG↗

arXiv2022

What's Wrong with Deep Learning in Tree Search for Combinatorial Optimization

Combinatorial optimization lies at the core of many real-world problems. Especially since the rise of graph neural networks (GNNs), the deep learning community has been developing solvers that derive solutions to NP-hard problems by learning the problem-specific solution structure. However, reproducing the results of these publications proves to be difficult. We make three contributions. First, we present an open-source benchmark suite for the NP-hard Maximum Independent Set problem, in both its weighted and unweighted variants. The suite offers a unified interface to various state-of-the-art traditional and machine learning-based solvers. Second, using our benchmark suite, we conduct an in-depth analysis of the popular guided tree search algorithm by Li et al. [NeurIPS 2018], testing various configurations on small and large synthetic and real-world graphs. By re-implementing their algorithm with a focus on code quality and extensibility, we show that the graph convolution network used in the tree search does not learn a meaningful representation of the solution structure, and can in fact be replaced by random values. Instead, the tree search relies on algorithmic techniques like graph kernelization to find good solutions. Thus, the results from the original publication are not reproducible. Third, we extend the analysis to compare the tree search implementations to other solvers, showing that the classical algorithmic solvers often are faster, while providing solutions of similar quality. Additionally, we analyze a recent solver based on reinforcement learning and observe that for this solver, the GNN is responsible for the competitive solution quality.

cs.LG↗

arXiv2021

Fixed-Parameter Sensitivity Oracles

We combine ideas from distance sensitivity oracles (DSOs) and fixed-parameter tractability (FPT) to design sensitivity oracles for FPT graph problems. An oracle with sensitivity $f$ for an FPT problem $Π$ on a graph $G$ with parameter $k$ preprocesses $G$ in time $O(g(f,k) \cdot \textsf{poly}(n))$. When queried with a set $F$ of at most $f$ edges of $G$, the oracle reports the answer to the $Π$-with the same parameter $k$-on the graph $G-F$, i.e., $G$ deprived of $F$. The oracle should answer queries in a time that is significantly faster than merely running the best-known FPT algorithm on $G-F$ from scratch. We mainly design sensitivity oracles for the $k$-Path and the $k$-Vertex Cover problem. Following our line of research connecting fault-tolerant FPT and shortest paths problems, we also introduce parameterization to the computation of distance preservers. We study the problem, given a directed unweighted graph with a fixed source $s$ and parameters $f$ and $k$, to construct a polynomial-sized oracle that efficiently reports, for any target vertex $v$ and set $F$ of at most $f$ edges, whether the distance from $s$ to $v$ increases at most by an additive term of $k$ in $G-F$.

cs.DS↗

arXiv2021

Space-Efficient Fault-Tolerant Diameter Oracles

We design $f$-edge fault-tolerant diameter oracles ($f$-FDOs). We preprocess a given graph $G$ on $n$ vertices and $m$ edges, and a positive integer $f$, to construct a data structure that, when queried with a set $F$ of $|F| \leq f$ edges, returns the diameter of $G-F$. For a single failure ($f=1$) in an unweighted directed graph of diameter $D$, there exists an approximate FDO by Henzinger et al. [ITCS 2017] with stretch $(1+\varepsilon)$, constant query time, space $O(m)$, and a combinatorial preprocessing time of $\widetilde{O}(mn + n^{1.5} \sqrt{Dm/\varepsilon})$.We present an FDO for directed graphs with the same stretch, query time, and space. It has a preprocessing time of $\widetilde{O}(mn + n^2/\varepsilon)$. The preprocessing time nearly matches a conditional lower bound for combinatorial algorithms, also by Henzinger et al. With fast matrix multiplication, we achieve a preprocessing time of $\widetilde{O}(n^{2.5794} + n^2/\varepsilon)$. We further prove an information-theoretic lower bound showing that any FDO with stretch better than $3/2$ requires $Ω(m)$ bits of space. For multiple failures ($f>1$) in undirected graphs with non-negative edge weights, we give an $f$-FDO with stretch $(f+2)$, query time $O(f^2\log^2{n})$, $\widetilde{O}(fn)$ space, and preprocessing time $\widetilde{O}(fm)$. We complement this with a lower bound excluding any finite stretch in $o(fn)$ space. We show that for unweighted graphs with polylogarithmic diameter and up to $f = o(\log n/ \log\log n)$ failures, one can swap approximation for query time and space. We present an exact combinatorial $f$-FDO with preprocessing time $mn^{1+o(1)}$, query time $n^{o(1)}$, and space $n^{2+o(1)}$. When using fast matrix multiplication instead, the preprocessing time can be improved to $n^{ω+o(1)}$, where $ω< 2.373$ is the matrix multiplication exponent.

cs.DS↗

arXiv2021

Near-Optimal Deterministic Single-Source Distance Sensitivity Oracles

Given a graph with a source vertex $s$, the Single Source Replacement Paths (SSRP) problem is to compute, for every vertex $t$ and edge $e$, the length $d(s,t,e)$ of a shortest path from $s$ to $t$ that avoids $e$. A Single-Source Distance Sensitivity Oracle (Single-Source DSO) is a data structure that answers queries of the form $(t,e)$ by returning the distance $d(s,t,e)$. We show how to deterministically compress the output of the SSRP problem on $n$-vertex, $m$-edge graphs with integer edge weights in the range $[1,M]$ into a Single-Source DSO of size $O(M^{1/2}n^{3/2})$ with query time $\widetilde{O}(1)$. The space requirement is optimal (up to the word size) and our techniques can also handle vertex failures. Chechik and Cohen [SODA 2019] presented a combinatorial, randomized $\widetilde{O}(m\sqrt{n}+n^2)$ time SSRP algorithm for undirected and unweighted graphs. Grandoni and Vassilevska Williams [FOCS 2012, TALG 2020] gave an algebraic, randomized $\widetilde{O}(Mn^ω)$ time SSRP algorithm for graphs with integer edge weights in the range $[1,M]$, where $ω<2.373$ is the matrix multiplication exponent. We derandomize both algorithms for undirected graphs in the same asymptotic running time and apply our compression to obtain deterministic Single-Source DSOs. The $\widetilde{O}(m\sqrt{n}+n^2)$ and $\widetilde{O}(Mn^ω)$ preprocessing times are polynomial improvements over previous $o(n^2)$-space oracles. On sparse graphs with $m=O(n^{5/4-\varepsilon}/M^{7/4})$ edges, for any constant $\varepsilon > 0$, we reduce the preprocessing to randomized $\widetilde{O}(M^{7/8}m^{1/2}n^{11/8})=O(n^{2-\varepsilon/2})$ time. This is the first truly subquadratic time algorithm for building Single-Source DSOs on sparse graphs.

cs.DS↗

arXiv2020

ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation

Optical character recognition (OCR) systems performance have improved significantly in the deep learning era. This is especially true for handwritten text recognition (HTR), where each author has a unique style, unlike printed text, where the variation is smaller by design. That said, deep learning based HTR is limited, as in every other task, by the number of training examples. Gathering data is a challenging and costly task, and even more so, the labeling task that follows, of which we focus here. One possible approach to reduce the burden of data annotation is semi-supervised learning. Semi supervised methods use, in addition to labeled data, some unlabeled samples to improve performance, compared to fully supervised ones. Consequently, such methods may adapt to unseen images during test time. We present ScrabbleGAN, a semi-supervised approach to synthesize handwritten text images that are versatile both in style and lexicon. ScrabbleGAN relies on a novel generative model which can generate images of words with an arbitrary length. We show how to operate our approach in a semi-supervised manner, enjoying the aforementioned benefits such as performance boost over state of the art supervised HTR. Furthermore, our generator can manipulate the resulting text style. This allows us to change, for instance, whether the text is cursive, or how thin is the pen stroke.

cs.CV↗

arXiv2019

Deterministic Combinatorial Replacement Paths and Distance Sensitivity Oracles

In this work we derandomize two central results in graph algorithms, replacement paths and distance sensitivity oracles (DSOs) matching in both cases the running time of the randomized algorithms. For the replacement paths problem, let G = (V,E) be a directed unweighted graph with n vertices and m edges and let P be a shortest path from s to t in G. The {\sl replacement paths} problem is to find for every edge e \in P the shortest path from s to t avoiding e. Roditty and Zwick [ICALP 2005] obtained a randomized algorithm with running time of ~O(m \sqrt{n}). Here we provide the first deterministic algorithm for this problem, with the same ~O(m \sqrt{n}) time. For the problem of distance sensitivity oracles, let G = (V,E) be a directed graph with real-edge weights. An f-Sensitivity Distance Oracle (f-DSO) gets as input the graph G=(V,E) and a parameter f, preprocesses it into a data-structure, such that given a query (s,t,F) with s,t \in V and F \subseteq E \cup V, |F| \le f being a set of at most f edges or vertices (failures), the query algorithm efficiently computes the distance from s to t in the graph G \setminus F ({\sl i.e.}, the distance from s to t in the graph G after removing from it the failing edges and vertices F). For weighted graphs with real edge weights, Weimann and Yuster [FOCS 2010] presented a combinatorial randomized f-DSO with ~O(mn^{4-α}) preprocessing time and subquadratic ~O(n^{2-2(1-α)/f}) query time for every value of 0 < α< 1. We derandomize this result and present a combinatorial deterministic f-DSO with the same asymptotic preprocessing and query time.

cs.DS↗

arXiv2018

Dynamic Matching: Reducing Integral Algorithms to Approximately-Maximal Fractional Algorithms

We present a simple randomized reduction from fully-dynamic integral matching algorithms to fully-dynamic "approximately-maximal" fractional matching algorithms. Applying this reduction to the recent fractional matching algorithm of Bhattacharya, Henzinger, and Nanongkai (SODA 2017), we obtain a novel result for the integral problem. Specifically, our main result is a randomized fully-dynamic $(2+ε)$-approximate integral matching algorithm with small polylog worst-case update time. For the $(2+ε)$-approximation regime only a \emph{fractional} fully-dynamic $(2+ε)$-matching algorithm with worst-case polylog update time was previously known, due to Bhattacharya et al.~(SODA 2017). Our algorithm is the first algorithm that maintains approximate matchings with worst-case update time better than polynomial, for any constant approximation ratio. As a consequence, we also obtain the first constant-approximate worst-case polylogarithmic update time maximum weight matching algorithm.

cs.DS↗

arXiv2013

Minimal Indices for Successor Search

We give a new successor data structure which improves upon the index size of the Pǎtraşcu-Thorup data structures, reducing the index size from $O(n w^{4/5})$ bits to $O(n \log w)$ bits, with optimal probe complexity. Alternatively, our new data structure can be viewed as matching the space complexity of the (probe-suboptimal) $z$-fast trie of Belazzougui et al. Thus, we get the best of both approaches with respect to both probe count and index size. The penalty we pay is an extra $O(\log w)$ inter-register operations. Our data structure can also be used to solve the weak prefix search problem, the index size of $O(n \log w)$ bits is known to be optimal for any such data structure. The technical contributions include highly efficient single word indices, with out-degree $w/\log w$ (compared to the $w^{1/5}$ out-degree of fusion tree based indices). To construct such high efficiency single word indices we device highly efficient bit selectors which, we believe, are of independent interest.

cs.DS↗