arXiv subjects

Gal Mendelson

Publications and source records attributed to Gal Mendelson.

11 recordsLinked to original sources

arXiv2026

Approximating Uniform Random Rotations by Two-Block Structured Hadamard Rotations in High Dimensions

Uniform random rotations are a useful primitive in applications such as fast Johnson-Lindenstrauss embeddings, kernel approximation, communication-efficient learning, and recent AI compression pipelines, but they are computationally expensive to generate and apply in high dimensions. A common practical replacement is repeated structured random rotations built from Walsh-Hadamard transforms and random sign diagonals. Applying the structured random rotation twice has been shown empirically to be useful, but the supporting theory is still limited. In this paper we study the approximation quality achieved when using this two-block structured Hadamard rotation. Our results are both positive and negative. On the positive side, we prove that every fixed coordinate of the two-block transform converges uniformly, over all inputs, to the corresponding coordinate of a uniformly rotated vector, with an explicit Kolmogorov-distance bound of order $d^{-1/5}$. On the negative side, we prove an explicit lower bound on the Wasserstein distance between the full vector distributions, showing that the two-block transform is not a globally accurate surrogate for a uniform random rotation in the worst case. For the extremal input used in the lower bound, we also prove a matching asymptotic upper bound, showing that the lower-bound scale is sharp for that input. Taken together, the results identify a clear separation between one-dimensional marginal behavior, where approximation improves with dimension, and full high-dimensional geometry, where a nonvanishing discrepancy remains. This provides a partial theoretical explanation for the empirical success of structured Hadamard rotations in some algorithms, while also clarifying the limitations of treating them as drop-in replacements for true uniform random rotations.

cs.LG↗

arXiv2026

A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work

This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any $b>0$ bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant$_{\text{mse}}$ is a special case of EDEN obtained by fixing EDEN's scalar scale parameter to $S=1$. EDEN supports both biased and unbiased quantization, each optimized by a different $S$ (chosen via methods described in the EDEN works). The fixed choice $S=1$ used by TurboQuant is generally suboptimal, although the optimal $S$ for biased EDEN converges to $1$ as the dimension grows; accordingly TurboQuant$_{\text{mse}}$ approaches EDEN's behavior for large $d$. Second, TurboQuant$_{\text{prod}}$ combines a biased $(b-1)$-bit EDEN step with an unbiased 1-bit QJL quantization of the residual. It is suboptimal in three ways: (1) its $(b-1)$-bit step uses the suboptimal $S=1$; (2) its 1-bit unbiased residual quantization has worse MSE than (unbiased) 1-bit EDEN; (3) chaining a biased $(b-1)$-bit step with a 1-bit unbiased residual step is inferior to unbiasedly quantizing the input directly with $b$-bit EDEN. Third, some of the analysis in the TurboQuant work mirrors that of the EDEN works: both exploit the connection between random rotations and the shifted Beta distribution, use the Lloyd-Max algorithm, and note that Randomized Hadamard Transforms can replace uniform random rotations. Experiments support these claims: biased EDEN (with optimized $S$) is more accurate than TurboQuant$_{\text{mse}}$, and unbiased EDEN is markedly more accurate than TurboQuant$_{\text{prod}}$, often by more than a bit (e.g., 2-bit EDEN beats 3-bit TurboQuant$_{\text{prod}}$). We also repeat all accuracy experiments from the TurboQuant paper, showing that EDEN outperforms it in every setup we have tried.

cs.LG↗

arXiv2025

Fooling Algorithms in Non-Stationary Bandits using Belief Inertia

We study the problem of worst case regret in piecewise stationary multi armed bandits. While the minimax theory for stationary bandits is well established, understanding analogous limits in time-varying settings is challenging. Existing lower bounds rely on what we refer to as infrequent sampling arguments, where long intervals without exploration allow adversarial reward changes that induce large regret. In this paper, we introduce a fundamentally different approach based on a belief inertia argument. Our analysis captures how an algorithm's empirical beliefs, encoded through historical reward averages, create momentum that resists new evidence after a change. We show how this inertia can be exploited to construct adversarial instances that mislead classical algorithms such as Explore Then Commit, epsilon greedy, and UCB, causing them to suffer regret that grows linearly with T and with a substantial constant factor, regardless of how their parameters are tuned, even with a single change point. We extend the analysis to algorithms that periodically restart to handle non stationarity and prove that, even then, the worst case regret remains linear in T. Our results indicate that utilizing belief inertia can be a powerful method for deriving sharp lower bounds in non stationary bandits.

cs.LG↗

arXiv2024

The Hybrid Hospital: Balancing On-Site and Remote Hospitalization

Hybrid hospitals offer on-site and remote hospitalization through telemedicine. These new healthcare models require novel operational policies to balance costs, efficiency, and patient well-being. Our study addresses two first-order questions: (i) how to direct patient admission and call-in based on individual characteristics and proximity and (ii) how to determine the optimal allocation of medical resources between these two hospitalization options and among different patient types. We develop a model that uses Brownian Motion to capture the patient's health evolution during remote/on-site hospitalization and during travel. Under cost-minimizing call-in policies, we find that remote hospitalization can be cost-effective for moderately distant patients, as the optimal call-in threshold is non-monotonic in the patient's travel time. Subject to scarce resources, the optimal solution structure becomes equivalent to a simultaneous, identically sized increase of remote and on-site costs under abundant resources. When limited resources must be divided among multiple patient types, the optimal thresholds shift in non-obvious ways as resource availability changes. Finally, we develop a practical and efficient policy that allows for swapping an on-site patient with a remote patient when the latter is called-in and sufficient resources are not available to treat both on-site. Contrary to the widely held view that telemedicine can mitigate rural and non-rural healthcare disparities, our research suggests that on-site care may actually be more cost-effective than remote hospitalization for patients in distant locations, due to (potentially overlooked) risks during patient travel. This finding may be of particular concern in light of the growing number of ``hospital deserts'' amid recent rural hospital closures, as these communities may in fact not be well-served through at-home care.

eess.SY↗

arXiv2024

Learning Service Slowdown using Observational Data

Being able to identify service slowdowns is crucial to many operational problems. We study how to use observational congestion data to learn service slowdown in a multi-server system that uses adaptive congestion control mechanisms. We show that a commonly used summary statistic that relies on the marginal congestion measured at individual servers can be highly inaccurate in the presence of adaptive congestion control. We propose a new statistic based on potential routing actions, and show it provides a much more robust signal for server slowdown in these settings. Unlike the marginal statistic, potential action aims to detect changes in the routing actions, and is able to uncover slowdowns even when they do not reflect in marginal congestion. Our results highlight the complexity in performing observational statistical analysis for service systems in the presence of adaptive congestion control. They also suggest that practitioners may want to combine multiple, orthogonal statistics to achieve reliable slowdown detection.

physics.soc-ph↗

arXiv2022

Load Balancing Using Sparse Communication

Load balancing across parallel servers is an important class of congestion control problems that arises in service systems. An effective load balancer relies heavily on accurate, real-time congestion information to make routing decisions. However, obtaining such information can impose significant communication overheads, especially in demanding applications like those found in modern data centers. We introduce a framework for communication-aware load balancing and design new load balancing algorithms that perform exceptionally well even in scenarios with sparse communication patterns. Central to our approach is state approximation, where the load balancer first estimates server states through a communication protocol. Subsequently, it utilizes these approximate states within a load balancing algorithm to determine routing decisions. We demonstrate that by using a novel communication protocol, one can achieve accurate queue length approximation with sparse communication: for a maximal approximation error of x, the communication frequency only needs to be O(1/x^2). We further show, via a diffusion analysis, that a constant maximal approximation error is sufficient for achieving asymptotically optimal performance. Taken together, these results therefore demonstrate that highly performant load balancing is possible with very little communication. Through simulations, we observe that the proposed designs match or surpass the performance of state-of-the-art load balancing algorithms while drastically reducing communication rates by up to 90%.

cs.DC↗

arXiv2021

EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning

Distributed Mean Estimation (DME) is a central building block in federated learning, where clients send local gradients to a parameter server for averaging and updating the model. Due to communication constraints, clients often use lossy compression techniques to compress the gradients, resulting in estimation inaccuracies. DME is more challenging when clients have diverse network conditions, such as constrained communication budgets and packet losses. In such settings, DME techniques often incur a significant increase in the estimation error leading to degraded learning performance. In this work, we propose a robust DME technique named EDEN that naturally handles heterogeneous communication budgets and packet losses. We derive appealing theoretical guarantees for EDEN and evaluate it empirically. Our results demonstrate that EDEN consistently improves over state-of-the-art DME techniques.

cs.LG↗

arXiv2021

DRIVE: One-bit Distributed Mean Estimation

We consider the problem where $n$ clients transmit $d$-dimensional real-valued vectors using $d(1+o(1))$ bits each, in a manner that allows the receiver to approximately reconstruct their mean. Such compression problems naturally arise in distributed and federated learning. We provide novel mathematical results and derive computationally efficient algorithms that are more accurate than previous compression techniques. We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.

cs.LG↗

arXiv2021

On the Persistent-Idle Load Distribution Policy Under Batch Arrivals and Random Service Capacity

The Persistent-Idle (PI) load distribution policy was recently introduced as an appealing alternative to current low-communication load balancing techniques. In PI, servers only update the dispatcher when they become idle, and the dispatcher always sends jobs to the last server that reported being idle. PI is unique in that it does not seek to push the server queue lengths towards equalization greedily. Rather, it aggressively pulls the servers away from starvation. As a result, PI's analysis requires different tools than other load balancing approaches. So far, PI was proven to achieve the stability region for Bernoulli arrivals and deterministic and constant service capacities. Our main contribution is proving that PI achieves the stability region in a model with batch arrivals and random service capacities. Proving this result requires developing tighter bounds on quantities of interest and proving the fulfillment of a generalized version of the state-dependent drift criteria previously considered. We also present PI-Split, which allows batches to be split among idle servers and prove its stability. Finally, we conduct simulations comparing PI and PI-Split to other load balancing policies. Simulation results indicate that both achieve excellent performance for the model under consideration for a wide range of parameters.

math.PR↗

arXiv2020

A Lower Bound on the stability region of Redundancy-d with FIFO service discipline

Redundancy-d (R(d)) is a load balancing method used to route incoming jobs to K servers, each with its own queue. Every arriving job is replicated into 2<=d<=K tasks, which are then routed to d servers chosen uniformly at random. When the first task finishes service, the remaining d-1 tasks are cancelled and the job departs the system. Despite the fact that R(d) is known, under certain conditions, to substantially improve job completion times compared to not using redundancy at all, little is known on a more fundamental performance criterion: what is the set of arrival rates under which the R(d) queueing system with FIFO service discipline is stable? In this context, due to the complex dynamics of systems with redundancy and cancellations, existing results are scarce and are limited to very special cases with respect to the joint service time distribution of tasks. In this paper we provide a non-trivial, closed form lower bound on the stability region of R(d) for a general joint service time distribution of tasks with finite first and second moments. We consider a discrete time system with Bernoulli arrivals and assume that jobs are processed by their order of arrival. We use the workload processes and a quadratic Lyapunov function to characterize the set of arrival rates for which the system is stable. While simulation results indicate our bound is not tight, it provides an easy-to-check performance guarantee.

cs.PF↗

arXiv2018

AnchorHash: A Scalable Consistent Hash

Consistent hashing (CH) is a central building block in many networking applications, from datacenter load-balancing to distributed storage. Unfortunately, state-of-the-art CH solutions cannot ensure full consistency under arbitrary changes and/or cannot scale while maintaining reasonable memory footprints and update times. We present AnchorHash, a scalable and fully-consistent hashing algorithm. AnchorHash achieves high key lookup rates, a low memory footprint, and low update times. We formally establish its strong theoretical guarantees, and present advanced implementations with a memory footprint of only a few bytes per resource. Moreover, extensive evaluations indicate that it outperforms state-of-the-art algorithms, and that it can scale on a single core to 100 million resources while still achieving a key lookup rate of more than 15 million keys per second.

cs.DS↗