arXiv subjects

Roy Friedman

Publications and source records attributed to Roy Friedman.

At least 19 recordsLinked to original sources

arXiv2026

FinWhale: An Optimally Resilient Two-Round Terminating DAG Protocol

DAG based Byzantine Fault Tolerant protocols provide high throughput consensus under partial synchrony but existing DAG protocols still require at least three message delays to commit decisions. In contrast fast path Byzantine Fault Tolerant protocols can achieve optimal two message delay termination under favorable conditions though they do not naturally extend to DAGs. We present FinWhale the first DAG based Byzantine Fault Tolerant protocol with a two message delay fast path. FinWhale extends Mysticeti with a novel fast path commit mechanism that safely coexists with the protocol's original slow path rules. To preserve safety across different local DAG views we introduce new commit structures based on fast path evidence blocks enabling validators to combine fast path and slow path reasoning consistently. FinWhale operates in the partially synchronous model with n equals three f plus two p minus one validators matching the known lower bound for fast Byzantine consensus. The protocol tolerates up to f Byzantine faults and achieves fast termination whenever at most p validators fail during the fast path where p is between one and f. Our results show that optimal latency fast paths can be integrated into uncertified DAG consensus protocols.

cs.DC↗

arXiv2026

From Exact Hits to Close Enough: Semantic Caching for LLM Embeddings

The rapid adoption of large language models (LLMs) has created demand for faster responses and lower costs. Semantic caching, reusing semantically similar requests via their embeddings, addresses this need but breaks classic cache assumptions and raises new challenges. In this paper, we explore offline policies for semantic caching, proving that implementing an optimal offline policy is NP-hard, and propose several polynomial-time heuristics. We also present online semantic aware cache policies that combine recency, frequency, and locality. Evaluations on diverse datasets show that while frequency based policies are strong baselines, our novel variant improves semantic accuracy. Our findings reveal effective strategies for current systems and highlight substantial headroom for future innovation. All code is open source.

cs.CL↗

arXiv2025

Distributed Recoverable Sketches (Extended Version)

Sketches are commonly used in computer systems and network monitoring tools to provide efficient query executions while maintaining a compact data representation. Switches and routers maintain sketches to track statistical characteristics of network traffic. The availability of such data is essential for the network analysis as a whole. Consequently, being able to recover sketches is critical after a switch crash. In this work, we explore how nodes in a network environment can cooperate to recover sketch data whenever any subset of them crashes. In particular, we focus on frequency estimation linear sketches, such as the Count-Min Sketch. We consider various approaches to ensure data reliability and explore the trade-offs between space consumption, runtime overheads, and traffic during recovery, which we point out as design guidelines. Besides different aspects of efficacy, we design a modular system for ease of maintenance and further scaling. A key aspect we examine is how the nodes update each other regarding their sketch content as it evolves over time. In particular, we compare periodic full updates vs incremental updates. We also examine several data structures to economically represent and encode a batch of latest changes. Our framework is generic, and other data structures can be plugged-in via an abstract API as long as they implement the corresponding API methods.

cs.DC↗

arXiv2025

A New Probabilistic Mobile Byzantine Failure Model for Self-Protecting Systems

Modern distributed systems face growing security threats, as attackers continuously enhance their skills and vulnerabilities span across the entire system stack, from hardware to the application layer. In the system design phase, fault tolerance techniques can be employed to safeguard systems. From a theoretical perspective, an attacker attempting to compromise a system can be abstracted by considering the presence of Byzantine processes in the system. Although this approach enhances the resilience of the distributed system, it introduces certain limitations regarding the accuracy of the model in reflecting real-world scenarios. In this paper, we consider a self-protecting distributed system based on the \emph{Monitoring-Analyse-Plan-Execute over a shared Knowledge} (MAPE-K) architecture, and we propose a new probabilistic Mobile Byzantine Failure (MBF) that can be plugged into the Analysis component. Our new model captures the dynamics of evolving attacks and can be used to drive the self-protection and reconfiguration strategy. We analyze mathematically the time that it takes until the number of Byzantine nodes crosses given thresholds, or for the system to self-recover back into a safe state, depending on the rates of Byzantine infection spreading \emph{vs.} the rate of self-recovery. We also provide simulation results that illustrate the behavior of the system under such assumptions.

cs.DC↗

arXiv2025

MODE: Learning compositional representations of complex systems with Mixtures Of Dynamical Experts

Dynamical systems in the life sciences are often composed of complex mixtures of overlapping behavioral regimes. Cellular subpopulations may shift from cycling to equilibrium dynamics or branch towards different developmental fates. The transitions between these regimes can appear noisy and irregular, posing a serious challenge to traditional, flow-based modeling techniques which assume locally smooth dynamics. To address this challenge, we propose MODE (Mixture Of Dynamical Experts), a graphical modeling framework whose neural gating mechanism decomposes complex dynamics into sparse, interpretable components, enabling both the unsupervised discovery of behavioral regimes and accurate long-term forecasting across regime transitions. Crucially, because agents in our framework can jump to different governing laws, MODE is especially tailored to the aforementioned noisy transitions. We evaluate our method on a battery of synthetic and real datasets from computational biology. First, we systematically benchmark MODE on an unsupervised classification task using synthetic dynamical snapshot data, including in noisy, few-sample settings. Next, we show how MODE succeeds on challenging forecasting tasks which simulate key cycling and branching processes in cell biology. Finally, we deploy our method on human, single-cell RNA sequencing data and show that it can not only distinguish proliferation from differentiation dynamics but also predict when cells will commit to their ultimate fate, a key outstanding challenge in computational biology.

cs.LG↗

arXiv2025

Ethereum Conflicts Graphed

Ethereum, a leading blockchain platform, has revolutionized the digital economy by enabling decentralized transactions and the execution of smart contracts. Ethereum transactions form the backbone of its network, facilitating peer-to-peer exchanges and interactions with complex decentralized applications. Smart contracts extend Ethereum's capabilities by automating processes and enabling trustless execution of agreements. Hence, understanding how these smart contracts interact is important in order to facilitate various performance optimizations, such as warming objects before they are being accessed and enabling concurrent execution. Of particular interest to us are the development of the calling graph, as well as the read sets and write sets of invocations within the same block, and the properties of the associated conflict graph that is derived from them. The latter is important for understanding the parallelization potential of smart contracts on Ethereum. We traced upwards of 2 million recent Ethereum blocks using call tracer and prestate tracer, out of a total of 21.4 million blocks at the time of writing. We report on the transactions per block distribution, the structure of call trees in smart contract invocations, the ratio of value-transfer transactions to smart contract invocations, as well as provide a comprehensive study of the structure of blocks' conflict graphs. We find that conflict graphs predominantly show a star like configuration, as well as other noteworthy structural properties.

cs.DC↗

arXiv2025

On Quorum Sizes in DAG-Based BFT Protocols

Several prominent DAG-based blockchain protocols, such as DAG-Rider, Tusk, and Bullshark, completely separate between equivocation elimination and committing; equivocation is handled through the use of a reliable Byzantine broadcast black-box protocol, while committing is handled by an independent DAG-based protocol. With such an architecture, a natural question that we study in this paper is whether the DAG protocol would work when the number of nodes (or validators) is only $2f+1$ (when equivocation is eliminated), and whether there are benefits in working with larger number of nodes, i.e., a total of $kf+1$ nodes for $k > 3$. We find that while DAG-Rider's correctness is maintained with $2f+1$ nodes, the asynchronous versions of both Tusk and Bullshark inherently depends on having $3f+1$ nodes, regardless of equivocation. We also explore the impact of having larger number of nodes on the expected termination time of these three protocols.

cs.DC↗

arXiv2025

Characterizing Nonlinear Dynamics via Smooth Prototype Equivalences

Characterizing the long term behavior of dynamical systems given limited measurements is a common challenge throughout the physical and biological sciences. This is a challenging task due to the sparsity and noise inherent to empirical observations, as well as the variability of possible long-term dynamics. We address this by introducing smooth prototype equivalences (SPE), a framework for matching sparse observations to prototypical behaviors using invertible neural networks which model smooth phase space deformations. SPE can localize the invariant sets describing long-term behavior of the observed dynamics through the learned mapping from prototype space to data space. Furthermore, SPE can classify dynamical regimes by comparing the data residual of the deformed measurements to prototype dynamics. Our method outperforms existing techniques in the classification of oscillatory systems and can efficiently identify invariant structures like limit cycles and fixed points in an equation-free manner, even when only a small, noisy subset of the phase space is observed. SPE further reveals driving genes in synthetic oscillators such as the repressilator regulatory circuit, and traces cyclic biological processes like the cell cycle trajectory directly from experimental high-dimensional single-cell gene expression data.

cs.LG↗

arXiv2024

Control+Shift: Generating Controllable Distribution Shifts

We propose a new method for generating realistic datasets with distribution shifts using any decoder-based generative model. Our approach systematically creates datasets with varying intensities of distribution shifts, facilitating a comprehensive analysis of model performance degradation. We then use these generated datasets to evaluate the performance of various commonly used networks and observe a consistent decline in performance with increasing shift intensity, even when the effect is almost perceptually unnoticeable to the human eye. We see this degradation even when using data augmentations. We also find that enlarging the training dataset beyond a certain point has no effect on the robustness and that stronger inductive biases increase robustness.

cs.CV↗

arXiv2024

Intriguing Properties of Modern GANs

Modern GANs achieve remarkable performance in terms of generating realistic and diverse samples. This has led many to believe that ``GANs capture the training data manifold''. In this work we show that this interpretation is wrong. We empirically show that the manifold learned by modern GANs does not fit the training distribution: specifically the manifold does not pass through the training examples and passes closer to out-of-distribution images than to in-distribution images. We also investigate the distribution over images implied by the prior over the latent codes and study whether modern GANs learn a density that approximates the training distribution. Surprisingly, we find that the learned density is very far from the data distribution and that GANs tend to assign higher density to out-of-distribution images. Finally, we demonstrate that the set of images used to train modern GANs are often not part of the typical set described by the GANs' distribution.

cs.LG↗

arXiv2024

Batch-Schedule-Execute: On Optimizing Concurrent Deterministic Scheduling for Blockchains (Extended Version)

Executing smart contracts is a compute and storage-intensive task, which currently dominates modern blockchain's performance. Given that computers are becoming increasingly multicore, concurrency is an attractive approach to improve programs' execution runtime. A unique challenge of blockchains is that all replicas (miners or validators) must execute all smart contracts in the same logical order to maintain the semantics of State Machine Replication (SMR). In this work, we study the maximal level of parallelism attainable when focusing on the conflict graph between transactions packaged in the same block. This exposes a performance vulnerability that block creators may exploit against existing blockchain concurrency solutions, which rely on a total ordering phase for maintaining consistency amongst all replicas. To facilitate the formal aspects of our study, we develop a novel generic framework for Active State Machine Replication (ASMR) that is strictly serializable. We introduce the concept of graph scheduling and the definition of the minimal latency scheduling problem, which we prove to be NP-hard. We show that the restricted version of this problem for homogeneous transactions is equivalent to the classic Graph Vertex Coloring Problem, yet show that the heterogeneous case is more complex. We discuss the practical implications of these results.

cs.DC↗

arXiv2023

An Evaluation of Software Sketches

This work presents a detailed evaluation of Rust (software) implementations of several popular sketching solutions, as well as recently proposed optimizations. We compare these solutions in terms of computational speed, memory consumption, and several approximation error metrics. Overall, we find a simple hashing based solution employed with the Nitro sampling technique [22] gives the best trade-off between memory, error and speed. Our findings also include some novel insights about how to best combine sampling with Counting Cuckoo filters depending on the application.

cs.DB↗

arXiv2022

HIGlow: Conditional Normalizing Flows for High-Fidelity HI Map Modeling

Extracting the maximum amount of cosmological and astrophysical information from upcoming large-scale surveys remains a challenge. This includes evaluating the exact likelihood, parameter inference and generating new diverse synthetic examples of the incoming high-dimensional data sets. In this work, we propose the use of normalizing flows as a generative model of the neutral hydrogen (HI) maps from the CAMELS project. Normalizing flows have been very successful at parameter inference and generating new, realistic examples. Our model utilizes the spatial structure of the HI maps in order to faithfully follow the statistics of the data, allowing for high-fidelity sample generation and efficient parameter inference.

astro-ph.CO↗

arXiv2022

Multilevel Bidirectional Cache Filter

Modern caches are often required to handle a massive amount of data, which exceeds the amount of available memory; thus, hybrid caches, specifically DRAM/SSD combination, become more and more prevalent. In such environments, in addition to the classical hit-ratio target, saving writes to the second-level cache is a dominant factor to avoid write amplification and wear out, two notorious phenomena of SSD. This paper presents BiDiFilter, a novel multilevel caching scheme that controls demotions and promotions between cache levels using a frequency sketch filter. Further, it splits the higher cache level into two areas to keep the most recent and the most frequent items close to the user. We conduct an extensive evaluation over real-world traces, comparing to previous multilevel policies. We show that using our mechanism yields an x10 saving of writes in almost all cases and often improving latencies by up to 20%.

cs.OS↗

arXiv2022

Limited Associativity Caching in the Data Plane

In-network caching promises to improve the performance of networked and edge applications as it shortens the paths data need to travel. This is by storing so-called hot items in the network switches on-route between clients who access the data and the storage servers who maintain it. Since the data flows through those switches in any case, it is natural to cache hot items there. Most software-managed caches treat the cache as a fully associative region. Alas, a fully associative design seems to be at odds with programmable switches' goal of handling packets in a short bounded amount of time, as well as their restricted programming model. In this work, we present PKache, a generic limited associativity cache implementation in the programmable switches' domain-specific P4 language, and demonstrate its utility by realizing multiple popular cache management schemes.

cs.NI↗

arXiv2022

SQUAD: Combining Sketching and Sampling Is Better than Either for Per-item Quantile Estimation

Stream monitoring is fundamental in many data stream applications, such as financial data trackers, security, anomaly detection, and load balancing. In that respect, quantiles are of particular interest, as they often capture the user's utility. For example, if a video connection has high tail latency, the perceived quality will suffer, even if the average and median latencies are low. In this work, we consider the problem of approximating the per-item quantiles. Elements in our stream are (ID, latency) tuples, and we wish to track the latency quantiles for each ID. Existing quantile sketches are designed for a single number stream (e.g., containing just the latency). While one could allocate a separate sketch instance for each ID, this may require an infeasible amount of memory. Instead, we consider tracking the quantiles for the heavy hitters (most frequent items), which are often considered particularly important, without knowing them beforehand. We first present a simple sampling algorithm that serves as a benchmark. Then, we design an algorithm that augments a quantile sketch within each entry of a heavy hitter algorithm, resulting in similar space complexity but with a deterministic error guarantee. Finally, we present SQUAD, a method that combines sampling and sketching while improving the asymptotic space complexity. Intuitively, SQUAD uses a background sampling process to capture the behaviour of the latencies of an item before it is allocated with a sketch, thereby allowing us to use fewer samples and sketches. Our solutions are rigorously analyzed, and we demonstrate the superiority of our approach using extensive simulations.

cs.DS↗

arXiv2021

On the data persistency of replicated erasure codes in distributed storage systems

This paper studies the fundamental problem of data persistency for a general family of redundancy schemes in distributed storage systems, called replicated erasure codes. Namely, we analyze two strategies of replicated erasure codes distribution: random and symmetric. For both strategies we derive closed analytical and asymptotic formulas for expected data persistency despite nodes failure.

cs.DC↗

arXiv2021

Limited Associativity Makes Concurrent Software Caches a Breeze

Software caches optimize the performance of diverse storage systems, databases and other software systems. Existing works on software caches automatically resort to fully associative cache designs. Our work shows that limited associativity caches are a promising direction for concurrent software caches. Specifically, we demonstrate that limited associativity enables simple yet efficient realizations of multiple cache management schemes that can be trivially parallelized. We show that the obtained hit ratio is usually similar to fully associative caches of the same management policy, but the throughput is improved by up to X5 compared to production-grade caching libraries, especially in multi-threaded executions.

cs.AR↗