arXiv subjects

Philip Levis

Publications and source records attributed to Philip Levis.

At least 19 recordsLinked to original sources

arXiv2026

The Kernel's Write: Application Read-Only Memory

Alongside power, DRAM has become a major limiting factor in datacenter growth. As DRAM's cost-per-bit has plateaued over the past decade, a class of emerging memory technologies, called Long-term RAM (LtRAM), offers a path to denser and cheaper main memory. However, LtRAM has three main drawbacks: asymmetric read/write latencies, limited endurance, and coarse write granularity. In an attempt to isolate software from these drawbacks, LtRAM technologies such as Intel Optane copy an approach from flash devices and introduce a translation layer that manages wear-leveling, address remapping, and read/write caching. Prior experimental studies have found these operations add significantly to LtRAM latency. Rather than making LtRAM look like DRAM, we propose redesigning the hardware/software interface to offload more responsibility to the operating system. This design hinges on one central property, Application Read-Only Memory (AROM): LtRAM pages are read-only to applications and written only by the OS during page migrations. AROM is enforced by leveraging copy-on-write (CoW): application writes to LtRAM trigger a fault that migrates the page back to DRAM before the store is applied. This invariant allows us to shift LtRAM management from the on-DIMM controller to the operating system, drastically simplifying the DIMM's hardware. With this approach, we aim to match the performance of pure DRAM on read-mostly workloads while delivering LtRAM's density and cost advantages.

cs.AR↗

arXiv2026

Federation of Experts: Communication Efficient Distributed Inference for Large Language Models

Mixture of experts has emerged as the primary mechanism for making Large Language Models (LLMs) computationally efficient. However, in distributed settings, communicating token embeddings between experts is a significant bottleneck. We present the novel Federation of Experts (FoE) architecture. FoE restructures the MoE block of a transformer layer into multiple MoE clusters. Each cluster is responsible for only one of the KV heads and expert parallelism is applied between those experts. Between clusters, a sum synchronizes the post-attention residuals, which then drives routing and dispatch for the next MoE block. In a single-node setting, FoE completely eliminates all-to-all communication as all experts within a group are contained on the same GPU. In multi-node settings, FoE confines all-to-all communication to the intra-node fabric, thus significantly reducing communication overhead. An implementation of FoE finds that on LongBench, FoE significantly improves inference throughput and latency in both single-node and multi-node settings, reducing the end-to-end forward-pass latency by up to 5.2x, TTFT by 3.62x, and TBT by 1.95x. It does so while achieving comparable generation quality to a mixture of experts model of the same size and training configuration.

cs.LG↗

arXiv2026

CvxCluster: Solving Large, Complex, Granular Resource Allocation Problems 100-1000x Faster

Cluster resource allocation is a multidimensional search problem that finds the best allocation of tasks to servers. Because the search space grows exponentially, modern approaches frame it as a mixed integer program (MIP) or a complex set of search heuristics. This paper proposes using a different approach: convex optimization, which has extremely fast solution methods. The research challenge is devising how to transform cluster resource allocation into a convex problem that generates good placements. We describe CvxCluster, which allocates cluster resources with a two-stage algorithm. The first stage solves a convex relaxation of the placement problem to yield a principled set of per-machine resource prices. The second stage uses these prices to drive a lightweight greedy procedure to place tasks. Experimental results with Azure traces find that CvxCluster scales to 100,480 servers under proportional workload growth and sustains arrival rates up to 500,000x the baseline trace. CvxCluster runs 100 to 2,500x faster than a state-of-the-art MIP solver while remaining within 3% of the optimal objective. CvxCluster can support complex constraints such as job anti-affinity, machine types, and GPU servers. The key insight behind CvxCluster is that reformulating placement as a continuous rather than discrete problem enables much faster methods that find solutions just as good or better than prior heuristics.

cs.DC↗

arXiv2026

Tock: From Research to Securing 10 Million Computers

Tock began 10 years ago as a research operating system developed by academics to help other academics build urban sensing applications. By leveraging a new language (Rust) and new hardware protection mechanisms, Tock enabled Multiprogramming a 64 kB Computer Safely and Efficiently. Today, it is an open source project with a vibrant community of users and contributors. It is deployed on root of trust hardware in data center servers and on millions of laptops; it is used to develop automotive and space products, wearable electronics, and hardware security tokens--all while remaining a platform for operating systems research. This paper focuses on the impact of Tock's technical design on its adoption, the challenges and unexpected benefits of using a type safe language (Rust)--particularly in security sensitive settings--and the experience of supporting a production open4source operating system from academia.

cs.OS↗

arXiv2025

The Future of Memory: Limits and Opportunities

Memory latency, bandwidth, capacity, and energy increasingly limit performance. In this paper, we reconsider proposed system architectures that consist of huge (many-terabyte to petabyte scale) memories shared among large numbers of CPUs. We argue two practical engineering challenges, scaling and signaling, limit such designs. We propose the opposite approach. Rather than create large, shared, homogenous memories, systems explicitly break memory up into smaller slices more tightly coupled with compute elements. Leveraging advances in 2.5D/3D integration, this compute-memory node provisions private local memory, enabling accesses of node-exclusive data through micrometer-scale distances, and dramatically reduced access cost. In-package memory elements support shared state within a processor, providing far better bandwidth and energy-efficiency than DRAM, which is used as main memory for large working sets and cold data. Hardware making memory capacities and distances explicit allows software to efficiently compose this hierarchy, managing data placement and movement.

cs.AR↗

arXiv2025

Towards Memory Specialization: A Case for Long-Term and Short-Term RAM

Both SRAM and DRAM have stopped scaling: there is no technical roadmap to reduce their cost (per byte/GB). As a result, memory now dominates system cost. This paper argues for a paradigm shift from today's simple memory hierarchy toward specialized memory architectures that exploit application-specific access patterns. Rather than relying solely on traditional off-chip DRAM and on-chip SRAM, we envisage memory systems equipped with additional types of memory whose performance trade-offs benefit workloads through non-hierarchical optimization. We propose two new memory classes deserving explicit OS support: long-term RAM (LtRAM) optimized for read-intensive data with long lifetimes, and short-term RAM (StRAM) designed for transient, frequently-accessed data with short lifetimes. We explore underlying device technologies that could implement these classes, including their evolution and their potential integration into current system designs given emerging workload requirements. We identify critical research challenges to realize what we believe is a necessary evolution toward more efficient and scalable computing systems capable of meeting future demands.

cs.AR↗

arXiv2025

Optimizing Software Defined Battery Systems for Transformer Protection

Residential electric vehicle charging causes large spikes in electricity demand that risk violating neighborhood transformer power limits. Battery energy storage systems reduce these transformer limit violations, but operating them individually is not cost-optimal. Instead of individual optimization, aggregating, or sharing, these batteries leads to cost-optimal performance, but homeowners must relinquish battery control. This paper leverages virtualization to propose battery sharing optimization schemes to reduce electricity costs, extend the lifetime of a residential transformer, and maintain homeowner control over the battery. A case study with simulated home loads, solar generation, and electric vehicle charging profiles demonstrates that joint, or shared, optimization reduces consumer bills by 56% and transformer aging by 48% compared to individual optimization. Hybrid and dynamic optimization schemes that provide owners with autonomy have similar transformer aging reduction but are slightly less cost-effective. These results suggest that controlling shared batteries with virtualization is an effective way to delay transformer upgrades in the face of growing residential electric vehicle charging penetration.

eess.SY↗

arXiv2025

Aging-Aware Battery Control via Convex Optimization

We consider the task of controlling a battery while balancing two competing objectives that evolve over different time scales. The short-term objective, such as arbitrage or load smoothing, improves with more battery cycling, while the long-term objective is to maximize battery lifetime, which discourages cycling. Using a semi-empirical aging model, we formulate this problem as a convex optimization problem. We use model predictive control (MPC) with a convex approximation of aging dynamics to optimally manage the trade-off between performance and degradation. Through simulations, we quantify this trade-off in both economic and smoothing applications.

math.OC↗

arXiv2025

GainSight: A Unified Framework for Data Lifetime Profiling and Heterogeneous Memory Composition

As AI workloads drive increasing memory requirements, domain-specific accelerators need higher-density on-chip memory beyond what current SRAM scaling trends can provide. Simultaneously, the vast amounts of short-lived data in these workloads make SRAM overprovisioned in retention capability. To address this mismatch, we propose a wholesale shift from uniform SRAM arrays to heterogeneous on-chip memory, incorporating denser short-term RAM (StRAM) devices whose limited retention times align with transient data lifetimes. To facilitate this shift, we introduce GainSight, the first comprehensive, open-source framework that aligns dynamic, fine-grained workload lifetime profiles with memory device characteristics to enable generation of optimal StRAM memory compositions. GainSight combines retargetable profiling backends with an architecture-agnostic analytical frontend. The various backends capture cycle-accurate data lifetimes, while the frontend correlates workload patterns with StRAM retention properties to generate optimal memory compositions and project performance. GainSight elevates data lifetime to a first-class design consideration for next-generation AI accelerators, enabling systematic exploitation of data transience for improved on-chip memory density and efficiency. Applying GainSight to MLPerf Inference and PolyBench workloads reveals that 64.3% of first-level GPU cache accesses and 79.01% of systolic array scratchpad accesses exhibit sub-microsecond lifetimes suitable for high-density StRAM, with optimal heterogeneous on-chip memory compositions achieving up to 3x active energy and 4x area reductions compared to uniform SRAM hierarchies. To facilitate adoption and further research, GainSight is open-sourced at https://gainsight.stanford.edu/.

cs.AR↗

arXiv2024

Alto: Orchestrating Distributed Compound AI Systems with Nested Ancestry

Compound AI applications chain together subcomponents such as generative language models, document retrievers, and embedding models. Applying traditional systems optimizations such as parallelism and pipelining in compound AI systems is difficult because each component has different constraints in terms of the granularity and type of data that it ingests. New data is often generated during intermediate computations, and text streams may be split into smaller, independent fragments (such as documents to sentences) which may then be re-aggregated at later parts of the computation. Due to this complexity, existing systems to serve compound AI queries do not fully take advantage of parallelism and pipelining opportunities. We present Alto, a framework that automatically optimizes execution of compound AI queries through streaming and parallelism. Bento introduces a new abstraction called nested ancestry, a metadata hierarchy that allows the system to correctly track partial outputs and aggregate data across the heterogeneous constraints of the components of compound AI applications. This metadata is automatically inferred from the programming model, allowing developers to express complex dataflow patterns without needing to reason manually about the details of routing and aggregation. Implementations of four applications in Alto outperform or match implementations in LangGraph, a popular existing AI programming framework. Alto implementations match or improve latency by between 10-30%.

cs.AI↗

arXiv2021

Towards Retina-Quality VR Video Streaming: 15ms Could Save You 80% of Your Bandwidth

Virtual reality systems today cannot yet stream immersive, retina-quality virtual reality video over a network. One of the greatest challenges to this goal is the sheer data rates required to transmit retina-quality video frames at high resolutions and frame rates. Recent work has leveraged the decay of visual acuity in human perception in novel gaze-contingent video compression techniques. In this paper, we show that reducing the motion-to-photon latency of a system itself is a key method for improving the compression ratio of gaze-contingent compression. Our key finding is that a client and streaming server system with sub-15ms latency can achieve 5x better compression than traditional techniques while also using simpler software algorithms than previous work.

cs.NI↗

arXiv2020

Approximate Partition Selection for Big-Data Workloads using Summary Statistics

Many big-data clusters store data in large partitions that support access at a coarse, partition-level granularity. As a result, approximate query processing via row-level sampling is inefficient, often requiring reads of many partitions. In this work, we seek to answer queries quickly and approximately by reading a subset of the data partitions and combining partial answers in a weighted manner without modifying the data layout. We illustrate how to efficiently perform this query processing using a set of pre-computed summary statistics, which inform the choice of partitions and weights. We develop novel means of using the statistics to assess the similarity and importance of partitions. Our experiments on several datasets and data layouts demonstrate that to achieve the same relative error compared to uniform partition sampling, our techniques offer from 2.7$\times$ to $70\times$ reduction in the number of partitions read, and the statistics stored per partition require fewer than 100KB.

cs.DB↗

arXiv2020

GRIP: A Graph Neural Network Accelerator Architecture

We present GRIP, a graph neural network accelerator architecture designed for low-latency inference. AcceleratingGNNs is challenging because they combine two distinct types of computation: arithmetic-intensive vertex-centric operations and memory-intensive edge-centric operations. GRIP splits GNN inference into a fixed set of edge- and vertex-centric execution phases that can be implemented in hardware. We then specialize each unit for the unique computational structure found in each phase.For vertex-centric phases, GRIP uses a high performance matrix multiply engine coupled with a dedicated memory subsystem for weights to improve reuse. For edge-centric phases, GRIP use multiple parallel prefetch and reduction engines to alleviate the irregularity in memory accesses. Finally, GRIP supports severalGNN optimizations, including a novel optimization called vertex-tiling which increases the reuse of weight data.We evaluate GRIP by performing synthesis and place and route for a 28nm implementation capable of executing inference for several widely-used GNN models (GCN, GraphSAGE, G-GCN, and GIN). Across several benchmark graphs, it reduces 99th percentile latency by a geometric mean of 17x and 23x compared to a CPU and GPU baseline, respectively, while drawing only 5W.

cs.AR↗

arXiv2020

Design Considerations for Low Power Internet Protocols

Over the past 10 years, low-power wireless networks have transitioned to supporting IPv6 connectivity through 6LoWPAN, a set of standards which specify how to aggressively compress IPv6 packets over low-power wireless links such as 802.15.4. We find that different low-power IPv6 stacks are unable to communicate using 6LoWPAN, and therefore IP, due to design tradeoffs between code size and energy efficiency. We argue that applying traditional protocol design principles to low-power networks is responsible for these failures, in part because receivers must accommodate a wide range of senders. Based on these findings, we propose three design principles for Internet protocols on low-power networks. These principles are based around the importance of providing flexible tradeoffs between code size and energy efficiency. We apply these principles to 6LoWPAN and show that the resulting design of the protocol provides developers a wide range of tradeoff points while allowing implementations with different choices to seamlessly communicate.

cs.NI↗

arXiv2019

Learning in situ: a randomized experiment in video streaming

We describe the results of a randomized controlled trial of video-streaming algorithms for bitrate selection and network prediction. Over the last eight months, we have streamed 14.2 years of video to 56,000 users across the Internet. Sessions are randomized in blinded fashion among algorithms, and client telemetry is recorded for analysis. We found that in this real-world setting, it is difficult for sophisticated or machine-learned control schemes to outperform a "simple" scheme (buffer-based control), notwithstanding good performance in network emulators or simulators. We performed a statistical analysis and found that the variability and heavy-tailed nature of network and algorithm behavior create hurdles for robust learned algorithms in this area. We developed an ABR algorithm that robustly outperforms other schemes in practice, by combining classical control with a learned network predictor, trained with supervised learning in situ on data from the real deployment environment. To support further investigation, we are publishing an archive of traces and results each day, and will open our ongoing study to the community. We welcome other researchers to use this platform to develop and validate new algorithms for bitrate selection, network prediction, and congestion control.

cs.NI↗

arXiv2019

Smart Contracts for Machine-to-Machine Communication: Possibilities and Limitations

Blockchain technologies, such as smart contracts, present a unique interface for machine-to-machine communication that provides a secure, append-only record that can be shared without trust and without a central administrator. We study the possibilities and limitations of using smart contracts for machine-to-machine communication by designing, implementing, and evaluating AGasP, an application for automated gasoline purchases. We find that using smart contracts allows us to directly address the challenges of transparency, longevity, and trust in IoT applications. However, real-world applications using smart contracts must address their important trade-offs, such as performance, privacy, and the challenge of ensuring they are written correctly.

cs.CY↗

arXiv2018

Locality-Sensitive Hashing for Earthquake Detection: A Case Study of Scaling Data-Driven Science

In this work, we report on a novel application of Locality Sensitive Hashing (LSH) to seismic data at scale. Based on the high waveform similarity between reoccurring earthquakes, our application identifies potential earthquakes by searching for similar time series segments via LSH. However, a straightforward implementation of this LSH-enabled application has difficulty scaling beyond 3 months of continuous time series data measured at a single seismic station. As a case study of a data-driven science workflow, we illustrate how domain knowledge can be incorporated into the workload to improve both the efficiency and result quality. We describe several end-to-end optimizations of the analysis pipeline from pre-processing to post-processing, which allow the application to scale to time series data measured at multiple seismic stations. Our optimizations enable an over 100$\times$ speedup in the end-to-end analysis pipeline. This improved scalability enabled seismologists to perform seismic analysis on more than ten years of continuous time series data from over ten seismic stations, and has directly enabled the discovery of 597 new earthquakes near the Diablo Canyon nuclear power plant in California and 6123 new earthquakes in New Zealand.

cs.DB↗

arXiv2018

Fonduer: Knowledge Base Construction from Richly Formatted Data

We focus on knowledge base construction (KBC) from richly formatted data. In contrast to KBC from text or tabular data, KBC from richly formatted data aims to extract relations conveyed jointly via textual, structural, tabular, and visual expressions. We introduce Fonduer, a machine-learning-based KBC system for richly formatted data. Fonduer presents a new data model that accounts for three challenging characteristics of richly formatted data: (1) prevalent document-level relations, (2) multimodality, and (3) data variety. Fonduer uses a new deep-learning model to automatically capture the representation (i.e., features) needed to learn how to extract relations from richly formatted data. Finally, Fonduer provides a new programming model that enables users to convert domain expertise, based on multiple modalities of information, to meaningful signals of supervision for training a KBC system. Fonduer-based KBC systems are in production for a range of use cases, including at a major online retailer. We compare Fonduer against state-of-the-art KBC approaches in four different domains. We show that Fonduer achieves an average improvement of 41 F1 points on the quality of the output knowledge base---and in some cases produces up to 1.87x the number of correct entries---compared to expert-curated public knowledge bases. We also conduct a user study to assess the usability of Fonduer's new programming model. We show that after using Fonduer for only 30 minutes, non-domain experts are able to design KBC systems that achieve on average 23 F1 points higher quality than traditional machine-learning-based KBC approaches.

cs.DB↗