arXiv subjects

Diyu Zhou

Publications and source records attributed to Diyu Zhou.

6 recordsLinked to original sources

arXiv2025

SLOTH: Lightweight Detection and Localization of On-Chip Fail-Slow Failures for DNN Accelerators

Spatial DNN accelerators are essential for high-performance inference, but their performance is undermined by widespread fail-slow failures. Detecting such failures on-chip is challenging, as prior methods from distributed systems are unsuitable due to strict memory limits and their inability to track failures across the hardware topology. We present SLOTH, a lightweight, hardware-aware framework for practical on-chip fail-slow detection in DNN accelerators. SLOTH combines workload-aware instrumentation for operator-level monitoring with minimal overhead, on-the-fly trace compression to operate within kilobytes of memory, and a novel topology-aware ranking algorithm to pinpoint a failure's root cause. We evaluate SLOTH on a wide range of representative DNN workloads. The results demonstrate that SLOTH reduces the storage overhead by an average of 115.9$\times$, while achieving an average fail-slow detection accuracy from 69.68\% to 86.69\%.

cs.AR↗

arXiv2025

XBOF: A Cost-Efficient CXL JBOF with Inter-SSD Compute Resource Sharing

Enterprise SSDs integrate numerous computing resources (e.g., ARM processor and onboard DRAM) to satisfy the ever-increasing performance requirements of I/O bursts. While these resources substantially elevate the monetary costs of SSDs, the sporadic nature of I/O bursts causes severe SSD resource underutilization in just a bunch of flash (JBOF) level. Tackling this challenge, we propose XBOF, a cost-efficient JBOF design, which only reserves moderate computing resources in SSDs at low monetary cost, while achieving demanded I/O performance through efficient inter-SSD resource sharing. Specifically, XBOF first disaggregates SSD architecture into multiple disjoint parts based on their functionality, enabling fine-grained SSD internal resource management. XBOF then employs a decentralized scheme to manage these disaggregated resources and harvests the computing resources of idle SSDs to assist busy SSDs in handling I/O bursts. This idea is facilitated by the cache-coherent capability of Compute eXpress Link (CXL), with which the busy SSDs can directly utilize the harvested computing resources to accelerate metadata processing. The evaluation results show that XBOF improves SSD resource utilization by 50.4% and saves 19.0% monetary costs with a negligible performance loss, compared to existing JBOF designs.

cs.OS↗

arXiv2025

Asterinas: A Linux ABI-Compatible, Rust-Based Framekernel OS with a Small and Sound TCB

How can one build a feature-rich, general-purpose, Rust-based operating system (OS) with a minimal and sound Trusted Computing Base (TCB) for memory safety? Existing Rust-based OSes fall short due to their improper use of unsafe Rust in kernel development. To address this challenge, we propose a novel OS architecture called framekernel that realizes Rust's full potential to achieve intra-kernel privilege separation, ensuring TCB minimality and soundness. We present OSTD, a streamlined framework for safe Rust OS development, and Asterinas, a Linux ABI-compatible framekernel OS implemented entirely in safe Rust using OSTD. Supporting over 210 Linux system calls, Asterinas delivers performance on par with Linux, while maintaining a minimized, memory-safety TCB of only about 14.0% of the codebase. These results underscore the practicality and benefits of the framekernel architecture in building safe and efficient OSes.

cs.OS↗

arXiv2025

Memory Offloading for Large Language Model Inference with Latency SLO Guarantees

Offloading large language models (LLMs) state to host memory during inference promises to reduce operational costs by supporting larger models, longer inputs, and larger batch sizes. However, the design of existing memory offloading mechanisms does not take latency service-level objectives (SLOs) into consideration. As a result, they either lead to frequent SLO violations or underutilize host memory, thereby incurring economic loss and thus defeating the purpose of memory offloading. This paper presents Select-N, a latency-SLO-aware memory offloading system for LLM serving. A key challenge in designing Select-N is to reconcile the tension between meeting SLOs and maximizing host memory usage. Select-N overcomes it by exploiting a unique characteristic of modern LLMs: during serving, the computation time of each decoder layer is deterministic. Leveraging this, Select-N introduces offloading interval, an internal tunable knob that captures the tradeoff between SLOs and host memory usage, thereby reducing the aforementioned challenge to pick an optimal offloading interval. With that, Select-N proposes a two-stage approach to automatically pick the offloading interval. The first stage is offline that generates the range of optimal offloading interval, while the second stage adjusts offloading interval at the granularity of inference iteration based on runtime hardware status. Our evaluation shows that Select-N consistently meets SLOs and improves the serving throughput over existing mechanisms by 1.85X due to maximizing the use of host memory.

cs.DC↗

arXiv2023

Productive Development of Scalable Network Functions with NFork

Despite decades of research, developing correct and scalable concurrent programs is still challenging. Network functions (NFs) are not an exception. This paper presents NFork, a system that helps NF domain experts to productively develop concurrent NFs by abstracting away concurrency from developers. The key scheme behind NFork's design is to exploit NF characteristics to overcome the limitations of prior work on concurrency programming. Developers write NFs as sequential programs, and during runtime, NFork performs transparent parallelization by processing packets in different cores. Exploiting NF characteristics, NFork leverages transactional memory and develops efficient concurrent data structures to achieve scalability and guarantee the absence of concurrency bugs. Since NFork manages concurrency, it further provides (i) a profiler that reveals the root causes of scalability bottlenecks inherent to the NF's semantics and (ii) actionable recipes for developers to mitigate these root causes by relaxing the NF's semantics. We show that NFs developed with NFork achieve competitive scalability with those in Cisco VPP [16], and NFork's profiler and recipes can effectively aid developers in optimizing NF scalability.

cs.NI↗

arXiv2021

HyCoR: Fault-Tolerant Replicated Containers Based on Checkpoint and Replay

HyCoR is a fully-operational fault tolerance mechanism for multiprocessor workloads, based on container replication, using a hybrid of checkpointing and replay. HyCoR derives from two insights regarding replication mechanisms: 1) deterministic replay can overcome a key disadvantage of checkpointing alone -- unacceptably long delays of outputs to clients, and 2) checkpointing can overcome a key disadvantage of active replication with deterministic replay alone -- vulnerability to even rare replay failures due to an untracked nondeterministic events. With HyCoR, the primary sends periodic checkpoints to the backup and logs the outcomes of sources of nondeterminism. Outputs to clients are delayed only by the short time it takes to send the corresponding log to the backup. Upon primary failure, the backup replays only the short interval since the last checkpoint, thus minimizing the window of vulnerability. HyCoR includes a "best effort" mechanism that results in a high recovery rate even in the presence of data races, as long as their rate is low. The evaluation includes measurement of the recovery rate and recovery latency based on fault injection. On average, HyCoR delays responses to clients by less than 1ms and recovers in less than 1s. For a set of eight real-world benchmarks, if data races are eliminated, the performance overhead of HyCoR is under 59%.

cs.DC↗