arXiv subjects

Sihang Liu

Publications and source records attributed to Sihang Liu.

26 records · Page 2Linked to original sources

arXiv2024

GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions

LLMs have been widely adopted across many real-world applications. However, their widespread use comes with significant environmental costs due to their high computational intensity and resource demands. Specifically, this has driven the development of new generations of high-performing GPUs, exacerbating the problem of electronic waste and accelerating the premature disposal of devices. To address this problem, this paper focuses on reducing the carbon emissions of LLM serving by reusing older, low-performing GPUs. We present GreenLLM, an SLO-aware LLM serving framework designed to minimize carbon emissions by reusing older GPUs. GreenLLM builds on two identified use cases that disaggregate specific computations onto older GPUs, reducing carbon emissions while meeting performance goals. To deepen our understanding of the potential carbon savings from disaggregation, we also provide a theoretical analysis of its relationship with carbon intensity and GPU lifetime. Our evaluations show that GreenLLM reduces carbon emissions by up to 40.6% compared to running standard LLM serving on new GPU only, meeting latency SLOs for over 90% of requests across various applications, latency requirements, carbon intensities, and GPU lifetimes.

cs.AR↗

arXiv2024

FlexCache: Flexible Approximate Cache System for Video Diffusion

Text-to-Video applications receive increasing attention from the public. Among these, diffusion models have emerged as the most prominent approach, offering impressive quality in visual content generation. However, it still suffers from substantial computational complexity, often requiring several minutes to generate a single video. While prior research has addressed the computational overhead in text-to-image diffusion models, the techniques developed are not directly suitable for video diffusion models due to the significantly larger cache requirements and enhanced computational demands associated with video generation. We present FlexCache, a flexible approximate cache system that addresses the challenges in two main designs. First, we compress the caches before saving them to storage. Our compression strategy can reduce 6.7 times consumption on average. Then we find that the approximate cache system can achieve higher hit rate and computation savings by decoupling the object and background. We further design a tailored cache replacement policy to support the two techniques mentioned above better. Through our evaluation, FlexCache reaches 1.26 times higher throughput and 25% lower cost compared to the state-of-the-art diffusion approximate cache system.

cs.MM↗

arXiv2024

A Flat Plasmonic Biosensing Interface on Optical Fiber End-Facet via SPP-MIM Hybridization

We found that the specific dispersion of metal-insulator-metal (MIM) waveguide allows the hybridization of surface plasmon polaritons (SPPs) and the waveguide, which is not possible with dielectric waveguides. The SPP-MIM hybridization structure forms such a meta-film that integrates the previously incompatible respective merits of SPR and LSPR, including flat interfaces, high sensitivities, short evanescent fields and easy coupling with confined light. On the other hand, to achieve stable and reproducible performance is one of the greatest unresolved challenges for the development of nanophotonic biosensors. We point out that the key is to obtain well-controlled biomolecular behaviors using simple physical interfaces, for which the SPP-MIM meta-film provides a capable solution. We embed the SPP-MIM meta-film with a plasmonic crystal cavity and integrate it on a single-mode fiber's end-facet to detect biomolecular interactions. This device demonstrates highly reproducible sensorgrams and convincing detection of biotinylated proteins at down to 30 fM, with the sensorgrams following the Langmuir model. By unprecedentedly having both high sensitivity and high reproducibility, our device proposal provides a comprehensive solution for optical fiber-tip plasmonic devices to turn into a useful industrial biosensing technology.

physics.optics↗

arXiv2024

Uncertainty-Aware Decarbonization for Datacenters

This paper represents the first effort to quantify uncertainty in carbon intensity forecasting for datacenter decarbonization. We identify and analyze two types of uncertainty -- temporal and spatial -- and discuss their system implications. To address the temporal dynamics in quantifying uncertainty for carbon intensity forecasting, we introduce a conformal prediction-based framework. Evaluation results show that our technique robustly achieves target coverages in uncertainty quantification across various significance levels. We conduct two case studies using production power traces, focusing on temporal and spatial load shifting respectively. The results show that incorporating uncertainty into scheduling decisions can prevent a 5% and 14% increase in carbon emissions, respectively. These percentages translate to an absolute reduction of 2.1 and 10.4 tons of carbon emissions in a 20 MW datacenter cluster.

cs.DC↗

arXiv2023

Efficient 2D Graph SLAM for Sparse Sensing

Simultaneous localization and mapping (SLAM) plays a vital role in mapping unknown spaces and aiding autonomous navigation. Virtually all state-of-the-art solutions today for 2D SLAM are designed for dense and accurate sensors such as laser range-finders (LiDARs). However, these sensors are not suitable for resource-limited nano robots, which become increasingly capable and ubiquitous nowadays, and these robots tend to mount economical and low-power sensors that can only provide sparse and noisy measurements. This introduces a challenging problem called SLAM with sparse sensing. This work addresses the problem by adopting the form of the state-of-the-art graph-based SLAM pipeline with a novel frontend and an improvement for loop closing in the backend, both of which are designed to work with sparse and uncertain range data. Experiments show that the maps constructed by our algorithm have superior quality compared to prior works on sparse sensing. Furthermore, our method is capable of running in real-time on a modern PC with an average processing time of 1/100th the input interval time.

cs.RO↗

arXiv2023

NearPM: A Near-Data Processing System for Storage-Class Applications

Persistent Memory (PM) technologies enable program recovery to a consistent state in a case of failure. To ensure this crash-consistent behavior, programs need to enforce persist ordering by employing mechanisms, such as logging and checkpointing, which introduce additional data movement. The emerging near-data processing (NDP) architec-tures can effectively reduce this data movement overhead. In this work we propose NearPM, a near data processor that supports accelerable primitives in crash consistent programs. Using these primitives NearPM accelerate commonly used crash consistency mechanisms logging, checkpointing, and shadow-paging. NearPM further reduces the synchronization overheads between the NDP and the CPU to guarantee persistent ordering by moving ordering handling near memory. We ensures a correct persist ordering between CPU and NDP devices, as well as among multiple NDP devices with Partitioned Persist Ordering (PPO). We prototype NearPM on an FPGA platform.1 NearPM executes data-intensive operations in crash consistency mechanisms with correct ordering guarantees while the rest of the program runs on the CPU. We evaluate nine PM workloads, where each work load supports three crash consistency mechanisms -logging, checkpointing, and shadow paging. Overall, NearPM achieves 4.3-9.8X speedup in the NDP-offloaded operations and 1.22-1.35X speedup in end-to-end execution.

cs.CE↗

arXiv2022

Long-Term Mentoring for Computer Science Researchers

Early in the pandemic, we -- leaders in the research areas of programming languages (PL) and computer architecture (CA) -- realized that we had a problem: the only way to form new lasting connections in the community was to already have lasting connections in the community. Both of our academic communities had wonderful short-term mentoring programs to address this problem, but it was clear that we needed long-term mentoring programs. Those of us in CA approached this scientifically, making an evidence-backed case for community-wide long-term mentoring. In the meantime, one of us in PL had impulsively launched an unofficial long-term mentoring program, founded on chaos and spreadsheets. In January 2021, the latter grew to an official cross-institutional long-term mentoring program called SIGPLAN-M; in January 2022, the former grew to Computer Architecture Long-term Mentoring (CALM). The impacts have been strong: SIGPLAN-M reaches 328 mentees and 234 mentors across 41 countries, and mentees have described it as "life changing" and "a career saver." And while CALM is in its pilot phase -- with 13 mentors and 21 mentees across 7 countries -- it has received very positive feedback. The leaders of SIGPLAN-M and CALM shared our designs, impacts, and challenges along the way. Now, we wish to share those with you. We hope this will kick-start a larger long-term mentoring effort across all of computer science.

cs.CY↗

arXiv2017

Deconstructing the Tail at Scale Effect Across Network Protocols

Network latencies have become increasingly important for the performance of web servers and cloud computing platforms. Identifying network-related tail latencies and reasoning about their potential causes is especially important to gauge application run-time in online data-intensive applications, where the 99th percentile latency of individual operations can significantly affect the the overall latency of requests. This paper deconstructs the "tail at scale" effect across TCP-IP, UDP-IP, and RDMA network protocols. Prior scholarly works have analyzed tail latencies caused by extrinsic network parameters like network congestion and flow fairness. Contrary to existing literature, we identify surprising rare tails in TCP-IP round-trip measurements that are as enormous as 110x higher than the median latency. Our experimental design eliminates network congestion as a tail-inducing factor. Moreover, we observe similar extreme tails in UDP-IP packet exchanges, ruling out additional TCP-IP protocol operations as the root cause of tail latency. However, we are unable to reproduce similar tail latencies in RDMA packet exchanges, which leads us to conclude that the TCP/UDP protocol stack within the operating system kernel is likely the primary source of extreme latency tails.

cs.NI↗