Searcharxiv⌕ Search

arXiv subjects

Hayden So

Publications and source records attributed to Hayden So.

3 recordsLinked to original sources

A Per-Step Load-Balancing Engine for Expert-Parallel MoE Serving

In expert-parallel Mixture-of-Experts serving, a layer finishes only when the most loaded GPU of its group does. Token routing shifts at every decode step and concentrates on a few experts, so most of the group's reserved GPU time waits on stragglers. Recovering that time means re-deciding, at every layer of every decode step, how each replicated expert's tokens divide between its copies. The exact linear-program solvers that make this decision optimally take longer than many of the layers they serve, and their working set outgrows GPU shared memory at deployment scale. We introduce Relief, a bidirectional pairwise relaxation that makes the linear program unnecessary, coming within a fraction of a point of the per-step optimum in a few passes of integer halving over the token counts. We build and measure three deployments, CUDA kernels on the serving GPU, the same verified engine on an FPGA over PCIe, and a fixed-function block beside the accelerator. On decode routing from three open MoE families, Relief holds the balance near the per-step optimum at every group size and layer time measured. Scored as effective utilization relative to that optimum, the kernel alone matches or exceeds every deployable scheme at every group size, from the hundred-microsecond layer on. The on-die engine widens that lead at every point, recovering up to 37.3 percentage points over the deployed static placement, while adding 0.02% of a current accelerator's transistors.

cs.AR↗

Analysis of Parallel Boarding Methods in a Multi-Aisle Flying Wing Aircraft

We examine the speed of different boarding methods in a proposed Flying Wing aircraft design with four aisles using an agent-based model. We study the effect of various passenger movement variables on the boarding process. We evaluate the impact of these factors on the boarding time when the boarding process runs sequentially and in parallel with the aisles of the Flying Wing layout. Then, we analyze the impact of an increase in the number of aisles on the relative speed of all boarding methods and conclude that methods utilizing boarding of the separate aisles simultaneously (parallel boarding) converge to the fastest boarding time given by the Steffen method. With parallel boarding of the aisles the relative advantage of the Steffen method compared to Windows-Middle-Aisle (WMA) or Back-to-front boarding decreases, from being 1.6-2.1 times as fast to being approximately equal for our fiducial Flying Wing seating arrangement. Standard methods such as Back-to-front or WMA are about twice as fast to board a four-aisle Flying Wing plane, compared to a single-aisle aircraft with the same number of passengers. We also investigate the difference between the optimal approach to parallel boarding, where consecutive passengers always enter separate aisles, and a less optimal but more practical approach. The practical approach is only up to 1.06 times slower than the optimal, meaning that the advantages of parallel boarding can be utilized without resorting to impractical boarding methods. Hence, the introduction of multiple aisles into aircraft seating design offers the prospect of significantly decreasing the boarding time for passengers, without the introduction of inconvenient boarding methods.

physics.soc-ph↗

Random resistive memory-based deep extreme point learning machine for unified visual processing

Visual sensors, including 3D LiDAR, neuromorphic DVS sensors, and conventional frame cameras, are increasingly integrated into edge-side intelligent machines. Realizing intensive multi-sensory data analysis directly on edge intelligent machines is crucial for numerous emerging edge applications, such as augmented and virtual reality and unmanned aerial vehicles, which necessitates unified data representation, unprecedented hardware energy efficiency and rapid model training. However, multi-sensory data are intrinsically heterogeneous, causing significant complexity in the system development for edge-side intelligent machines. In addition, the performance of conventional digital hardware is limited by the physically separated processing and memory units, known as the von Neumann bottleneck, and the physical limit of transistor scaling, which contributes to the slowdown of Moore's law. These limitations are further intensified by the tedious training of models with ever-increasing sizes. We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM), that offers efficient unified point set analysis. We show the system's versatility across various data modalities and two different learning tasks. Compared to a conventional digital hardware-based system, our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems. Our random resistive memory-based deep extreme point learning machine may pave the way for energy-efficient and training-friendly edge AI across various data modalities and tasks.

cs.LG↗