arXiv subjects

Mingtao Hu

Publications and source records attributed to Mingtao Hu.

6 recordsLinked to original sources

arXiv2026

SuperInfer: SLO-Aware Rotary Scheduling and Memory Management for LLM Inference on Superchips

Large Language Model (LLM) serving faces a fundamental tension between stringent latency Service Level Objectives (SLOs) and limited GPU memory capacity. When high request rates exhaust the KV cache budget, existing LLM inference systems often suffer severe head-of-line (HOL) blocking. While prior work explored PCIe-based offloading, these approaches cannot sustain responsiveness under high request rates, often failing to meet tight Time-To-First-Token (TTFT) and Time-Between-Tokens (TBT) SLOs. We present SuperInfer, a high-performance LLM inference system designed for emerging Superchips (e.g., NVIDIA GH200) with tightly coupled GPU-CPU architecture via NVLink-C2C. SuperInfer introduces RotaSched, the first proactive, SLO-aware rotary scheduler that rotates requests to maintain responsiveness on Superchips, and DuplexKV, an optimized rotation engine that enables full-duplex transfer over NVLink-C2C. Evaluations on GH200 using various models and datasets show that SuperInfer improves TTFT SLO attainment rates by up to 74.7% while maintaining comparable TBT and throughput compared to state-of-the-art systems, demonstrating that SLO-aware scheduling and memory co-design unlocks the full potential of Superchips for responsive LLM serving. Code is available in https://github.com/Supercomputing-System-AI-Lab/SuperInfer.

cs.DC↗

arXiv2025

Compute-in-Memory Implementation of State Space Models for Event Sequence Processing

State space models (SSMs) have recently emerged as a powerful framework for long sequence processing, outperforming traditional methods on diverse benchmarks. Fundamentally, SSMs can generalize both recurrent and convolutional networks and have been shown to even capture key functions of biological systems. Here we report an approach to implement SSMs in energy-efficient compute-in-memory (CIM) hardware to achieve real-time, event-driven processing. Our work re-parameterizes the model to function with real-valued coefficients and shared decay constants, reducing the complexity of model mapping onto practical hardware systems. By leveraging device dynamics and diagonalized state transition parameters, the state evolution can be natively implemented in crossbar-based CIM systems combined with memristors exhibiting short-term memory effects. Through this algorithm and hardware co-design, we show the proposed system offers both high accuracy and high energy efficiency while supporting fully asynchronous processing for event-based vision and audio tasks.

eess.SP↗

arXiv2025

State Space Models Naturally Produce Time Cell and Oscillatory Behaviors and Scale to Abstract Cognitive Functions

A grand challenge in modern neuroscience is to bridge the gap between the detailed mapping of microscale neural circuits and mechanistic understanding of cognitive functions. While extensive knowledge exists about neuronal connectivity and biophysics, how these low-level phenomena eventually produce abstract behaviors remains largely unresolved. Here, we propose that a model based on State Space Models, an emerging class of deep learning architectures, can be a potential biological model for analysis. We suggest that the differential equations governing elements in a State Space Model are conceptually consistent with the dynamics of biophysical processes, while the model offers a scalable framework to build on the dynamics to produce emergent behaviors observed in experimental neuroscience. We test this model by training a network employing a diagonal state transition matrix on temporal discrimination tasks with reinforcement learning. Our results suggest that neural behaviors such as time cells naturally emerge from two fundamental principles: optimal pre-configuration and rotational dynamics. These features are shown mathematically to optimize history compression, and naturally generate structured temporal dynamics even prior to training, mirroring recent findings in biological circuits. We show that learning acts primarily as a selection mechanism that fine-tunes these pre-configured oscillatory modes, rather than constructing temporal codes de novo. The model can be readily scaled to abstract cognitive functions such as event counting, supporting the use of State Space Models as a computationally tractable framework for understanding neural activities.

q-bio.NC↗

arXiv2024

Nanoscale Engineering of Wurtzite Ferroelectrics: Unveiling Phase Transition and Ferroelectric Switching in ScAlN Nanowires

The pursuit of extreme device miniaturization and the exploration of novel physical phenomena have spurred significant interest in crystallographic phase control and ferroelectric switching in reduced dimensions. Recently, wurtzite ferroelectrics have emerged as a new class of functional materials, offering intriguing piezoelectric and ferroelectric properties, CMOS compatibility, and seamless integration with mainstream semiconductor technology. However, the exploration of crystallographic phase and ferroelectric switching in reduced dimensions, especially in nanostructures, has remained a largely uncharted territory. In this study, we present the first comprehensive investigation into the crystallographic phase transition of ScAlN nanowires across the full Sc compositional range. While a gradual transition from wurtzite to cubic phase was observed with increasing Sc composition, we further demonstrated that a highly ordered wurtzite phase ScAlN could be confined at the ScAlN/GaN interface for Sc contents surpassing what is possible in conventional films, holding great potential to addressing the fundamental high coercive field of wurtzite ferroelectrics. In addition, we provide the first evidence of ferroelectric switching in ScAlN nanowires, a result that holds significant implications for future device miniaturization. Our demonstration of tunable ferroelectric ScAlN nanowires opens new possibilities for nanoscale, domain, alloy, strain, and quantum engineering of wurtzite ferroelectrics, representing a significant stride towards the development of next-generation, miniaturized devices based on wurtzite ferroelectrics.

cond-mat.mtrl-sci↗

arXiv2023

Electric-Field-Induced Domain Walls in Wurtzite Ferroelectrics

Wurtzite ferroelectrics possess transformative potential for next-generation microelectronics. A comprehensive understanding of their ferroelectric properties and domain energetics is crucial for tailoring their ferroelectric characteristics and exploiting their functional properties in practical devices. Despite burgeoning interest, the exact configurations, and electronic structures of the domain walls in wurtzite ferroelectrics remain elusive. In this work, we elucidate the atomic configurations and electronic properties of electric-field-induced domain walls in ferroelectric ScGaN. By combining transmission electron microscopy and theoretical calculations, a novel charged domain wall with a buckled two-dimensional hexagonal phase is revealed. The dangling bonds associated with these domain walls give rise to unprecedented metallic-like mid-gap states within the forbidden band. Quantitative analysis further unveils a universal charge-compensation mechanism stabilizing antipolar domain walls in ferroelectric materials, wherein the polarization discontinuity at the 180{\deg} domain wall is compensated by the dangling bond electrons. Furthermore, the reconfigurable conductivity of these domain walls is experimentally demonstrated, showcasing their potential for ultra-scaled device applications. Our findings represent a pivotal advancement in understanding the structural and electronic properties of wurtzite ferroelectric domain walls and lay the groundwork for fundamental physics studies and device applications.

cond-mat.mtrl-sci↗

arXiv2023

LLM-Mini-CEX: Automatic Evaluation of Large Language Model for Diagnostic Conversation

There is an increasing interest in developing LLMs for medical diagnosis to improve diagnosis efficiency. Despite their alluring technological potential, there is no unified and comprehensive evaluation criterion, leading to the inability to evaluate the quality and potential risks of medical LLMs, further hindering the application of LLMs in medical treatment scenarios. Besides, current evaluations heavily rely on labor-intensive interactions with LLMs to obtain diagnostic dialogues and human evaluation on the quality of diagnosis dialogue. To tackle the lack of unified and comprehensive evaluation criterion, we first initially establish an evaluation criterion, termed LLM-specific Mini-CEX to assess the diagnostic capabilities of LLMs effectively, based on original Mini-CEX. To address the labor-intensive interaction problem, we develop a patient simulator to engage in automatic conversations with LLMs, and utilize ChatGPT for evaluating diagnosis dialogues automatically. Experimental results show that the LLM-specific Mini-CEX is adequate and necessary to evaluate medical diagnosis dialogue. Besides, ChatGPT can replace manual evaluation on the metrics of humanistic qualities and provides reproducible and automated comparisons between different LLMs.

cs.CL↗