arXiv subjects

Ting Cao

Publications and source records attributed to Ting Cao.

At least 91 records · Page 5Linked to original sources

arXiv2025

Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling

Video activity recognition has become increasingly important in robots and embodied AI. Recognizing continuous video activities poses considerable challenges due to the fast expansion of streaming video, which contains multi-scale and untrimmed activities. We introduce a novel system, CARS, to overcome these issues through adaptive video context modeling. Adaptive video context modeling refers to selectively maintaining activity-related features in temporal and spatial dimensions. CARS has two key designs. The first is an activity spatial feature extraction by eliminating irrelevant visual features while maintaining recognition accuracy. The second is an activity-aware state update introducing dynamic adaptability to better preserve the video context for multi-scale activity recognition. Our CARS runs at speeds $>$30 FPS on typical edge devices and outperforms all baselines by 1.2\% to 79.7\% in accuracy. Moreover, we explore applying CARS to a large video model as a video encoder. Experimental results show that our CARS can result in a 0.46-point enhancement (on a 5-point scale) on the in-distribution video activity dataset, and an improvement ranging from 1.19\% to 4\% on zero-shot video activity datasets.

cs.CV↗

arXiv2025

Magnetoelectric Control of Helical Light Emission in a Moiré Chern Magnet

Magnetoelectric effects and their coupling to light helicity are important for both fundamental science and applications in sensing, communication, and data storage. Traditional approaches require complex device architectures, involving separate spin-injection, ferromagnetic, and optically active layers. Recently, the emergence of 2D semiconductor moiré superlattices with flat Chern bands and strong light-matter interactions has established a simple yet powerful platform for exploring the coupling between photon, electron, and spin degrees of freedom. Here, we report efficient current control of spontaneous ferromagnetism and associated helicity of light emission in moiré MoTe2 bilayer - a system which hosts a rich variety of topological phases, including newly discovered zero-field fractional Chern insulators. We show that the current control is effective over a wide range of doping of the first moiré Chern band, implying the uniformity of the Berry curvature distribution over the flat band. By setting the system into the anomalous Hall metal phase, a current as small as 10nA is sufficient to switch the magnetic order, a substantial improvement over both conventional spin torque architectures and other moiré systems. The realized current control of ferromagnetism leads to continuous tuning of trion photoluminescence helicity from left to right circular via spin/valley Hall torque at zero magnetic field. Our results pave the way for topological opto-spintronics based on semiconductors with synthetic flat Chern bands.

cond-mat.mes-hall↗

arXiv2025

Higher Landau-Level Analogs and Signatures of Non-Abelian States in Twisted Bilayer MoTe$_2$

Recent experimental discovery of fractional Chern insulators at zero magnetic field in moiré superlattices has sparked intense interests in bringing Landau level physics to flat Chern bands. In twisted MoTe$_2$ bilayers (tMoTe$_2$), recent theoretical and experimental studies have found three consecutive flat Chern bands at twist angle $\sim 2^\circ$. In this work, we investigate whether higher Landau level physics can be found in these consecutive Chern bands. At twist angles $2.00^\circ$ and $1.89^\circ$, we identify four consecutive $C = 1$ bands for the $K$ valley in tMoTe$_2$. By constructing Wannier functions directly from density functional theory (DFT) calculations, a six-orbital model is developed to describe the consecutive Chern bands, with the orbitals forming a honeycomb lattice. Exact diagonalization on top of Hartree-Fock calculations are carried out with the Wannier functions. Especially, when the second moiré miniband is half-filled, signatures of non-Abelian states are found. Our Wannier-based approach in modelling moiré superlattices is faithful to DFT wave functions and can serve as benchmarks for continuum models. The possibility of realizing non-Abelian anyons at zero magnetic field also opens up a new pathway for fault-tolerant quantum information processing.

cond-mat.str-el↗

arXiv2025

SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs

Attention is the cornerstone of modern Large Language Models (LLMs). Yet its quadratic complexity hinders efficiency and scalability, especially for long-context processing. A promising approach is to leverage sparsity in attention. However, existing sparsity-based solutions predominantly rely on predefined patterns or heuristics at the attention head level, struggling to adapt dynamically to different contexts efficiently. We propose SeerAttention, a simple yet effective attention mechanism that directly learns the block-level attention sparsity from the LLM itself. Inspired by the gating mechanism in Mixture of Experts (MoE), SeerAttention augments the conventional attention with a learnable gate that selectively activates important blocks within the attention map. Specifically, the gate first pools the query (Q) and key (K) tensors along the sequence dimension and processes them through learnable linear layers. The resulting matrices are then multiplied together to produce the gating scores, which are used to predict block-level attention sparsity. Combined with our block-sparse FlashAttention kernel, SeerAttention can achieve significant speedup on GPUs. When applied to pre-trained LLMs, SeerAttention only requires training the gate parameters in a lightweight self-distillation manner, allowing rapid convergence. Our evaluation results demonstrate that SeerAttention achieves better model accuracy and lower latency for long-context pre-filling compared to prior methods. Code is available at: https://github.com/microsoft/SeerAttention

cs.CL↗

arXiv2025

Bitnet.cpp: Efficient Edge Inference for Ternary LLMs

The advent of 1-bit large language models (LLMs), led by BitNet b1.58, has spurred interest in ternary LLMs. Despite this, research and practical applications focusing on efficient edge inference for ternary LLMs remain scarce. To bridge this gap, we introduce Bitnet.cpp, an inference system optimized for BitNet b1.58 and ternary LLMs. Given that mixed-precision matrix multiplication (mpGEMM) constitutes the bulk of inference time in ternary LLMs, Bitnet.cpp incorporates a novel mpGEMM library to facilitate sub-2-bits-per-weight, efficient and lossless inference. The library features two core solutions: Ternary Lookup Table (TL), which addresses spatial inefficiencies of previous bit-wise methods, and Int2 with a Scale (I2_S), which ensures lossless edge inference, both enabling high-speed inference. Our experiments show that Bitnet.cpp achieves up to a 6.25x increase in speed over full-precision baselines and up to 2.32x over low-bit baselines, setting new benchmarks in the field. Additionally, we expand TL to element-wise lookup table (ELUT) for low-bit LLMs in the appendix, presenting both theoretical and empirical evidence of its considerable potential. Bitnet.cpp is publicly available at https://github.com/microsoft/BitNet/tree/paper , offering a sophisticated solution for the efficient and practical deployment of edge LLMs.

cs.LG↗

arXiv2025

LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator

The emergence of neural network capabilities invariably leads to a significant surge in computational demands due to expanding model sizes and increased computational complexity. To reduce model size and lower inference costs, recent research has focused on simplifying models and designing hardware accelerators using low-bit quantization. However, due to numerical representation limits, scalar quantization cannot reduce bit width lower than 1-bit, diminishing its benefits. To break through these limitations, we introduce LUT-DLA, a Look-Up Table (LUT) Deep Learning Accelerator Framework that utilizes vector quantization to convert neural network models into LUTs, achieving extreme low-bit quantization. The LUT-DLA framework facilitates efficient and cost-effective hardware accelerator designs and supports the LUTBoost algorithm, which helps to transform various DNN models into LUT-based models via multistage training, drastically cutting both computational and hardware overhead. Additionally, through co-design space exploration, LUT-DLA assesses the impact of various model and hardware parameters to fine-tune hardware configurations for different application scenarios, optimizing performance and efficiency. Our comprehensive experiments show that LUT-DLA achieves improvements in power efficiency and area efficiency with gains of $1.4$~$7.0\times$ and $1.5$~$146.1\times$, respectively, while maintaining only a modest accuracy drop. For CNNs, accuracy decreases by $0.1\%$~$3.1\%$ using the $L_2$ distance similarity, $0.1\%$~$3.4\%$ with the $L_1$ distance similarity, and $0.1\%$~$3.8\%$ when employing the Chebyshev distance similarity. For transformer-based models, the accuracy drop ranges from $1.4\%$ to $3.0\%$.

cs.AR↗

arXiv2025

LeMo: Enabling LEss Token Involvement for MOre Context Fine-tuning

The escalating demand for long-context applications has intensified the necessity of extending the LLM context windows. Despite recent fine-tuning approaches successfully expanding context lengths, their high memory footprints, especially for activations, present a critical practical limitation. Current parameter-efficient fine-tuning methods prioritize reducing parameter update overhead over addressing activation memory constraints. Similarly, existing sparsity mechanisms improve computational efficiency but overlook activation memory optimization due to the phenomenon of Shadowy Activation. In this paper, we propose LeMo, the first LLM fine-tuning system that explores and exploits a new token-level sparsity mechanism inherent in long-context scenarios, termed Contextual Token Sparsity. LeMo minimizes redundant token involvement by assessing the informativeness of token embeddings while preserving model accuracy. Specifically, LeMo introduces three key techniques: (1) Token Elimination, dynamically identifying and excluding redundant tokens across varying inputs and layers. (2) Pattern Prediction, utilizing well-trained predictors to approximate token sparsity patterns with minimal overhead. (3) Kernel Optimization, employing permutation-free and segment-based strategies to boost system performance. We implement LeMo as an end-to-end fine-tuning system compatible with various LLM architectures and other optimization techniques. Comprehensive evaluations demonstrate that LeMo reduces memory consumption by up to 1.93x and achieves up to 1.36x speedups, outperforming state-of-the-art fine-tuning systems.

cs.CL↗

arXiv2025

Adiabatic Pumping of Orbital Magnetization by Spin Precession

We propose adiabatic pumping of orbital magnetization driven by coherent spin precession, facilitating the rectification of this precession. The orbital magnetization originates from the adiabatic evolution of valence electrons with a topological bulk contribution expressed as a Chern-Simons form. When the precession cone angle of spin $\mathbf{S}$ is small, the resulting magnetization is proportional to $\mathbf{S}\times \dot{\mathbf{S}}$, contributing to the magnon Zeeman effect. With a large cone angle, the magnetization can reach its natural unit, $e/T$, in an antiferromagnetic topological insulator with $e$ as the elementary charge and $T$ as the precession period. This significant magnetization is related to the global properties of the electronic geometric phases in the parameter space spanned by $\mathbf{S}$ and momentum $\mathbf{k}$. When the pumped magnetization is inhomogeneous, induced by spin textures or electronic topological phase domains, a dissipationless charge current is also pumped. At last, we discuss the boundary contributions from the spin-driving edge states, which are intricately linked to the gauge-dependent quantum uncertainty of the Chern-Simons form.

cond-mat.mtrl-sci↗

arXiv2025

Dissecting Bit-Level Scaling Laws in Quantizing Vision Generative Models

Vision generative models have recently made significant advancements along two primary paradigms: diffusion-style and language-style, both of which have demonstrated excellent scaling laws. Quantization is crucial for efficiently deploying these models, as it reduces memory and computation costs. In this work, we systematically investigate the impact of quantization on these two paradigms. Surprisingly, despite achieving comparable performance in full precision, language-style models consistently outperform diffusion-style models across various quantization settings. This observation suggests that language-style models have superior bit-level scaling laws, offering a better tradeoff between model quality and total bits. To dissect this phenomenon, we conduct extensive experiments and find that the primary reason is the discrete representation space of language-style models, which is more tolerant of information loss during quantization. Furthermore, our analysis indicates that improving the bit-level scaling law of quantized vision generative models is challenging, with model distillation identified as a highly effective approach. Specifically, we propose TopKLD to optimize the transfer of distilled knowledge by balancing ``implicit knowledge'' and ``explicit knowledge'' during the distillation process. This approach elevates the bit-level scaling laws by one level across both integer and floating-point quantization settings.

cs.CV↗

arXiv2024

Matryoshka: Optimization of Dynamic Diverse Quantum Chemistry Systems via Elastic Parallelism Transformation

AI infrastructures, predominantly GPUs, have delivered remarkable performance gains for deep learning. Conversely, scientific computing, exemplified by quantum chemistry systems, suffers from dynamic diversity, where computational patterns are more diverse and vary dynamically, posing a significant challenge to sponge acceleration off GPUs. In this paper, we propose Matryoshka, a novel elastically-parallel technique for the efficient execution of quantum chemistry system with dynamic diversity on GPU. Matryoshka capitalizes on Elastic Parallelism Transformation, a property prevalent in scientific systems yet underexplored for dynamic diversity, to elastically realign parallel patterns with GPU architecture. Structured around three transformation primitives (Permutation, Deconstruction, and Combination), Matryoshka encompasses three core components. The Block Constructor serves as the central orchestrator, which reformulates data structures accommodating dynamic inputs and constructs fine-grained GPU-efficient compute blocks. Within each compute block, the Graph Compiler operates offline, generating high-performance code with clear computational path through an automated compilation process. The Workload Allocator dynamically schedules workloads with varying operational intensities to threads online. It achieves highly efficient parallelism for compute-intensive operations and facilitates fusion with neighboring memory-intensive operations automatically. Extensive evaluation shows that Matryoshka effectively addresses dynamic diversity, yielding acceleration improvements of up to 13.86x (average 9.41x) over prevailing state-of-the-art approaches on 13 quantum chemistry systems.

cs.DC↗

arXiv2024

Ferromagnetism and Topology of the Higher Flat Band in a Fractional Chern Insulator

The recent observation of the fractional quantum anomalous Hall effect in moiré fractional Chern insulators provides an opportunity to investigate zero magnetic field anyons. To potentially realize non-Abelian anyons, one approach is to engineer higher flat Chern bands that mimic higher Landau levels. Here, we investigate the interaction, topology, and ferromagnetism of the second moiré miniband in twisted MoTe2 bilayers. At half filling of the second miniband, we observe spontaneous ferromagnetism and an incipient Chern insulator state. The Chern numbers of the top two moiré flat bands exhibit opposite signs for twist angles above 3.1°, but share the same sign near 2.6°, consistent with theoretical predictions. In the 2.6° device, increasing magnetic field induces a topological phase transition via band crossing between opposite valleys, resulting in an emergent state with Chern number C = -2. Additionally, an insulating state at half filling of the second valley-polarized band suggests a charge-ordered state is favored over the fractional Chern insulator state. These findings lay a foundation for understanding the higher flat Chern bands, crucial for the discovery of non-Abelian fractional Chern insulators.

cond-mat.mes-hall↗

arXiv2024

VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models

Scaling model size significantly challenges the deployment and inference of Large Language Models (LLMs). Due to the redundancy in LLM weights, recent research has focused on pushing weight-only quantization to extremely low-bit (even down to 2 bits). It reduces memory requirements, optimizes storage costs, and decreases memory bandwidth needs during inference. However, due to numerical representation limitations, traditional scalar-based weight quantization struggles to achieve such extreme low-bit. Recent research on Vector Quantization (VQ) for LLMs has demonstrated the potential for extremely low-bit model quantization by compressing vectors into indices using lookup tables. In this paper, we introduce Vector Post-Training Quantization (VPTQ) for extremely low-bit quantization of LLMs. We use Second-Order Optimization to formulate the LLM VQ problem and guide our quantization algorithm design by solving the optimization. We further refine the weights using Channel-Independent Second-Order Optimization for a granular VQ. In addition, by decomposing the optimization problem, we propose a brief and effective codebook initialization algorithm. We also extend VPTQ to support residual and outlier quantization, which enhances model accuracy and further compresses the model. Our experimental results show that VPTQ reduces model quantization perplexity by $0.01$-$0.34$ on LLaMA-2, $0.38$-$0.68$ on Mistral-7B, $4.41$-$7.34$ on LLaMA-3 over SOTA at 2-bit, with an average accuracy improvement of $0.79$-$1.5\%$ on LLaMA-2, $1\%$ on Mistral-7B, $11$-$22\%$ on LLaMA-3 on QA tasks on average. We only utilize $10.4$-$18.6\%$ of the quantization algorithm execution time, resulting in a $1.6$-$1.8\times$ increase in inference throughput compared to SOTA.

cs.AI↗

arXiv2024

Anatomizing Deep Learning Inference in Web Browsers

Web applications have increasingly adopted Deep Learning (DL) through in-browser inference, wherein DL inference performs directly within Web browsers. The actual performance of in-browser inference and its impacts on the quality of experience (QoE) remain unexplored, and urgently require new QoE measurements beyond traditional ones, e.g., mainly focusing on page load time. To bridge this gap, we make the first comprehensive performance measurement of in-browser inference to date. Our approach proposes new metrics to measure in-browser inference: responsiveness, smoothness, and inference accuracy. Our extensive analysis involves 9 representative DL models across Web browsers of 50 popular PC devices and 20 mobile devices. The results reveal that in-browser inference exhibits a substantial latency gap, averaging 16.9 times slower on CPU and 4.9 times slower on GPU compared to native inference on PC devices. The gap on mobile CPU and mobile GPU is 15.8 times and 7.8 times, respectively. Furthermore, we identify contributing factors to such latency gap, including underutilized hardware instruction sets, inherent overhead in the runtime environment, resource contention within the browser, and inefficiencies in software libraries and GPU abstractions. Additionally, in-browser inference imposes significant memory demands, at times exceeding 334.6 times the size of the DL models themselves, partly attributable to suboptimal memory management. We also observe that in-browser inference leads to a significant 67.2% increase in the time it takes for GUI components to render within Web browsers, significantly affecting the overall user QoE of Web applications reliant on this technology

cs.LG↗

arXiv2024

Uniaxial plasmon polaritons $\textit{via}$ charge transfer at the graphene/CrSBr interface

Graphene is a privileged 2D platform for hosting confined light-matter excitations known as surface plasmon-polaritons (SPPs), as it possesses low intrinsic losses with a high degree of optical confinement. However, the inherently isotropic optical properties of graphene limit its ability to guide and focus SPPs, making it less suitable than anisotropic elliptical and hyperbolic materials as a platform for polaritonic lensing and canalization. Here, we present the graphene/CrSBr heterostructure as an engineered 2D interface that hosts highly anisotropic SPP propagation over a wide range of frequencies in the mid-infrared and terahertz. Using a combination of scanning tunneling microscopy (STM), scattering-type scanning near-field optical microscopy (s-SNOM), and first-principles calculations, we demonstrate mutual doping in excess of 10$^{13}$ cm$^{-2}$ holes/electrons between the interfacial layers of graphene/CrSBr heterostructures. SPPs in graphene activated by charge transfer interact with charge-induced anisotropic intra- and interband transitions in the interfacial doped CrSBr, leading to preferential SPP propagation along the quasi-1D chains that compose each CrSBr layer. This multifaceted proximity effect both creates SPPs and endows them with anisotropic transport and propagation lengths that differ by an order-of-magnitude between the two in-plane crystallographic axes of CrSBr.

cond-mat.mes-hall↗

arXiv2024

Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations

Web is increasingly becoming the primary platform to deliver AI services onto edge devices, making in-browser deep learning (DL) inference more prominent. Nevertheless, the heterogeneity of edge devices, combined with the underdeveloped state of Web hardware acceleration practices, hinders current in-browser inference from achieving its full performance potential on target devices. To address this issue, this paper presents the pioneering inbrowser inference system, nnJIT, which enables just-in-time (JIT) auto-generation of optimized computing kernels for edge devices. nnJIT is built upon two novel techniques that significantly reduce kernel search and compilation overhead while improving performance firmly: Tensor-Web Compiling Co-Design lowers compiling costs by around 100X through eliminating redundant and ineffective compiling passes; Web-Specific Lite Kernel Optimization Space reduces kernel tuning costs by focusing on Web programming requirements and efficient device resource utilization, pruning the optimization space from millions to only dozens. nnJIT is evaluated for modern models, e.g., BART, T5, and Llama 2, on a range of edge devices including laptops and smartphones using different browsers and hardware from ARM, Intel, AMD and Nvidia. The results show that nnJIT can achieve up to 8.2X faster within 30 seconds compared to the existing baselines.

cs.AI↗

arXiv2024

Visualizing the microscopic origins of topology in twisted molybdenum ditelluride

In moiré materials with flat electronic bands and suitable quantum geometry, strong correlations can give rise to novel topological states of matter. The nontrivial band topology of twisted molybdenum ditelluride (tMoTe$_2$) -- responsible for its fractional quantum anomalous Hall (FQAH) states -- is predicted to arise from a layer-pseudospin skyrmion lattice. Tracing the layer polarization of wavefunctions within the moiré unit cell can thus offer crucial insights into the band topology. Here, we use scanning tunneling microscopy and spectroscopy (STM/S) to probe the layer-pseudospin skyrmion textures of tMoTe$_2$. We do this by simultaneously visualizing the moiré lattice structure and the spatial localization of its electronic states. We find that the wavefunctions associated with the topological flat bands exhibit a spatially-dependent layer polarization within the moiré unit cell. This is in excellent agreement with our theoretical modeling, thereby revealing a direct microscopic connection between the structural properties of tMoTe$_2$ and its band topology. Our work enables new pathways for engineering FQAH states with strain, as well as future STM studies of the intertwined correlated and topological states arising in gate-tunable devices.

cond-mat.mes-hall↗

arXiv2024

Gauge theory of giant phonon magnetic moment in doped Dirac semimetals

We present a quantum theory of phonon magnetic moment in doped Dirac semimetals. Our theory is based on an emergent gauge field approach to the electron-phonon coupling, applicable to both gapless and gapped systems. We find that the magnetic moment is directly proportional to the electrical Hall conductivity through the phonon Hall viscosity. Our theory is combined with the first-principles calculations, allowing us to quantitatively implement it to realistic materials. Magnetic moments are found to be on the order of Bohr magneton for certain phonon modes in graphene and $\text{Cd}_3 \text{As}_2$. Our results provide practical guidance for the dynamical generation of large magnetization in the topological quantum materials.

cond-mat.mtrl-sci↗

arXiv2024

Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference

Large language models (LLMs) based on transformers have made significant strides in recent years, the success of which is driven by scaling up their model size. Despite their high algorithmic performance, the computational and memory requirements of LLMs present unprecedented challenges. To tackle the high compute requirements of LLMs, the Mixture-of-Experts (MoE) architecture was introduced which is able to scale its model size without proportionally scaling up its computational requirements. Unfortunately, MoE's high memory demands and dynamic activation of sparse experts restrict its applicability to real-world problems. Previous solutions that offload MoE's memory-hungry expert parameters to CPU memory fall short because the latency to migrate activated experts from CPU to GPU incurs high performance overhead. Our proposed Pre-gated MoE system effectively tackles the compute and memory challenges of conventional MoE architectures using our algorithm-system co-design. Pre-gated MoE employs our novel pre-gating function which alleviates the dynamic nature of sparse expert activation, allowing our proposed system to address the large memory footprint of MoEs while also achieving high performance. We demonstrate that Pre-gated MoE is able to improve performance, reduce GPU memory consumption, while also maintaining the same level of model quality. These features allow our Pre-gated MoE system to cost-effectively deploy large-scale LLMs using just a single GPU with high performance.

cs.LG↗