Searcharxiv⌕ Search

arXiv subjects

Liangtao Dai

Publications and source records attributed to Liangtao Dai.

7 recordsLinked to original sources

Mutable Inference: Architecting LLM Accelerators for Inference-Time Learning

Long-context LLMs must retain growing histories, but full attention makes each new token increasingly expensive. Inference-time learning (ITL) offers another approach. The model keeps training on the context during inference, retaining what it learns in its weights. Recent work shows that ITL can match full-context attention quality while attending only to a recent fixed-size window. This bounds attention costs but adds Learning (backward computation and weight updates) alongside Prediction (the forward pass) costs. Inference accelerators assume fixed weights, while training accelerators target complete training steps. This work identifies two opportunities. Attention's non-matrix operations leave enough idle matrix capacity for Learning, while selected layers can tolerate delayed weight updates with little quality loss. We exploit both with Learning While Serving (LWS), an inference accelerator architecture that shares compute hardware between latency-critical Prediction and deadline-flexible Learning. Hardware-managed per-layer weight versioning allows Learning to publish updates while in-flight computations read older versions. Resumable reduction blocks let Learning kernels span fragmented idle windows and yield to Prediction, while dedicated softmax-gradient units relieve vector contention. On modeled datacenter and edge NPUs, LWS hides Learning within Prediction, sustaining Prediction-only throughput while preserving 99.3% of fully fresh Learning's quality benefit for TTT-E2E 3B at 128K tokens. Across nine models, LWS achieves 1.6x the throughput of serial execution and 1.3x that of the best prior co-execution architecture. RTL synthesis shows that LWS adds 1.5% to NPU core area without another MAC array. By efficiently turning idle inference capacity into learning capacity, LWS establishes inference-time learning as a first-class workload for future LLM accelerators.

cs.AR↗

SLED-IFV: Solver-Validated LLM-Guided Decomposition for Scalable Hardware Information-Flow Verification

Formal hardware information-flow verification (IFV) provides strong guarantees against secret-dependent timing and control behavior, but often scales poorly on realistic RTL. We identify two recurring proof barriers in self-composed IFV: implementation complexity, where proof-hard datapath logic dominates even though the property needs only a compact boundary relation, and relational inductive complexity, where the proof depends on cross-copy public-control facts that the backend prover does not infer efficiently. To address them, we introduce two semantic proof decomposition forms: functional simplification, which replaces a proof-hard RTL region with a validated over-approximate summary, and relational strengthening, which exposes and proves the cross-copy relations needed for induction. We further present SLED-IFV, a solver-validated LLM-guided flow that automates the selection of these forms and their concrete targets. Given a self-composed miter and an oracle-free decision sheet, the LLM proposes a decomposition, then materializes it into proof artifacts under controller checks. The controller compiles the checked artifacts into proof obligations, and the formal verification backend remains the sole authority for acceptance. Across nine nontrivial benchmarks constructed from real RTL, SLED-IFV achieves up to 603x solver-only speedup and converts two 12-hour timeouts into completed proofs. The closed-loop flow produces verifier-accepted decompositions for all cases.

cs.CR↗

FlexPosit: Tunable Fractional Precision for LLM Inference Accelerators

Large language models (LLMs) offer remarkable capabilities but impose prohibitive compute and energy costs. Quantization governs the trade-offs between accuracy and hardware efficiency across granularity and bit-width. Finer granularity (e.g., group-wise) provides high accuracy but incurs scaling and control overhead, while coarser granularity (e.g., channel-wise) has lower overhead but loses accuracy at low precision. Meanwhile, mixed-precision quantization exposes rich accuracy-efficiency trade-offs algorithmically, but existing LLM accelerators remain limited to discrete precision modes, leaving the fractional design space between them unexplored. FlexPosit bridges these gaps through co-design of Posit-based quantization and a precision-tunable bit-serial architecture. Algorithmically, FlexPosit employs distribution-aware quantization with hardware-aligned, sensitivity-guided mixed-precision allocation, leveraging the Posit format's tapered precision to achieve group-wise-like accuracy with channel-wise-like regularity. Architecturally, FlexPosit is a unified bit-serial systolic array with lightweight per-column decoders, unified Processing Elements (PEs), and a global precision controller, enabling tunable fractional precision while preserving fully regular systolic dataflow. Across diverse LLMs, FlexPosit achieves near-FP16 accuracy with sub-5-bit fractional weights. It achieves 1.8x higher throughput and 1.2x lower energy than BitMoD (group-wise quantization), and 1.5x higher throughput and 2.0x lower energy than OliVe (channel-wise quantization), establishing a new Pareto frontier for precision-tunable LLM acceleration.

cs.AR↗

High-Performance Low-Power Adiabatic Systolic Array Design in Advanced FinFET Nodes

Adiabatic logic has traditionally been recognized as a low-power solution but constrained to low clock speeds to preserve adiabatic behavior. For advanced FinFET nodes, however, clock frequencies have plateaued due to power/thermal concerns (dark silicon) even as the intrinsic device speeds have continued to scale. This convergence opens an opportunity for adiabatic logic to maintain adiabatic behavior even at GHz clocks. We demonstrate an adiabatic logic (AL) design methodology through a MAC systolic array implemented in commercial 16 nm FinFET technology with a resonant 4-phase power clock (PCK) generator, including digital-to-AL and AL-to-digital interfaces. Simulations show that the AL MAC systolic array at 1 GHz achieves power reductions of up to 42% and 36% at the core and system levels, respectively, compared to digital counterparts. Scaling to more advanced nodes should provide even better power/performance metrics.

cs.AR↗

Delayed Coupling Restores Ising Phase Dynamics in Physical Oscillator Networks

Oscillator-based Ising machines, in which the phases of coupled self-sustaining oscillators evolve toward decreasing an Ising Hamiltonian, are commonly interpreted as physical realizations of the Ising model. This interpretation, however, requires the phase dynamics generated by the physical oscillator network to match a prescribed Ising dynamics. Here we show that this correspondence is generally not guaranteed. For arbitrary self-sustaining oscillators under weak coupling, we derive the physical phase interaction from the harmonic overlap between the injected waveform and the perturbation projection vector (also referred to as impulse sensitivity function). We find that uncompensated harmonic phase mismatches between these two quantities generate even components in the physical coupling function, causing a network with the correct coupling topology to implement a non-Ising dynamics. We further show that delayed coupling provides a universal phase-compensation mechanism. For a fixed delay, we derive a condition on the delay under which the even components is minimized in the sense of L2-norm, and oscillator examples confirm that the predicted delay substantially suppresses the even components and brings the realized coupling function closer to the prescribed odd interaction. We then show that a periodically modulated delay can, under suitable moment conditions, eliminate the even components in the phase dynamics. These results establish a general design principle for implementing prescribed energy-based dynamics in physical oscillator networks.

physics.app-ph↗

Guarded Equivalence Predicates for Scalable Formal Hardware Information-Flow Verification

Formal hardware information-flow verification is a principled way to rule out secret-dependent functional or timing observations, but scaling such proofs remains difficult. Self-composition reduces information-flow verification to safety checking over two circuit copies, creating relational proof obligations that are hard for a generic PDR engine to discover from bit-level logic alone. Recent PDR-based techniques exploit this duplicated structure through copy symmetry and global cross-copy equivalence predicates. These predicates are effective when corresponding internal signals agree throughout the reachable state space, but they do not capture equalities that are relevant only in a specific control context. We observe that such contextual relations arise naturally in hardware IFV proofs: an internal signal pair may need to agree only within a control phase, transaction window, loop state, or protocol region. We introduce guarded equivalence predicates to expose these relations to PDR. Rather than treating a proposed contextual equality as an assumption, the verifier submits the corresponding mismatch condition as an auxiliary blocking obligation. Guards are proposed from relational counterexamples-to-induction using CTI-local extraction and state-split search; only candidates proved unreachable by the backend affect the proof. Across 12 IFV benchmarks and two PDR backends, guarded predicates convert two contextual baseline timeouts into completed proofs within 34.2--89.5s under an 1800s limit, while reducing proof time by up to 10.8x on additional benchmarks.

cs.CR↗

A Numerical-based Parametric Error Analysis Method for Goldschmidt Floating Point Division

This paper proposes a parametric error analysis method for Goldschmidt floating point division, which reveals how the errors of the intermediate results accumulate and propagate during the Goldschmidt iterations. The analysis is developed by separating the error terms with and without convergence to zero, which are the key parts of the iterative approximate value. The proposed method leads to a state-of-the-art wordlength reduction for intermediate results during the Goldschmidt iterative computation. It enables at least half of the calculation precision reduction for the iterative factor to implement the rectangular multiplier in the divider through flexible numerical method, which can also be applied in the analysis of Goldschmidt iteration with iterative factors assigned under other ways for faster convergence. Based on the proposed method, two proof-of-concept divider models with different configurations are developed, which are verified by more than 100 billion random test vectors to show the correctness and tightness of the proposed error analysis.

math.NA↗