arXiv subjects

Shimeng Yu

Publications and source records attributed to Shimeng Yu.

At least 19 recordsLinked to original sources

arXiv2026

Hardware Acceleration of Block-Diffusion LLM for Edge Devices

Single-stream (batch-one) edge inference cannot amortize weight traffic across requests. Full-attention diffusion LLMs recompute the entire sequence at every step; native block diffusion makes completed blocks immutable and exactly cacheable, yet refinement still streams prefix KV and FFN weights. We co-design WIFiV-LPDDR, a wide-I/O LPDDR system for precision-tagged reads, BRQ-KV for a canonical low-rank-plus-INT8-residual prefix with query-dependent per-entry precision, and DAT-FFN for drift-mapped canonical replacement, adjacent-stage-corrected low-bit delta, or cached-state carry while keeping live activations unquantized. Both map to an input-stationary mixed-precision systolic array. For the evaluated 1.5B/7B models on modeled Jetson-class platforms, the full stack provides arithmetic-mean energy-reduction factors of 3.79x/3.96x and arithmetic-mean latency speedups of 2.88x/4.44x at the reported DAT-FFN settings; every corresponding compressed model-benchmark score drops by less than one absolute percentage point from its baseline.

cs.AR↗

arXiv2026

Thermal Tuning Overhead in Wafer-Scale Optical Interconnects for LLM MoE Training: A Cross-Layer Analysis and Ferroelectric-Based Mitigation

The rapid scaling of large language models (LLMs), particularly mixture-of-experts (MoE) architectures, has intensified interconnect demands because expert-parallel execution is communication-intensive. Wafer-scale optical interconnects based on dense wavelength-division multiplexing (DWDM) offer a promising path to higher bandwidth; however, conventional microring-resonator (MRR)-based links rely on thermo-optic tuning and are therefore vulnerable to workload-induced thermal fluctuations. In this work, we present a cross-layer analysis of wafer-scale optical interconnects for MoE workloads that combines workload profiling, packet-level network simulation, and transient thermal analysis. We implement a wafer-scale topology in the ht-sim simulator and construct an Ansys thermal model of a 3D-integrated GPU/EIC/PIC stack. Our results show that transient temperature variations can exceed the tracking capability of conventional thermo-optic control loops and thereby introduce repeated tuning stalls during communication phases. The stall durations injected into the network simulation are derived directly from the thermal model rather than assumed. We further evaluate a ferroelectric-based electro-optic tuning mechanism that removes the continuous thermal-tuning requirement. In a four-layer proxy simulation across three MoE models, eliminating the tuning stalls yields speedups of 2.7x for Mixtral 8x7B, 3.8x for Qwen-MoE 14.3B, and 3.3x for LLaMA-MoE 6.7B relative to the thermo-optic case. These results indicate that minimizing photonic tuning latency is important for realizing the performance potential of optical interconnects in large-scale AI systems.

cs.AR↗

arXiv2026

LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving

The energy consumption of Large Language Model (LLM) serving is becoming a major system challenge as deployment scales, driven by hardware power and thermal constraints and rising electricity costs. A key contributor to chip energy dissipation is data movement between limited on-chip cache and off-chip High Bandwidth Memory (HBM). Meanwhile, emerging memory technologies such as monolithic 3D (M3D) integration of cache memories at the Back-End-Of-Line (BEOL) of logic chips enable larger and denser on-chip memories, creating new opportunities to reduce costly off-chip traffic. However, it remains unclear whether continuously scaling on-chip memory using emerging technologies can effectively improve the energy efficiency of LLM serving. To address this gap, we develop LLMET (LLM with Emerging Technology), a validated cross-layer simulation framework, and conduct a comprehensive study on the impact of large-capacity on-chip memory technologies across a broad range of models, applications and platforms. Utilizing M3D technology to expand the L2 cache from 40MB to 1GB yields a 44% reduction in chip energy during the Llama3.1-70B prefill phase with a 16K context window, based on LLMET simulation on a dual NVIDIA A100 GPU setup. On the 8x NVIDIA B200-like platform, extending the L2 cache from 128MB to 4GB saves the prefill energy by up to 24%. For the edge platform and workloads, the decode energy saving reaches 30% when increasing the 8MB cache size to 256MB. These results highlight the promise of ultra-large on-chip memories for energy-efficient LLM serving systems.

cs.AR↗

arXiv2026

Cross-Domain Acceleration of Open Modification Search: From Commodity Platforms to Emerging Memory and Storage Devices

Open modification search (OMS) in mass spectrometry (MS) is a data-intensive workload whose performance is dominantly limited by reference data movement rather than computation. Prior OMS accelerators have largely been evaluated in isolation, making it difficult to understand system-level trade-offs across platforms. This paper presents the first workload-driven, cross-platform survey of accelerators for MS search by studying not only commodity platforms, but also emerging memory- and storage-centric architectures, including GPUs, near-storage FPGAs, DRAM near-memory processing, ReRAM/PCM in-memory processing, and 3D NAND/FeNAND in-storage processing, under consistent algorithmic and accuracy assumptions. Leveraging a binary hyperdimensional computing (HDC)-based OMS formulation that reduces similarity evaluation to lightweight bitwise primitives and tolerates device-level non-idealities, we enable a robust execution on memory-centric architectures despite device-level non-idealities and limited computing capability. Overall, this study identifies memory- and storage-centric architectures as a key architectural breakthrough for large-scale, high-speed search acceleration, delivering up to >100x speedup and >40,000x improvement in energy efficiency.

cs.AR↗

arXiv2026

Probabilistic Memory for Trustworthy Edge Intelligence

Probabilistic computation plays an important role in trustworthy edge intelligence to quantify uncertainty, enhance robustness, reconstruct data, and protect privacy, but its adoption is limited by the orders-of-magnitude data throughput gap between Gaussian random number generation (GRNG) and computation, as well as instruction overhead. This paper introduces probabilistic memory (p-MEM), a unified memory primitive that stores distribution parameters, such as mean and standard deviation, and samples directly at the native memory bandwidth, where deterministic data becomes the zero-variance special case. Using a layout-validated p-MEM simulator, we comprehensively explore device choices, memory specifications, and technology nodes, showing that p-MEM can achieve more than 1000 GSa/s/mm^2 GRNG throughput, including memory-array access. Integrated into CPU/GPU systems, p-MEM reduces instruction count by up to 2.19x/4.37x, sampling latency by 562x/3.45x, and energy by 295.5x/3.53x for Bayesian neural network workloads, providing a scalable hardware substrate for trustworthy probabilistic AI.

cs.AR↗

arXiv2026

Thermal Processing Limits in Oxide-Channel Ferroelectric Field Effect Transistors

In this work, we report a systematic study of the impact of high-temperature post-capping thermal annealing on the memory characteristics of Oxide-semiconductor channel ferroelectric field-effect transistors (OS-FeFETs). Using an identical engineered ferroelectric gate stack 8nm Hf0.5Zr0.5O2 (HZO) / 3 nm Al2O3 / 8 nm HZO (8/3/8) and a hybrid capping layer (3 nm HfO2 + 3 nm Al2O3), 10 percent Ga doped InO (IGO) channel and 4 percent W doped InO (IWO) channel FeFETs remain functional after annealing at temperatures up to 650 C for durations of up to 30 min and 10 min, respectively; further annealing results in irreversible loss of conduction and device failure. Detailed electrical analysis reveals that the MW enhancement originates from a preferential positive shift in the erased-state threshold voltage, while the programmed-state threshold voltage remains comparatively stable. Grazing-incidence X-ray diffraction measurements further indicate structural evolution in the IWO and IGO oxide channels with increasing annealing temperature, supporting the observed electrical trends.

cond-mat.mtrl-sci↗

arXiv2026

System-Technology Co-Optimization of Bitline Routing and Bonding Pathways in Monolithic 3D DRAM Architectures

3D DRAM has emerged as a promising approach for continued density scaling, but its viability is limited by routing and hybrid bonding constraints to periphery, which may degrade sensing margin, latency, and array efficiency. With device characteristics and array parasitics extracted from TCAD, SPICE simulations are performed with peri logic in a CMOS-Bonded-Array (CBA). The analysis shows that the bitline strap architecture with amorphous oxide semiconductor (AOS) selectors is essential to manage routing congestion and parasitics. The optimized design achieves a bit density of 2.6 Gb/mm^2 (137 layers with Si access transistors or 87 layers with AOS), representing ~6x density scaling over D1b 2D DRAM. The design further demonstrates a nominal row cycle time (tRC) of 10.5 ns, compared to 21.3 ns in D1b, and a 60% reduction in read/write energy.

cs.AR↗

arXiv2026

ALD Oxidant as A Tuning Knob for Memory Window Expansion in Ferroelectric FETs for Vertical NAND Applications

Dielectric inserts are widely used to expand the memory window (MW) in ferroelectric FETs (FeFETs) for vertical NAND applications, with prior efforts focused primarily on material selection and stack positioning. Here, we demonstrate that the ALD oxidant used for the Al2O3 interlayer serves as a process-level tuning knob for MW engineering. H2O-grown Al2O3 yields a significantly larger MW (7-8 V) compared to O3 (4 V) for both gate-injection (12/3) and tunnel dielectric (8/3/8) configurations. While the tunnel dielectric (8/3/8) stack maintains robust retention up to 1e4s at 125C despite the larger MW, the gate-injection (12/3) configuration exhibits pronounced retention degradation for the H2O case. The enhanced MW is attributed to higher interlayer leakage associated with H2O-based ALD. These results establish oxidant choice as a key process parameter for co-optimizing MW and retention in ferroelectric NAND technologies.

cond-mat.mtrl-sci↗

arXiv2026

Physics-informed AI Accelerated Retention Analysis of Ferroelectric Vertical NAND: From Day-Scale TCAD to Second-Scale Surrogate Model

Ferroelectric field-effect transistors (FeFET)-based vertical NAND (Fe-VNAND) has emerged as a promising candidate to overcome z-scaling limitations with lower programming voltages. However, the data retention of 3D Fe-VNAND is hindered by the complex interaction between charge detrapping and ferroelectric depolarization. Developing optimized device designs requires exploring an extensive parameter space, but the high computational cost of conventional Technology Computer-Aided Design (TCAD) tools makes such wide-scale optimization impractical. To overcome these simulation barriers, we present a Physics-Informed Neural Operator (PINO)-based AI surrogate model designed for high-efficiency prediction of threshold voltage (Vth) shifts and retention behavior. By embedding fundamental physical principles into the learning architecture, our PINO framework achieves a speedup exceeding 10000x compared to TCAD while maintaining physical accuracy. The resulting surrogate provides a physics-consistent data engine for compact model parameter extraction and look-up-table (LUT) generation, directly supporting reliability-aware SPICE simulation of Fe-VNAND. This study demonstrates the model's effectiveness on a single FeFET configuration, serving as a pathway toward modeling the retention loss mechanisms.

cs.LG↗

arXiv2026

Scalable Digital Compute-in-Memory Ising Machines for Robustness Verification of Binary Neural Networks

Verification of binary neural network (BNN) robustness is NP-hard, as it can be formulated as a combinatorial search for an adversarial perturbation that induces misclassification. Exact verification methods therefore scale poorly with problem dimension, motivating the use of hardware-accelerated heuristics and unconventional computing platforms, such as Ising solvers, that can efficiently explore complex energy landscapes and discover high-quality solutions. In this work, we reformulate BNN robustness verification as a quadratic unconstrained binary optimization (QUBO) problem and solve it using a digital compute-in-memory (DCIM) SRAM-based Ising machine. Instead of requiring globally optimal solutions, we exploit imperfect solutions produced by the DCIM Ising machine to extract adversarial perturbations and thereby demonstrate the non-robustness of the BNN. The proposed architecture stores quantized QUBO coefficients in approximately 9.1~Mb of SRAM and performs annealing in memory via voltage-controlled pseudo-read dynamics, enabling iterative updates with minimal data movement. Experimental projections indicate that the proposed approach achieves a $178\times$ acceleration in convergence rate and a $1538\times$ improvement in power efficiency relative to conventional CPU-based implementations.

cs.ET↗

arXiv2026

ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization

Compute-in-Memory (CIM) architectures have been widely studied for deep neural network (DNN) acceleration by reducing data transfer overhead between the memory and computing units. In conventional CIM design flows, system-level CIM simulators (such as NeuroSim) are leveraged for design space exploration (DSE) across different hardware configurations and DNN workloads. However, CIM designers need to invest substantial effort in interpreting simulator manuals and understanding complex parameter dependencies. Moreover, extensive design-simulation iterations are often required to identify optimal CIM configurations under hardware constraints. These challenges severely prolong the DSE cycle and hinder rapid CIM deployment. To address these challenges, this work proposes ChatNeuroSim, a large language model (LLM)-based agent framework for automated CIM accelerator deployment and optimization. ChatNeuroSim automates the entire CIM workflow, including task scheduling, request parsing and adjustment, parameter dependency checking, script generation, and simulation execution. It also integrates the proposed CIM optimizer using design space pruning, enabling rapid identification of optimal configurations for different DNN workloads. ChatNeuroSim is evaluated on extensive request-level testbenches and demonstrates correct simulation and optimization behavior, validating its effectiveness in automatic request parsing and task execution. Furthermore, the proposed design space pruning technique accelerates CIM optimization process compared to no-pruning baseline. In the case study optimizing Swin Transformer Tiny under 22 nm technology, the proposed CIM optimizer achieves a 0.42$\times$-0.79$\times$ average runtime reduction compared to the same optimization algorithm without design space pruning.

cs.AR↗

arXiv2026

HAVEN: High-Bandwidth Flash Augmented Vector Engine for Large-Scale Approximate Nearest-Neighbor Search Acceleration

Retrieval-Augmented Generation (RAG) relies on large-scale Approximate Nearest Neighbor Search (ANNS) to retrieve semantically relevant context for large language models. Among ANNS methods, IVF-PQ offers an attractive balance between memory efficiency and search accuracy. However, achieving high recall requires reranking which fetches full-precision vectors for reranking, and the billion-scale vector databases need to reside in CPU DRAM or SSD due to the limited capacity of GPU HBM. This off-GPU data movement introduces substantial latency and throughput degradation. We propose HAVEN, a GPU architecture augmented with High-Bandwidth Flash (HBF) which is a recently introduced die-stacked 3D NAND technology engineered to deliver terabyte-scale capacity and hundreds of GB/s read bandwidth. By integrating HBF and near-storage search unit as an on-package complement to HBM, HAVEN enables the full-precision vector database to reside entirely on-device, eliminating PCIe and DDR bottlenecks during reranking. Through detailed modeling of re-architected 3D NAND subarrays, power-constrained HBF bandwidth, and end-to-end IVF-PQ pipelines, we demonstrate that HAVEN improves reranking throughput by up to 20x and latency up to 40x across billion-scale datasets compared to GPU-DRAM and GPU-SSD systems. Our results show that HBF-augmented GPUs enable high-recall retrieval at throughput previously achievable only without reranking, offering a promising direction for memory-centric AI accelerators.

cs.AR↗

arXiv2026

The First Switch Effect in Ferroelectric Field-Effect Transistors

In this work, a ferroelectric field-effect transistor (FEFET) is systematically characterized and compared with an equivalent standard MOSFET with an equivalent oxide thickness. We show that these two devices, with a silicon channel, exhibit similar pristine state transfer characteristics but starkly different endurance characteristics. In contrast to the MOSFET, the FEFET shows a significant increase in sub-threshold swing in the first write pulse. Based on this, we reveal that this first write pulse (cycle 1) generates more than half of the total traps generated during the fatigue cycling in FEFETs. We call this the 'First Switch Effect'. Further, by polarizing a pristine FEFET step by step, we demonstrate a direct correlation between the switched polarization and interface trap density during the first switch. Through charge pumping measurements, we also observe that continued cycling generates traps more towards the bulk of the stack, away from the Si/SiO2 interface in FEFETs. We establish that: (1) the first switch effect leads to approximately 50% of the total trap density (Nit) near the Si/SiO2 interface until memory window closure; and (2) further bipolar cycling leads to trap generation both at and away from Si/SiO2 interface in FEFETs.

cond-mat.mtrl-sci↗

arXiv2025

Pulse-Mode Operation and Reliability of BEOL-Compatible Ferroelectric Non-Volatile Capacitive Memories with Amorphous Oxide Semiconductor Channels

Non-volatile capacitive memories (nvCAPs) exhibiting AC small-signal capacitance on/off ratio (Con/Coff) with non-destructive read have emerged as a promising device for next-generation memory paradigms. Recently, BEOL-compatible ferroelectric nvCAPs with an amorphous oxide semiconductor channel have been reported, suggesting the possibility of monolithic 3D integration of nvCAPs on top of CMOS. So far, the characterization studies on oxide-channel ferroelectric nvCAPs have been done using dual DC sweep C-V measurements which are typically performed over a time scale of a few seconds. However, non-volatile memory arrays typically require nvCAPs to operate under pulse-mode. It is thus crucial to advance understanding of the behavior of oxide-channel ferroelectric nvCAPs under pulse-mode operation, governed by the unique interplay between ferroelectric layer and oxide channel physics. In this study, we provide a systematic study of the pulse-mode operation of ferroelectric nvCAPs with an amorphous oxide semiconductor channel, including its pulse-based write characteristics and reliability characteristics. We examine overlap area, wake-up and pulse-width dependent Con and Coff writing characteristics under pulse-mode. Further, we suggest the importance of optimizing ferroelectric depolarization for Con retention, while reducing read-after-delay for Coff retention under pulse-mode. Lastly, non-destructive read operation for >10^9 read stress cycles at |Vread|=1V is demonstrated.

cond-mat.mtrl-sci↗

arXiv2025

Row Hammer Effect and Floating Body Effect of Monolithic 3D Stackable 1T1C DRAM

Monolithic 3D stackable 1T1C DRAM technology is on the rise, with initial prototypes reported by the industry. This work presents a comprehensive reliability study focusing on the intricate interplay between the row hammer effect and the floating body effect. First, using a TCAD model of a 3D DRAM mini-array, we categorize different cases of adjacent cells and show that the notorious row hammer effect induced by charge migration is significantly mitigated compared to 2D DRAM. However, we found that when incorporating an impact ionization model to account for the floating body characteristics of the silicon access transistor, the capacitive coupling between vertically stacked cells is severely exacerbated. Second, we conduct an in-depth investigation into the floating body effect itself. We systematically examine the dependence of this effect on key device parameters, including body thickness, doping concentration, and gate work function.

physics.app-ph↗

arXiv2025

Cryogenic Characterization of Ferroelectric Non-volatile Capacitors

Ferroelectric-based capacitive crossbar arrays have been proposed for energy-efficient in-memory computing in the charge domain. They combat the challenges like sneak paths and high static power faced by resistive crossbar arrays but are susceptible to thermal noise limiting the effective number of bits (ENOB) for the weighted sum. A direct way to reduce this thermal noise is by lowering the temperature as thermal noise is proportional to temperature. In this work, we first characterize the non-volatile capacitors (nvCaps) on a foundry 28 nm platform at cryogenic temperatures to evaluate the memory window, ON state retention as a function of temperature down to 77K, and then use the calibrated device models to simulate the capacitive crossbar arrays in SPICE at lower temperatures to demonstrate higher ENOB (~5 bits) for 128x128 multiple-and-accumulate (MAC) operations.

cs.ET↗

arXiv2025

FeNOMS: Enhancing Open Modification Spectral Library Search with In-Storage Processing on Ferroelectric NAND (FeNAND) Flash

The rapid expansion of mass spectrometry (MS) data, now exceeding hundreds of terabytes, poses significant challenges for efficient, large-scale library search - a critical component for drug discovery. Traditional processors struggle to handle this data volume efficiently, making in-storage computing (ISP) a promising alternative. This work introduces an ISP architecture leveraging a 3D Ferroelectric NAND (FeNAND) structure, providing significantly higher density, faster speeds, and lower voltage requirements compared to traditional NAND flash. Despite its superior density, the NAND structure has not been widely utilized in ISP applications due to limited throughput associated with row-by-row reads from serially connected cells. To overcome these limitations, we integrate hyperdimensional computing (HDC), a brain-inspired paradigm that enables highly parallel processing with simple operations and strong error tolerance. By combining HDC with the proposed dual-bound approximate matching (D-BAM) distance metric, tailored to the FeNAND structure, we parallelize vector computations to enable efficient MS spectral library search, achieving 43x speedup and 21x higher energy efficiency over state-of-the-art 3D NAND methods, while maintaining comparable accuracy.

cs.AR↗

arXiv2025

Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving

As Large Language Models (LLMs) continue to evolve, Mixture of Experts (MoE) architecture has emerged as a prevailing design for achieving state-of-the-art performance across a wide range of tasks. MoE models use sparse gating to activate only a handful of expert sub-networks per input, achieving billion-parameter capacity with inference costs akin to much smaller models. However, such models often pose challenges for hardware deployment due to the massive data volume introduced by the MoE layers. To address the challenges of serving MoE models, we propose Stratum, a system-hardware co-design approach that combines the novel memory technology Monolithic 3D-Stackable DRAM (Mono3D DRAM), near-memory processing (NMP), and GPU acceleration. The logic and Mono3D DRAM dies are connected through hybrid bonding, whereas the Mono3D DRAM stack and GPU are interconnected via silicon interposer. Mono3D DRAM offers higher internal bandwidth than HBM thanks to the dense vertical interconnect pitch enabled by its monolithic structure, which supports implementations of higher-performance near-memory processing. Furthermore, we tackle the latency differences introduced by aggressive vertical scaling of Mono3D DRAM along the z-dimension by constructing internal memory tiers and assigning data across layers based on access likelihood, guided by topic-based expert usage prediction to boost NMP throughput. The Stratum system achieves up to 8.29x improvement in decoding throughput and 7.66x better energy efficiency across various benchmarks compared to GPU baselines.

cs.AR↗