arXiv subjects

Simeng Zhang

Publications and source records attributed to Simeng Zhang.

5 recordsLinked to original sources

arXiv2026

Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

Large language models often rely on Chain-of-Thought (CoT) reasoning to solve complex tasks, but verbose reasoning traces introduce substantial inference overhead. CoT compression shortens generation, yet aggressive compression may disrupt logical coherence and degrade performance. We formalize this trade-off as the Context-Generation Substitution Law, where explicit reasoning context substitutes for part of decode-time generation. Based on this principle, we propose Memory-Augmented Compression, a training-free framework that constructs reusable reasoning memories from historical traces and retrieves them as prefill-side scaffolds. Rather than using raw demonstrations, these memories summarize reusable reasoning patterns, key constraints, and critical operations to compensate for information lost during compression. Experiments show that Memory consistently improves prompt-based Chain-of-Draft (CoD) compression across mathematical reasoning, complex reasoning, and science question answering tasks, yielding accuracy gains of 21.4, 28.0, 29.5, and 6.61 points over CoD on GSM8K, MATH, BBH, and MMLU-Sci, while achieving a 1.14-1.49x latency speedup latency speedup over standard CoT. Memory is also compatible with token-level, reasoning-trace-level, and inference-state compression mechanisms.

cs.CL↗

arXiv2026

ConSA: Controllable Sparsity in Hybrid Attention via Learnable Allocation

Hybrid architectures combining full attention (FA) and sliding-window attention (SWA) are a promising paradigm for efficient LLM inference. However, existing methods typically rely on hand-crafted rules or simple post-hoc heuristics for FA/SWA allocation and offer limited analysis of the attention behaviors underlying these designs. We propose Controllable Sparsity in Hybrid Attention (ConSA), a framework that learns optimal FA/SWA assignment under a user-specified sparsity target. ConSA employs L0 regularization to learn binary masks selecting between FA and SWA for each attention unit, while an augmented Lagrangian constraint enforces the target sparsity at either layer or KV-head granularity. We evaluate ConSA on two LLMs at the 0.6B and 1.7B scales. Learned allocations consistently outperform rule-based baselines, with KV-head-wise allocation yielding clear gains over layer-wise allocation. The learned patterns place SWA in the bottom layers and concentrate FA into contiguous middle-layer blocks, diverging from evenly interleaved patterns in rule-based methods. This structure persists across model scales, sparsity levels, and allocation granularities, revealing a fine-grained spectrum of intrinsic attention behaviors that underlies the learned allocation.

cs.CL↗

arXiv2026

Biological Sequence Clustering: A Survey

The rapid development of high-throughput sequencing technologies has led to an explosive increase in biological sequence data, making sequence clustering a fundamental task in large-scale bioinformatics analyses. Unlike traditional clustering problems, biological sequence clustering faces unique challenges due to the lack of direct similarity measures, strict biological constraints, and demanding requirements for both scalability and accuracy. Over the past decades, a wide variety of methods have been developed, differing in how they model sequence similarity, construct clusters, and prioritize optimization objectives. In this review, we provide a comprehensive methodological overview of biological sequence clustering algorithms. We begin by summarizing the main strategies for modeling sequence similarity, which can be divided into three stages: sequence encoding, feature generation, and similarity measurement. Next, we discuss the major clustering paradigms, including greedy incremental, hierarchical, graph-based, model-based, partitional, and deep learning approaches, highlighting their methodological characteristics and practical trade-offs. We then discuss clustering objectives from three key perspectives: scalability and resource efficiency, biological interpretability, and robustness and clustering quality. Organizing existing methods along these dimensions allows us to explore the trade-offs in biological sequence clustering and clarify the contexts in which different approaches are most appropriate. Finally, we identify current limitations and challenges, providing guidance for researchers and directions for future method development.

q-bio.GN↗

arXiv2025

Two-cluster test

Cluster analysis is a fundamental research issue in statistics and machine learning. In many modern clustering methods, we need to determine whether two subsets of samples come from the same cluster. Since these subsets are usually generated by certain clustering procedures, the deployment of classic two-sample tests in this context would yield extremely smaller p-values, leading to inflated Type-I error rate. To overcome this bias, we formally introduce the two-cluster test issue and argue that it is a totally different significance testing issue from conventional two-sample test. Meanwhile, we present a new method based on the boundary points between two subsets to derive an analytical p-value for the purpose of significance quantification. Experiments on both synthetic and real data sets show that the proposed test is able to significantly reduce the Type-I error rate, in comparison with several classic two-sample testing methods. More importantly, the practical usage of such two-cluster test is further verified through its applications in tree-based interpretable clustering and significance-based hierarchical clustering.

cs.LG↗

arXiv2019

High-Performance Logic and Memory Devices Based on a Dual-Gated MoS2 Architecture

In this work, we demonstrate a dual-gated (DG) MoS2 field effect transistors (FETs) in which the degraded switching performance of multilayer MoS2 can be compensated by the DG structure. It produces large current density (>100 {\mu}A/{\mu}m for a monolayer), steep subthreshold swing (SS) (~100 mV/dec for 5 nm thickness), and high on/off current ratio (greater than 107 for 10 nm thickness). Such DG structure not only improves electrostatic control but also provides an extra degree of freedom for manipulating the threshold voltage (VTH) and SS by separately tuning the top and back gate voltages, which are demonstrated in a logic inverter. Dynamic random access memory (DRAM) has a short retention time because of large OFF-state current in the Si MOSFET. Based on our DG MoS2-FETs, and a DRAM unit cell with a long retention time of 1260 ms are realized. A large-scale isolated MoS2 DG-FETs based on CVD-synthesized continuous films is also demonstrated, which shows potential applications for future wafer-scale digital and low-power electronics.

physics.app-ph↗