Searcharxiv⌕ Search

arXiv subjects

Junchen Liu

Publications and source records attributed to Junchen Liu.

16 recordsLinked to original sources
arXiv2026

Test-Time Training with KV Binding Is Secretly Linear Attention

Test-time training (TTT) with KV binding as sequence modeling layer is commonly interpreted as a form of online meta-learning that memorizes a key-value mapping at test time. However, our analysis reveals multiple phenomena that contradict this memorization-based interpretation. Motivated by these findings, we revisit the formulation of TTT and show that a broad class of TTT architectures can be expressed as a form of learned linear attention operator. Beyond explaining previously puzzling model behaviors, this perspective yields multiple practical benefits: it enables principled architectural simplifications, admits fully parallel formulations that preserve performance while improving efficiency, and provides a systematic reduction of diverse TTT variants to a standard linear attention form. Overall, our results reframe TTT not as test-time memorization, but as learned linear attention with enhanced representational capacity. Project page: https://research.nvidia.com/labs/sil/projects/tttla/.

cs.LG↗
arXiv2026

Beyond Conditional Computation: Retrieval-Augmented Genomic Foundation Models with Gengram

Current genomic foundation models (GFMs) rely on extensive neural computation to implicitly approximate conserved biological motifs from single-nucleotide inputs. We propose Gengram, a conditional memory module that introduces an explicit and highly efficient lookup primitive for multi-base motifs via a genomic-specific hashing scheme, establishing genomic "syntax". Integrated into the backbone of state-of-the-art GFMs, Gengram achieves substantial gains (up to 14%) across several functional genomics tasks. The module demonstrates robust architectural generalization, while further inspection of Gengram's latent space reveals the emergence of meaningful representations that align closely with fundamental biological knowledge. By establishing structured motif memory as a modeling primitive, Gengram simultaneously boosts empirical performance and mechanistic interpretability, providing a scalable and biology-aligned pathway for the next generation of GFMs. The code is available at https://github.com/zhejianglab/Genos, and the model checkpoint is available at https://huggingface.co/ZhejiangLab/Gengram.

q-bio.GN↗
arXiv2025

A Comprehensive Survey of 3GPP Release 19 ISAC Channel Modeling: From Empirical Features to Unified Methodology and Standardized Simulator

Integrated Sensing and Communication (ISAC) has been identified as a key 6G application by ITU and 3GPP. Channel measurement and modeling is a prerequisite for ISAC system design and has attracted widespread attention from both academia and industry. 3GPP Release 19 initiated the ISAC channel study item in December 2023 and finalized its modeling specification in May 2025 after extensive technical discussions. However, a comprehensive survey that provides a systematic overview,from empirical channel features to modeling methodologies and standardized simulators,remains unavailable. In this paper, the key requirements and challenges in ISAC channel research are first analyzed, followed by a structured overview of the standardization workflow throughout the 3GPP Release 19 process. Then, critical aspects of ISAC channels, including physical objects, target channels, and background channels, are examined in depth, together with additional features such as spatial consistency, environment objects, Doppler characteristics, and shared clusters, supported by measurement-based analysis. To establish a unified ISAC channel modeling framework, an Extended Geometry-based Stochastic Model (E-GBSM) is proposed, incorporating all the aforementioned ISAC channel characteristics. Finally, a standardized simulator is developed based on E-GBSM, and a two-phase calibration procedure aligned with 3GPP Release 19 is conducted to validate both the model and the simulator, demonstrating close agreement with industrial reference results. Overall, this paper provides a systematic survey of 3GPP Release 19 ISAC channel standardization and offers insights into best practices for new feature characterization, unified modeling methodology, and standardized simulator implementation, which can effectively supporting ISAC technology evaluation and future 6G standardization.

eess.SP↗
arXiv2025

Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff

As a widely adopted model compression technique, model pruning has demonstrated strong effectiveness across various architectures. However, we observe that when sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance. This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms, rendering them inoperable. To overcome this limitation, we propose a bidirectional pruning-regrowth strategy. Starting from an extremely compressed network that satisfies hardware constraints, the method selectively regenerates critical connections to recover lost performance, effectively mitigating the sharp accuracy drop commonly observed under high sparsity conditions.

cs.LG↗
arXiv2025

A Novel Multi-Reference-Point Modeling Framework for Monostatic Background Channel: Toward 3GPP ISAC Standardization

Integrated Sensing and Communication (ISAC) has been identified as a key 6G application by ITU and 3GPP. A realistic, standard-compatible channel model is essential for ISAC system design. To characterize the impact of Sensing Targets (STs), 3GPP defines ISAC channel as a combination of target and background channels, comprising multipath components related to STs and those originating solely from the environment, respectively. Although the background channel does not carry direct ST information, its accurate modeling is critical for evaluating sensing performance, especially in complex environments. Existing communication standards characterize propagation between separated transmitter (Tx) and receiver (Rx). However, modeling background channels in the ISAC monostatic mode, where the Tx and Rx are co-located, remains a pressing challenge. In this paper, we firstly conduct ISAC monostatic background channel measurements for an indoor scenario at 28 GHz. Realistic channel parameters are extracted, revealing pronounced single-hop propagation and discrete multipath distribution. Inspired by these properties, a novel stochastic model is proposed to characterizing the ISAC monostatic background channel as the superposition of sub-channels between the monostatic Tx&Rx and multiple communication Rx-like Reference Points (RPs). This model is compatible with standardizations, and a 3GPP-extended implementation framework is introduced. Finally, a genetic algorithm-based method is proposed to extract the optimal number and placement of multi-RPs. The optimization approach and modeling framework are validated by comparing measured and simulated channel parameters. Results demonstrate that the proposed model effectively captures monostatic background channel characteristics, addresses a critical gap in ISAC channel modeling, and supports 6G standardization.

eess.SP↗
arXiv2025

SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment

While frontier large language models (LLMs) continue to push capability boundaries, their deployment remains confined to GPU-powered cloud infrastructure. We challenge this paradigm with SmallThinker, a family of LLMs natively designed - not adapted - for the unique constraints of local devices: weak computational power, limited memory, and slow storage. Unlike traditional approaches that mainly compress existing models built for clouds, we architect SmallThinker from the ground up to thrive within these limitations. Our innovation lies in a deployment-aware architecture that transforms constraints into design principles. First, We introduce a two-level sparse structure combining fine-grained Mixture-of-Experts (MoE) with sparse feed-forward networks, drastically reducing computational demands without sacrificing model capacity. Second, to conquer the I/O bottleneck of slow storage, we design a pre-attention router that enables our co-designed inference engine to prefetch expert parameters from storage while computing attention, effectively hiding storage latency that would otherwise cripple on-device inference. Third, for memory efficiency, we utilize NoPE-RoPE hybrid sparse attention mechanism to slash KV cache requirements. We release SmallThinker-4B-A0.6B and SmallThinker-21B-A3B, which achieve state-of-the-art performance scores and even outperform larger LLMs. Remarkably, our co-designed system mostly eliminates the need for expensive GPU hardware: with Q4_0 quantization, both models exceed 20 tokens/s on ordinary consumer CPUs, while consuming only 1GB and 8GB of memory respectively. SmallThinker is publicly available at hf.co/PowerInfer/SmallThinker-4BA0.6B-Instruct and hf.co/PowerInfer/SmallThinker-21BA3B-Instruct.

cs.LG↗
arXiv2025

Cameras as Relative Positional Encoding

Transformers are increasingly prevalent for multi-view computer vision tasks, where geometric relationships between viewpoints are critical for 3D perception. To leverage these relationships, multi-view transformers must use camera geometry to ground visual tokens in 3D space. In this work, we compare techniques for conditioning transformers on cameras: token-level raymap encodings, attention-level relative pose encodings, and a new relative encoding we propose -- Projective Positional Encoding (PRoPE) -- that captures complete camera frustums, both intrinsics and extrinsics, as a relative positional encoding. Our experiments begin by showing how relative camera conditioning improves performance in feedforward novel view synthesis, with further gains from PRoPE. This holds across settings: scenes with both shared and varying intrinsics, when combining token- and attention-level conditioning, and for generalization to inputs with out-of-distribution sequence lengths and camera intrinsics. We then verify that these benefits persist for different tasks, stereo depth estimation and discriminative spatial cognition, as well as larger model sizes.

cs.CV↗
arXiv2025

ISAC Channel Modelling -- Perspectives from ETSI

Integrated Sensing and Communications (ISAC) is defined as one of six usage scenarios in the ITU-R International Mobile Telecommunications (IMT) 2030 framework for 6G. ISAC is envisioned to introduce the sensing capability into the cellular network, where sensing may be obtained using the cellular radio frequency (RF) signals with or without additional auxiliary sensors. To enable ISAC, specification bodies such as European Telecommunications Standards Institute (ETSI) and Third Generation Partnership Project (3GPP) have already started to look into detailed ISAC use cases, their requirements, and the channel models and evaluation methodologies that are necessary to design and evaluate ISAC performance. With focus on the channel model, the current communication-centric channel models like those specified in 3GPP technical report (TR) 38.901 do not cover the RF signals interactions between the transmitter, target object, receiver and their surrounding environment. To bridge this gap, 3GPP has been looking into the basic changes that are necessary to make to their TR38.901 channel model with focus on selected use cases from the 3GPP SA1 5G-Advanced feasibility study. In parallel, ETSI ISAC Industry Specification Group (ISG) has been studying the more advanced ISAC channel modelling features that are needed to support the variety of ISAC use cases envisioned in 6G. In this paper, we present the baseline and advanced features developed thus far in 3GPP and ETSI ISAC ISG, respectively, towards a comprehensive view of the ISAC channel model in 6G.

eess.SP↗
arXiv2025

Local-in-time well-posedness for 2D compressible magneto-micropolar boundary layer in Sobolev spaces

In this paper, we study the two-dimensional compressible magneto-micropolar boundary layer equations on the half-plane, which are derived from 2D compressible magneto-micropolar fluid equations with the non-slip boundary condition on velocity, Dirichlet boundary condition on micro-rotational velocity and perfectly conducting boundary condition on magnetic field. Based on a nonlinear coordinate transformation proposed in \cite{LXY2019}, we first prove the local-in-time well-posedness for the compressible magneto-micropolar boundary layer system in Sobolev spaces, provided that initial tangential magnetic field is non-degenerate.

math.AP↗
arXiv2025

Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond

This paper introduces Light-R1, an open-source suite for training long reasoning models using reproducible and cost-effective methodology. Given the proprietary nature of data used in the DeepSeek-R1 series, we develop an alternative approach leveraging exclusively public data and models. Our curriculum training progressively increases data difficulty, combined with multi-staged post-training. Our Light-R1-32B model, trained from Qwen2.5-32B-Instruct, outperforms DeepSeek-R1-Distill-Qwen-32B in math reasoning. Experimental results show that this curriculum approach becomes more effective when distinct, diverse datasets are available for different training stages: fine-tuning DeepSeek-R1-Distilled models (pre-tuned by DeepSeek team on proprietary data) with 3,000 challenging examples from our curriculum dataset yielded state-of-the-art 7B and 14B models, while the 32B model, Light-R1-32B-DS performed comparably to QwQ-32B and DeepSeek-R1. Furthermore, we extend our work by applying GRPO on long reasoning models. Our final Light-R1-14B-DS achieves SOTA performance among 14B models in math, with AIME24 & 25 scores of 74.0 and 60.2 respectively, surpassing many 32B models and DeepSeek-R1-Distill-Llama-70B. Despite math-focused training, Light-R1-14B-DS demonstrates strong cross-domain generalization. Light-R1 represents a significant advancement in making sophisticated reasoning models more accessible and implementable in real-world applications. Our models, training data and code have been made available at https://github.com/Qihoo360/Light-R1.

cs.CL↗
arXiv2024

Long time well-posedness for the 3D Prandtl boundary layer equations with a special structure

This paper is concerned with existence, uniqueness and stability of the solution for the 3D Prandtl equation in a polynomial weighted Sobolev space. The main novelty of this paper is to directly prove the long time well-posedness to 3D Prandtl equation under monotonicity condition $\partial_{z} u >0$ and a special structural assumption $v=Ku$ $\big(\partial_{z}\big(\frac{v}{u}\big) \equiv 0\big)$ by the energy method. Moreover, the solution's lifespan can be extended to any large $T$, provided that the initial data with a perturbation lie in the monotonic shear profile of small size $e^{-T}$. This result extends the local well-posedness results established by Liu-Wang-Yang \cite{Liu-Wang-Yang-1-2017} (Adv. Math. 308 (2017) 1074-1126) and Qin-Wang \cite{Qin-Wang-2024} (J. Math. Pure. Appl. 194 (2025) 103670) for the 3D Prandtl equations to long-time well-posedness.

math.AP↗
arXiv2024

Explaining Spectrograms in Machine Learning: A Study on Neural Networks for Speech Classification

This study investigates discriminative patterns learned by neural networks for accurate speech classification, with a specific focus on vowel classification tasks. By examining the activations and features of neural networks for vowel classification, we gain insights into what the networks "see" in spectrograms. Through the use of class activation mapping, we identify the frequencies that contribute to vowel classification and compare these findings with linguistic knowledge. Experiments on a American English dataset of vowels showcases the explainability of neural networks and provides valuable insights into the causes of misclassifications and their characteristics when differentiating them from unvoiced speech. This study not only enhances our understanding of the underlying acoustic cues in vowel classification but also offers opportunities for improving speech recognition by bridging the gap between abstract representations in neural networks and established linguistic knowledge

eess.AS↗
arXiv2024

Composition Vision-Language Understanding via Segment and Depth Anything Model

We introduce a pioneering unified library that leverages depth anything, segment anything models to augment neural comprehension in language-vision model zero-shot understanding. This library synergizes the capabilities of the Depth Anything Model (DAM), Segment Anything Model (SAM), and GPT-4V, enhancing multimodal tasks such as vision-question-answering (VQA) and composition reasoning. Through the fusion of segmentation and depth analysis at the symbolic instance level, our library provides nuanced inputs for language models, significantly advancing image interpretation. Validated across a spectrum of in-the-wild real-world images, our findings showcase progress in vision-language models through neural-symbolic integration. This novel approach melds visual and language analysis in an unprecedented manner. Overall, our library opens new directions for future research aimed at decoding the complexities of the real world through advanced multimodal technologies and our code is available at \url{https://github.com/AnthonyHuo/SAM-DAM-for-Compositional-Reasoning}.

cs.CV↗
arXiv2024

Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solids

Despite significant advancements in Neural Radiance Fields (NeRFs), the renderings may still suffer from aliasing and blurring artifacts, since it remains a fundamental challenge to effectively and efficiently characterize anisotropic areas induced by the cone-casting procedure. This paper introduces a Ripmap-Encoded Platonic Solid representation to precisely and efficiently featurize 3D anisotropic areas, achieving high-fidelity anti-aliasing renderings. Central to our approach are two key components: Platonic Solid Projection and Ripmap encoding. The Platonic Solid Projection factorizes the 3D space onto the unparalleled faces of a certain Platonic solid, such that the anisotropic 3D areas can be projected onto planes with distinguishable characterization. Meanwhile, each face of the Platonic solid is encoded by the Ripmap encoding, which is constructed by anisotropically pre-filtering a learnable feature grid, to enable featurzing the projected anisotropic areas both precisely and efficiently by the anisotropic area-sampling. Extensive experiments on both well-established synthetic datasets and a newly captured real-world dataset demonstrate that our Rip-NeRF attains state-of-the-art rendering quality, particularly excelling in the fine details of repetitive structures and textures, while maintaining relatively swift training times.

cs.CV↗
arXiv2023

NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields

Neural radiance fields (NeRF) have revolutionized the field of image-based view synthesis. However, NeRF uses straight rays and fails to deal with complicated light path changes caused by refraction and reflection. This prevents NeRF from successfully synthesizing transparent or specular objects, which are ubiquitous in real-world robotics and A/VR applications. In this paper, we introduce the refractive-reflective field. Taking the object silhouette as input, we first utilize marching tetrahedra with a progressive encoding to reconstruct the geometry of non-Lambertian objects and then model refraction and reflection effects of the object in a unified framework using Fresnel terms. Meanwhile, to achieve efficient and effective anti-aliasing, we propose a virtual cone supersampling technique. We benchmark our method on different shapes, backgrounds and Fresnel terms on both real-world and synthetic datasets. We also qualitatively and quantitatively benchmark the rendering results of various editing applications, including material editing, object replacement/insertion, and environment illumination estimation. Codes and data are publicly available at https://github.com/dawning77/NeRRF.

cs.CV↗