Searcharxiv⌕ Search

arXiv subjects

Ling Liu

Publications and source records attributed to Ling Liu.

At least 19 recordsLinked to original sources

Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function

We expose a safety vulnerability on nascent agentic reinforcement-learning-as-a-service. Specifically, with a cost as low as \$20, we show that safety alignment of a large language model (e.g., Inkling) can be completely subverted through rounds of harmful agentic reinforcement learning (agentic RL) with a toxic environment from its users. We also demonstrate that upon a successful harmful agentic RL attack via the service API, the attackers can potentially use the safety-degraded model to assist with real-world harmful activities, e.g., exploit software vulnerabilities and launch code re-use attacks. To mitigate harmful agentic RL, we propose Bait, a defense that mitigates the attack via proactively probing the reward function. Experimental results show that such a simple defense can sufficiently mitigate learning from a harmful reward function and can still learn from a benign reward function that promotes benign agentic tasks. Further analysis on both empirical and theoretical aspects shows that an adaptive attacker who assumes the knowledge of Bait defense cannot perfectly subvert it due to a systematic challenge. Source code is available at https://github.com/git-disl/Bait.

cs.LG↗

Alliance of Stochastic Routing and Reinforcement Fusion for Multi-Model Reasoning

This paper studies dynamic model routing under uncertainty in multimodal settings, where a system must select and potentially compose multiple models to answer a query. Standard reinforcement learning approaches for routing often suffer from premature convergence to suboptimal models due to noisy rewards and insufficient exploration. We propose a unified vision-language framework (VL-Scout) for efficient multimodal and multi-step reasoning with turn-based reinforcement fusion. First, VL-Scout combines Bayesian priors, policy gradient optimization, and sequential decision-making to enable structured exploration and adaptive inference. A context-dependent Dirichlet belief model allows for epistemic uncertainty over model quality and is updated online based on observed rewards and context-specific evidence, enabling Thompson-style exploration while serving as a prior for regularization. Second, a routing policy is trained via a KL-regularized REINFORCE objective by maximizing a reward-weighted evidence lower bound, where the policy acts as an amortized posterior, and the Dirichlet belief serves as a prior. Third, we introduce a fusion policy trained with Group Relative Policy Optimization (GRPO) or Reinforcement Learning with Verifiable Rewards (RLVR), which operates over intermediate model rationales and selects between two actions: CONTINUE or ANSWER, terminating with a generated output. We evaluate VL-Scout on four representative multi-modal benchmarks (MMMU, MMMU-Pro, OKVQA, and RealWorldQA), compared against 11 representative routing and policy optimization baselines. Our method provides consistent performance improvements while reducing premature policy collapse. These results highlight the importance of integrating uncertainty-aware priors with both amortized policy learning and sequential fusion for robust and efficient model selection in discrete action spaces

cs.LG↗

Billion-Scale Thumbnail Optimization for Uncurated Short-Form Videos via Multi-Armed Bandits

This paper introduces a real-time thumbnail optimization system deployed at a global $O(B)$ scale on a major short-form video platform. Unlike traditional long-form content, where custom thumbnails are heavily curated by creators, a considerable fraction of short-form videos are published without human-selected artwork. To address this uncurated corpus, we present a fully automated, end-to-end framework that replaces static default frames with dynamic, data-driven selections across billions of videos. To the best of our knowledge, this is the first published work demonstrating an online Multi-Armed Bandit framework successfully deployed at an $O(B)$ scale for uncurated short-form video discovery. Our solution pairs a multi-stage candidate generation pipeline with a low-latency serving infrastructure. By initializing the exploration framework with image-specific priors derived from a deep visual quality model, the system minimizes exploration costs and dynamically serves optimal thumbnails at serving time. Global deployment demonstrates statistically significant improvements in core user discovery and engagement metrics.

cs.LG↗

Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents

The advancement of LLMs and their accessibility have triggered renewed interest in multi-agent reinforcement learning as robust and adaptive frameworks for dynamically changing environments. This paper introduces \texttt{RL-Focal}, a two-stage RL agent framework that routes and ensembles LLMs. \textit{First}, we develop the Decider RL-agent, which learns to dynamically select an ensemble of small size ($m_i$) among $N$ LLMs ($m_i \ll N$) for incoming queries from a user-defined downstream task $i$, by maximizing both error-diversity and reasoning-performance of the selected ensemble through iterative updates of task-adaptive rewards and policy. \textit{Second}, to enable effective fusion of dynamically selected LLMs, we develop the stage-2 Fusion RL-agent, which learns to resolve reasoning conflicts from different LLMs and dynamically adapt to different ensemble teams composed by the Decider Agent for different downstream tasks. {\em Third}, we introduce the focal diversity metric to better model the error correlations among multiple LLMs further improving the generalization performance of the Decider Agent, which actively prunes the ensemble combinations. By focal diversity, we enhance performance across tasks by effectively promoting reward-aware and policy-adaptive ensemble selection and inference fusion. Extensive evaluations on five benchmarks show that RL-Focal achieves the performance improvement of 8.48\% with an ensemble of small size compared to the best individual LLM in a pool and offers stronger robustness. Code is available \href{https://github.com/git-disl/RL-Focal}{here}.

cs.CL↗

On the Capacity of DNA Labeling in the Single-Label Setting

DNA labeling has attracted increasing attention in biomedical applications, including molecular imaging, diagnostics, and genomic analysis. In a DNA labeling process, a set of DNA sequence patterns, referred to as labels, is designed according to the requirements of a specific application. For each DNA sequence, the labeling process generates an output sequence that records the positions of the labels. DNA sequences with different labeling outputs can therefore be distinguished through the labeling process. To quantify this capability, the labeling capacity is defined as the exponential growth rate of the maximum number of DNA sequences that can be distinguished through the labeling process as the sequence length tends to infinity [2]. To date, the labeling capacities of several cases in the single-label setting have been determined. In this paper, we formulate the labeling process as a deterministic channel and show that its zero-error capacity is equal to the labeling capacity. For a single label, the corresponding channel can be represented by a star graph. Thus, characterizing the labeling capacity of a single label is equivalent to determining the zero-error capacity of the corresponding star graph. We derive the zero-error capacities of all star graphs, thereby providing a complete characterization of the labeling capacities for all single-label cases. Furthermore, we develop a general method for constructing capacity-achieving codes. These results apply to labeling problems over arbitrary finite alphabets and are not restricted to the DNA alphabet. Finally, for a fixed label length, we exactly characterize the range of achievable labeling capacities and identify all single-label structures that attain the minimum and maximum capacities.

cs.IT↗

Muon Detection and Direction Reconstruction with the Upgraded 1-ton Water Cherenkov Prototype Detector at CJPL-I

The 1-ton prototype detector for Jinping neutrino experiment (JNE-1ton) has completed its hardware upgrade and has been successfully operated in both water and liquid-scintillator modes at CJPL-I, which is situated under a 2400-m rock overburden. During the upgrade, the detector's mechanical support structure and PMT layout were redesigned. The original Hamamatsu PMTs were replaced with 8-inch MCP-PMTs from North Night Vision, increasing the PMT count from 30 to 60. The detector was then operated in water mode for 85 days (Water-I) and 79 days (Water-II). From the accumulated water-mode data, we derive a muon detection efficiency (Water-II) that is approximately 59% higher than that of the pre-upgrade liquid scintillator detector. The measured muon flux is $ϕ_{\text{I+II}} = (3.55 \pm 0.43_{\mathrm{stat}}\pm 0.28_{\mathrm{syst}}) \times 10^{-10}~\mathrm{cm}^{-2}\mathrm{s}^{-1}$, which is consistent with the previous measurement. Owing to the characteristic angular dependence of Cherenkov radiation and the increased PMT coverage, the uncertainty in muon direction reconstruction is approximately 6$^\circ$, which corresponds to a reduction to 30% of its previous value. Notably, one rare up-going muon event was clearly identified. It is excluded from the cosmic-ray muon flux sample and opens a new window for neutrino-induced event studies in the deepest underground laboratory in China. This work marks the first time that CJPL has employed a cost-effective water detector for muon flux measurement.

hep-ex↗

MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method

Molecular function is largely determined by structure. Accurately aligning molecular structure with natural language is therefore essential for enabling large language models (LLMs) to both understand structure for chemical reasoning and generate molecules from natural-language design intent. However, the substantial cost of human annotation makes it infeasible to construct large-scale, high-quality datasets of structure-grounded descriptions. This work proposes a fully automated annotation framework for generating precise molecular descriptions at scale, such that the original molecule can be unambiguously reconstructed from the description alone. Our approach extends a rule-based chemical nomenclature parser to interpret IUPAC names and construct enriched, XML metadata that explicitly encodes molecular structure. This is then used to guide LLMs in producing accurate natural-language descriptions. Using this framework, we curate MolLangData, a dataset of approximately $163$k molecule--description pairs. A rigorous validation protocol combining expert human and LLM-based reconstruction on a subset of $2,000$ molecules demonstrates $98.6\%$ description precision. Using the curated dataset, we train a $4$B-parameter LLM via large-scale reinforcement learning for language-conditional molecule generation. The proposed framework and dataset provide a reliable foundation for molecule--language alignment, readily beneficial to broader chemical tasks.

cs.CL↗

3DHarnessBench: Probing Agentic 3D-to-Code Capabilities of Frontier Vision-Language Models

We introduce 3DHarnessBench, a benchmark that evaluates the agentic ability of frontier vision-language models (VLMs) to recover 3D geometry as Blender Python code from a variety of inputs. Unlike previous frameworks that prompt the VLMs with a fixed input (e.g., a single rendering or a text description), 3DHarnessBench evaluates four separate harness settings that progressively enable active agentic exploration, facilitated by recent Blender MCP functionality. Our hierarchy from Single-view, Multi-view, Active Visual (arbitrary viewpoint access), and Full 3D Interaction (complete access to the target object through Blender function calls) probes the models' abilities in both visual perception and active inference, tool calling, and self-correction. We observe that the ability of all frontier models to recover 3D geometry improves significantly with richer function call access, although the improvements are strongly model-dependent, revealing highly uneven agentic 3D-to-code capabilities. We will release the benchmark, code, outputs, and agent trajectories for reproducible 3D evaluation.

cs.CV↗

A novel filtering method for generating desired density profiles of colliding nuclei

Accurate modeling of the density profile is essential for studying heavy-ion collisions (HICs) with a transport model. Within the framework of the quantum molecular dynamics (QMD)-type model, a novel method for generating desired nuclear density distributions based on Fourier series expansion is proposed. This new initialization method is further incorporated into the ultrarelativistic quantum molecular dynamics model, and the bubble-like density distribution of $^{96}$Ru is constructed. Then, by simulating $^{96}$Ru+$^{96}$Ru collisions at $E_{\rm lab}=1500$ MeV/nucleon with different equations of state (EoS) and initialization methods, the effects of the initial density distribution on the final state observables and the constrained information of EoS are analyzed. It is found that $^{96}$Ru nuclei with a bubble density profile lead to an increased maximum compression during the collision, which in turn enhances the collective flow. Moreover, a relatively stiff EoS with $K_0>280$ MeV is favored for the conventional Woods-Saxon type density profile, whereas an EoS with $K_0$=200-280 MeV is supported when a bubble-like density profile is employed. These results demonstrate that the initial nuclear density distribution plays a non-negligible role in dynamical observables and EoS constraints. The proposed method thus provides a powerful tool for constructing exotic profiles and investigating nuclear structure effects in HICs.

nucl-th↗

PACE: Adaptive Budget Allocation for Time-Efficient Embodied Planning

Reasoning-enhanced large language models have achieved remarkable improvements in planning tasks, yet their deployment in embodied systems remains impractical due to prohibitive inference delays-often exceeding minutes per planning instance. The fundamental bottleneck stems from the serial nature of existing paradigms: models must complete all reasoning before any action execution, leaving execution time windows entirely unexploited. We introduce PACE (Planning with Adaptive Cognitive Effort), a framework that enables interleaved reasoning and execution through two key innovations: an Interleaved Think-Act architecture that pipelines cognitive processing with action execution, and a Dynamic Budget Allocator that adapts reasoning token budgets to available execution time windows. On the Robotouille benchmark using Qwen3-8B-AWQ, PACE achieves a 10% success rate-representing a 67% improvement over the ReAct+Think baseline-while delivering 6.9 times acceleration in thinking time compared to unconstrained reasoning. The framework hides 66.8% of thinking time within execution windows, demonstrating that strategic cognitive effort allocation can simultaneously improve both planning quality and time efficiency. These results provide evidence that time-aware architectural innovations enable reasoning models to operate in latency-sensitive embodied domains where they were previously impractical.

cs.RO↗

Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems

Large-scale recommendation systems face "Memory Wall" bottlenecks due to massive, dense embedding tables. While generative retrieval uses discrete tokens for IDs, high-dimensional context still relies on inefficient dense formats. Inspired by computer vision data compression, we propose Dual-purpose Semantic IDs to achieve LLM-level I/O efficiency. Our methodology uses hierarchical quantization to condense continuous embeddings into discrete Semantic IDs performing two concurrent roles: (1) Collaborative Identity: modeling user-item interactions via learnable embedding table; and (2) Content Reconstruction: using a lightweight Semantic Decoder for on-the-fly embedding approximation. This approach replaces massive vector storage with on-demand reconstruction, reducing system overhead and data footprints. We demonstrate the efficacy of our framework through offline evaluations and successful online deployment in production-scale ranking and retrieval systems at a major video sharing platform, showing that discrete tokens are indeed all you need for highly efficient, content-rich recommendation.

cs.IR↗

Error-Building Decoding of Linear Block Codes

This paper proposes a novel maximum-likelihood (ML) soft-decision decoding framework for linear block codes, termed error-building decoding (EBD). The complete decoding process can be performed using only the parity-check matrix, without requiring any other pre-constructed information (such as trellis diagrams or error-pattern lists), and it has the potential to be further customized if specific properties of the code can be exploited. We formally define error-building blocks, and derive a recursive theorem that allows efficient construction of larger locally optimal blocks from smaller ones, thereby effectively searching for the block associated with the most likely error pattern. The EBD framework is further optimized for extended Hamming codes as an example, through offline and online exclusion mechanisms, leading to a substantial complexity reduction without loss of ML performance. Complexity analysis shows that, for extended Hamming codes of lengths 64, 128, and 256, the fully optimized EBD requires approximately an order of magnitude fewer floating-point operations on average than minimum-edge trellis Viterbi decoding at a frame error rate (FER) of $10^{-3}$.

cs.IT↗

Field-selective criticality in 2D melting revealed by multi-field Lee-Yang zeros

How a two-dimensional solid melts remains unsettled after 60 years of study, as theory, model systems, simulations, and atomic-resolution experiments continue to suggest conflicting scenarios. The same transition can appear continuous or abrupt depending on how it is observed, where this ambiguity is especially acute in confined water. Here we study bilayer water under nanoconfinement and ask not only where its phase boundaries lie, but how the system responds to the two fields that drive them: temperature and lateral pressure. Using Lee-Yang zeros together with enhanced sampling, we find that some phase boundaries are field-selective: the two responses can differ either in continuity itself, or in how strongly they are rounded in finite systems. This distinction changes the two-step melting picture. The solid--hexatic transition is field-selective first-order, with the density channel remaining unusually rounded, whereas the hexatic--liquid transition becomes a conventional first-order transition once larger cells reveal a hidden bimodal enthalpy distribution. This framework organizes the apparent disagreement among confined-water simulations, hard-disk models and AgI experiments by identifying which thermodynamic channel each probe sees.

cond-mat.soft↗

A Survey on Large Language Model-Based Game Agents

Game environments provide rich, controllable settings that stimulate many aspects of real-world complexity. As such, game agents offer a valuable testbed for exploring capabilities relevant to Artificial General Intelligence. Recently, the emergence of Large Language Models (LLMs) provides new opportunities to endow these agents with generalizable reasoning, memory, and adaptability in complex game environments. This survey offers an up-to-date review of LLM-based game agents (LLMGAs) through a unified reference architecture. At the single-agent level, we synthesize existing studies around three core components: memory, reasoning, and perception-action interfaces, which jointly characterize how language enables agents to perceive, think, and act. At the multi-agent level, we outline how communication protocols and organizational models support coordination, role differentiation, and large-scale social behaviors. To contextualize these designs, we introduce a challenge-centered taxonomy linking six major game genres to their dominant agent requirements, from low-latency control in action games to open-ended goal formation in sandbox worlds. A curated list of related papers is available at https://github.com/git-disl/awesome-LLM-game-agent-papers

cs.AI↗

Tracking metastable phases by complex Lee-Yang zeros

Metastable phases (MPs) are energetically unfavorable states typically suppressed in equilibrium phase diagrams. Rather than remaining ''hidden'', we show that they exist in the complex plane of thermal fields, as regions delineated by Lee-Yang zeros (LYZs). We demonstrate this numerically in a toy model with a tunable density of states featuring three Gaussian peaks and in a more realistic periodically driven system. In both cases, as artificial parameters or drive amplitudes increase, the LYZs bounding the MP approach the real axis and split into separated branches, signaling the emergence and stabilization of the MP within the enlarged gap between two adjacent stable phases. In the driven system, the imaginary part of LYZs correlates with drive strength, linking Lee-Yang theory to terahertz matter manipulation. These findings provide a scheme to describe MPs in phase diagram analysis. By viewing periodic drives as complex thermal fields, it also offers a new perspective for understanding and engineering non-equilibrium collective states.

cond-mat.stat-mech↗

Non-Invasive Reconstruction of Intracranial EEG Across the Deep Temporal Lobe from Scalp EEG based on Conditional Normalizing Flow

Although obtaining deep brain activity from non-invasive scalp electroencephalography (sEEG) is crucial for neuroscience and clinical diagnosis, directly generating high-fidelity intracranial electroencephalography (iEEG) signals remains a largely unexplored field, limiting our understanding of deep brain dynamics. Current research primarily focuses on traditional signal processing or source localization methods, which struggle to capture the complex waveforms and random characteristics of iEEG. To address this critical challenge, this paper introduces NeuroFlowNet, a novel cross-modal generative framework whose core contribution lies in the first-ever reconstruction of iEEG signals from the entire deep temporal lobe region using sEEG signals. NeuroFlowNet is built on Conditional Normalizing Flow (CNF), which directly models complex conditional probability distributions through reversible transformations, thereby explicitly capturing the randomness of brain signals and fundamentally avoiding the pattern collapse issues common in existing generative models. Additionally, the model integrates a multi-scale architecture and self-attention mechanisms to robustly capture fine-grained temporal details and long-range dependencies. Validation results on a publicly available synchronized sEEG-iEEG dataset demonstrate NeuroFlowNet's effectiveness in terms of temporal waveform fidelity, spectral feature reproduction, and functional connectivity restoration. This study establishes a more reliable and scalable new paradigm for non-invasive analysis of deep brain dynamics. The code of this study is available in https://github.com/hdy6438/NeuroFlowNet

q-bio.NC↗

MELT: A Behavioral Trace Dataset for High-Risk Memecoin Launch Detection

Launchpads have become the dominant mechanism for issuing memecoins, exposing investors to a new class of high-risk launches that existing rug-pull detection methods cannot capture. We argue that detecting these threats requires structured behavioral traces that underlie raw heterogeneous blockchain data, i.e., how insiders accumulate, coordinate, and unwind positions. To enable such analysis, we introduce MELT (MEmecoin Launch Trace, the first behavioral trace dataset for analyzing and detecting high-risk memecoin launches on Solana. MELT covers 41k+ memecoin launches with 200M+ transactions parsed into typed behavioral records that distinguish swaps, wash trades, transfers, and mints. Beyond per-account behaviors, MELT contributes bundle-trace data that links accounts controlled by the same entity, revealing that, on average, 36.5% of token supply is held by coordinated accounts, a concealment strategy that disguises the true ownership concentration from unsuspecting buyers. On top of these traces, MELT provides 122 behavioral features and risk-level annotations, enabling supervised learning at a population scale. We benchmark representative ML models on the high-risk launch detection task. Integrating their predictions into a simple memecoin selection strategy reduces investment loss significantly, demonstrating that behavioral traces can be translated into risk mitigation. Our dataset and code is available at https://github.com/git-disl/MELT.

cs.CR↗

Personalized Face Privacy Protection From a Single Image

Photos of faces uploaded online are vulnerable to malicious actors who can scrape facial images from online sources and intrude on personal privacy via unauthorized use of facial recognition models. This paper presents FaceCloak, a novel personalized face privacy protection system, which can generate defensive identity-specific universal face privacy masks from a single image of a user, causing facial recognition to fail. FaceCloak introduces a three-stage personalized face perturbation learning methodology: (1) It generates a small set of high-variety synthetic face images of a person based on a single image of the person. (2) It learns face cloaking by adding more protection to key facial-identity leakage regions through iterative perturbation generation over the small set of synthetic images, effectively shifting a user's identity embedding towards a distant anchor identity and away from a similar one. (3) It generates a personalized identity-protective mask in the form of pixel-wise cloaking, which is light-weight and can be efficiently applied to any facial image of a user while maintaining good perceptual quality. Extensive experiments on three popular face datasets across ten recognition models show the effectiveness of FaceCloak compared to 29 other existing representative methods. Code is available at https://github.com/zacharyyahn/FaceCloak

cs.CV↗