Searcharxiv⌕ Search

arXiv subjects

Ruiyang Qin

Publications and source records attributed to Ruiyang Qin.

At least 19 recordsLinked to original sources
arXiv2026

Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions

The remarkable capabilities of large language models (LLMs) are often undermined by their instability. Even subtle and semantically irrelevant changes in prompts can cause dramatic fluctuations in performance, a phenomenon known as prompt sensitivity. Previous studies typically evaluate prompt sensitivity by comparing the LLM's final outputs when prompts change. However, such coarse-grained metrics fail to explain the internal reasons for prompt sensitivity. In this paper, we introduce interactions as a fine-grained tool to analyze prompt sensitivity of LLMs. Specifically, we decompose the output score of the LLM into a set of interactions. Each interaction represents a nonlinear relationship involving a set of input variables. We discover that subtle changes to prompts can trigger severe instability in interactions, even when the outputs of the LLM remain the same. To this end, we propose an Interaction-based Prompt Sensitivity (IPS) metric by quantifying changes in interactions when we introduce subtle changes to prompts. We apply the IPS metric to 50 open-source LLMs and uncover four factors that reduce the prompt sensitivity of LLMs, including supervised fine-tuning, increased model scales, dense architectures, and few-shot learning. More crucially, we discover a common mechanism by which these four factors reduce prompt sensitivity: all four factors tend to reduce the prompt sensitivity of low-order interactions (i.e., interactions involving few input variables).

cs.LG↗
arXiv2026

PolySim: Deterministic Polynomial Surrogates for Cross-Modal Retrieval on CiM

Cross-modal retrieval on edge devices benefits from probabilistic embeddings that capture semantic uncertainty, but deploying them on compute-in-memory (CiM) hardware remains an open problem. The core difficulty is a sampling gap: probabilistic methods such as PCME rely on Monte Carlo sampling and nonlinear distance evaluation at inference, which are fundamentally incompatible with CiM crossbar arrays that support only deterministic, single-step matrix-vector multiplication. Few existing probabilistic retrieval methods can be executed on a conventional crossbar. To bridge this gap, we propose PolySim, a framework that reformulates probabilistic retrieval into a fully deterministic pipeline. PolySim approximates each Gaussian embedding dimension using low-order polynomial bases and computes similarity via a learnable order-bilinear kernel, eliminating stochastic sampling while preserving distributional information. In experiments on six benchmarks spanning video, image, and audio retrieval, PolySim improves R@1 over deterministic baselines by up to 10.3\% and matches or exceeds PCME, while reducing inference to a single crossbar-compatible matrix-vector multiplication. CrossSim evaluation under realistic device non-idealities confirms robust deployment on conventional crossbar arrays. To the best of our knowledge, PolySim is the first method to enable probabilistic cross-modal retrieval on CiM hardware.

cs.ET↗
arXiv2026

Probabilistic Memory for Trustworthy Edge Intelligence

Probabilistic computation plays an important role in trustworthy edge intelligence to quantify uncertainty, enhance robustness, reconstruct data, and protect privacy, but its adoption is limited by the orders-of-magnitude data throughput gap between Gaussian random number generation (GRNG) and computation, as well as instruction overhead. This paper introduces probabilistic memory (p-MEM), a unified memory primitive that stores distribution parameters, such as mean and standard deviation, and samples directly at the native memory bandwidth, where deterministic data becomes the zero-variance special case. Using a layout-validated p-MEM simulator, we comprehensively explore device choices, memory specifications, and technology nodes, showing that p-MEM can achieve more than 1000 GSa/s/mm^2 GRNG throughput, including memory-array access. Integrated into CPU/GPU systems, p-MEM reduces instruction count by up to 2.19x/4.37x, sampling latency by 562x/3.45x, and energy by 295.5x/3.53x for Bayesian neural network workloads, providing a scalable hardware substrate for trustworthy probabilistic AI.

cs.AR↗
arXiv2026

Mid-infrared snapshot spectral imaging via nonlinear radial dispersion

Mid-infrared (MIR) spectral imaging provides chemically specific contrast through molecular vibrational fingerprints, yet snapshot acquisition remains severely limited by the lack of high-sensitivity detectors and efficient spectral encoding mechanisms. Here we introduce snapshot MIR spectral imaging based on intrinsic nonlinear radial dispersion, in which wavelength-dependent phase matching simultaneously enables frequency upconversion and spectral multiplexing. Different spectral components are mapped to distinct output angles within a 4$f$ imaging architecture, enabling single-shot spectral encoding without external dispersive elements. In combination with speckle illumination encoding, spectral information is compressed and recovered without additional coding components. Leveraging nonlinear upconversion to the visible, the approach achieves room-temperature MIR spectral imaging with sensitivity approaching 1 photon/pixel/pulse across a broad spectral range from 2.5 to 4.0 $\mu$m. This work transforms spectral encoding from an external optical function into an inherent property of the nonlinear imaging process, providing a general route to high-sensitivity snapshot MIR spectral imaging.

physics.optics↗
arXiv2026

AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security

Modern open-world agents such as OpenClaw exhibit powerful cross-environment execution capabilities yet introduce broad new safety risk sources. Meanwhile, advanced frontier AI models drastically lower attack barriers, rendering current agent alignment frameworks inadequate for real-world deployment. To tackle these emerging threats, we propose a lightweight and scalable agent safety alignment framework. Specifically, we update the agent safety taxonomy to accommodate emergent risks from Codex and OpenClaw execution scenarios. We further build a taxonomy-guided data engine with influence-function purification to train lightweight AgentDoG 1.5 variants (0.8B, 2B, 4B, and 8B parameters) using only around 1k samples, achieving comparable performance with leading closed-source models (e.g., GPT-5.4). Based on AgentDoG 1.5, we construct a highly efficient agentic safety SFT and RL training environment, which reduces deployment overhead in Docker-level environments by two orders of magnitude. Finally, we deploy AgentDoG 1.5 as a training-free online guardrail for real-time safety moderation. Extensive experimental results indicate that AgentDoG 1.5 achieves state-of-the-art performance in diverse and complex interactive agentic scenarios. All models and datasets are openly released.

cs.AI↗
arXiv2026

Wide-field mid-infrared hyperspectral imaging beyond video rate

Mid-infrared hyperspectral imaging has become an indispensable tool to spatially resolve chemical information in a wide variety of samples. However, acquiring three-dimensional data cubes is typically time-consuming due to the limited speed of raster scanning or wavelength tuning, which impedes real-time visualization with high spatial definition across broad spectral bands. Here, we devise and implement a high-speed, wide-field mid-infrared hyperspectral imaging system relying on broadband parametric upconversion of high-brightness supercontinuum illumination at the Fourier plane. The upconverted replica is spectrally decomposed by a rapid acousto-optic tunable filter, which records high-definition monochromatic images at a frame rate of 10 kHz based on a megapixel silicon camera. Consequently, the hyperspectral imager allows us to acquire 100 spectral bands over 2600-4085 cm$^{-1}$ in 10 ms, corresponding to a refreshing rate of 100 Hz. Moreover, the angular dependence of phase matching in the image upconversion is leveraged to realize snapshot operation with spatial multiplexing for multiple spectral channels, which may further boost the spectral imaging rate. The high acquisition rate, wide-field operation, and broadband spectral coverage could open new possibilities for high-throughput characterization of transient processes in material and life sciences.

physics.optics↗
arXiv2026

Mid-Infrared Single-Photon Compressive Spectroscopy

Sensitive mid-infrared (MIR) spectroscopy plays an indispensable role in various photon-starved conditions. However, the detection sensitivity of conventional MIR spectrometers is severely limited by excessive noises of the involved infrared sensors, especially for multi-pixel arrays in parallel spectral acquisition. Here, we devise and implement an ultra-sensitive MIR single-pixel spectrometer, which relies on high-fidelity spectral upconversion and wavelength-encoding compressive measurement. Specifically, a MIR nanophotonic supercontinuum from 3.1 to 3.9 $\mu$m is nonlinearly converted to the near-infrared band via synchronous chirped-pulse pumping, which facilitates both the precise spectral mapping and sensitive upconversion detection. The upconverted signal is then spatially dispersed onto a programmable digital micromirror device, before being registered by a single-element silicon detector. Consequently, the spectral information can be deciphered from the correlation between encoded patterns and recorded measurements, which results in a spectral resolution of 0.5 cm$^{-1}$ under an illumination flux down to 0.01 photons/nm/pulse. Moreover, we demonstrate faithful reconstructions at sub-Nyquist sampling rates by using the compressive sensing algorithm, which leads to a 95\% reduction in data acquisition time. The presented single-pixel computational spectrometer features wavelength multiplexing, high throughput, and efficient sampling, which thus paves a new way for sensitive and fast spectroscopic analysis at the single-photon level.

physics.optics↗
arXiv2026

High-speed mid-infrared imaging via nonlinear multiplexed detection

High-speed mid-infrared (MIR) videography constitutes an enabling tool to monitor and analyze various dynamics in scientific research and industrial applications, such as combustion diagnostics, explosion reactions, photosynthetic tracking, and thermal surveillance. However, the frame rate of conventional MIR imagers is typically limited by readout electronics and detection sensitivity, especially for large spatial formats with massive pixels. Here, we devise and implement a high-speed MIR upconversion imaging system based on time-multiplexed nonlinear structured pumping. Specifically, the dynamic infrared scene is optically gated by a sequence of spatially periodical pump patterns in a nonlinear crystal, which facilitates both rapid temporal encryption and sensitive upconversion detection. Then, the upconverted frames are superimposed onto a silicon camera within a single exposure, thus resulting in a multiplexed snapshot in the spatial-frequency domain. Finally, the sub-exposure images, corresponding to distinct transient events, can be computationally deciphered and reconstructed by the frequency recognition algorithm based on band-pass filtering and Fourier transform operations. The achieved frame rate is tenfold boosted to 10,000 frames per second without compromising the megapixel spatial format, which allows continuous real-time MIR videography at high speed and high definition. The presented approach could be readily extended to far-infrared or terahertz spectral regions, with an aim of performing high-throughput and high-sensitivity observation of transient phenomena with high temporal complexity.

physics.optics↗
arXiv2026

Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement

Large Vision-Language Models (LVLMs) have achieved remarkable performance on diverse vision-language tasks. However, LVLMs still suffer from hallucinations, generating text that contradicts the visual input. Existing research has primarily focused on mitigating object hallucinations, but often overlooks more complex relation hallucinations, particularly action relations involving interactions between objects. In this study, we empirically observe that the primary cause of action-relation hallucinations in LVLMs is the insufficient attention allocated to visual information. Thus, we propose a framework to locate action-relevant image regions and enhance the LVLM's attention to those regions. Specifically, we define the Action-Relation Sensitivity (ARS) score to identify attention heads that are most sensitive to action-relation changes, thereby localizing action-relevant image regions that contain key visual cues. Then, we propose the Relation-aware Visual Enhancement (RVE) method to enhance the LVLM's attention to these action-relevant image regions. Extensive experiments demonstrate that, compared to existing baselines, our method achieves superior performance in mitigating action-relation hallucinations with negligible additional inference cost. Furthermore, it effectively generalizes to spatial-relation hallucinations and object hallucinations.

cs.CV↗
arXiv2026

A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions

Despite the success of knowledge distillation (KD) in Large Language Models (LLMs), the underlying mechanism behind its efficacy remains unclear. In this paper, we propose a unified approach to explore the common mechanism of various KD methods using interactions. Specifically, we decompose the output score of the LLM into the sum of numerous interactions. Each interaction represents a nonlinear relationship involving a set of input variables (e.g., words). Based on the decomposed interactions, we discover that the common mechanism underlying various KD methods is the sparsification of interactions, i.e., student models retain fewer interactions for inference while suppressing other interactions to zero effects. Furthermore, we discover that the performance variance across different KD methods arises from their capabilities in handling complex interactions. A KD method typically yields better performance if it enables the student model to achieve higher sparsity of complex interactions. Motivated by these insights, we propose a plug-and-play loss function called Complex Interaction Penalty (CIP) to explicitly enforce the sparsity of complex interactions during the distillation process. Extensive experiments demonstrate that integrating CIP consistently improves the performance of diverse KD methods on both in-domain and out-of-distribution benchmarks.

cs.LG↗
arXiv2026

Multilingual Safety Alignment via Self-Distillation

Large language models (LLMs) exhibit severe multilingual safety misalignment: they possess strong safeguards in high-resource languages but remain highly vulnerable to jailbreak attacks in low-resource languages. Current safety alignment methods generally rely on high-quality response data for each target language, which is expensive and difficult to generate. In this paper, we propose a cross-lingual safeguard transfer framework named Multilingual Self-Distillation (MSD). This framework transfers an LLM's inherent safety capabilities from high-resource (e.g., English) to low-resource (e.g., Javanese) languages, overcoming the need for response data in any language. Our framework is flexible and can be integrated with different self-distillation strategies. Specifically, we implement two concrete methods -- on-policy MSD and off-policy MSD -- both of which enable effective cross-lingual safety transfer using only multilingual queries. Furthermore, we propose Dual-Perspective Safety Weighting (DPSW), a divergence measure to optimize the distillation objective. By jointly considering the perspectives of both the teacher and the student, DPSW adaptively increases the penalty weights on safety-critical tokens while reducing the weights on non-critical tokens. Extensive experiments on representative LLMs across diverse multilingual jailbreak and utility benchmarks demonstrate that our method consistently achieves superior multilingual safety performance. Notably, it generalizes effectively to more challenging datasets and unseen languages while preserving the model's general capabilities.

cs.LG↗
arXiv2026

MorphOPC: Advancing Mask Optimization with Multi-scale Hierarchical Morphological Learning

As feature sizes shrink to the nanometer scale, accurately transferring circuit patterns from photomasks to silicon wafers becomes increasingly challenging. Optical proximity correction (OPC) is widely used to ensure pattern fidelity and manufacturability. Recent generative mask optimization models based on encoder-decoder architecture can synthesize near-optimal masks, serving as fast machine learning (ML) surrogates for traditional OPC. However, these models often fail to capture the geometric transformations from target layouts to mask patterns, leading to suboptimal quality. In this work, we formulate mask generation as a sequence of morphological operations on local layout features and propose \textit{MorphOPC}, a multi-scale hierarchical model with neural morphological modules to learn these transformations. Experiments on edge-based OPC and ILT benchmarks across metal and via layers show that \textit{MorphOPC} consistently outperforms state-of-the-art methods, achieving higher printing fidelity and lower manufacturing cost, demonstrating strong potential for scalable mask optimization.

cs.CV↗
arXiv2026

Toward Accountable AI-Generated Content on Social Platforms: Steganographic Attribution and Multimodal Harm Detection

The rapid growth of generative AI has introduced new challenges in content moderation and digital forensics. In particular, benign AI-generated images can be paired with harmful or misleading text, creating difficult-to-detect misuse. This contextual misuse undermines the traditional moderation framework and complicates attribution, as synthetic images typically lack persistent metadata or device signatures. We introduce a steganography enabled attribution framework that embeds cryptographically signed identifiers into images at creation time and uses multimodal harmful content detection as a trigger for attribution verification. Our system evaluates five watermarking methods across spatial, frequency, and wavelet domains. It also integrates a CLIP-based fusion model for multimodal harmful-content detection. Experiments demonstrate that spread-spectrum watermarking, especially in the wavelet domain, provides strong robustness under blur distortions, and our multimodal fusion detector achieves an AUC-ROC of 0.99, enabling reliable cross-modal attribution verification. These components form an end-to-end forensic pipeline that enables reliable tracing of harmful deployments of AI-generated imagery, supporting accountability in modern synthetic media environments. Our code is available at GitHub: https://github.com/bli1/steganography

cs.CV↗
arXiv2026

TME-PSR: Time-aware, Multi-interest, and Explanation Personalization for Sequential Recommendation

In this paper, we propose a sequential recommendation model that integrates Time-aware personalization, Multi-interest personalization, and Explanation personalization for Personalized Sequential Recommendation (TME-PSR). That is, we consider the differences across different users in temporal rhythm preference, multiple fine-grained latent interests, and the personalized semantic alignment between recommendations and explanations. Specifically, the proposed TME-PSR model employs a dual-view gated time encoder to capture personalized temporal rhythms, a lightweight multihead Linear Recurrent Unit architecture that enables fine-grained sub-interest modeling with improved efficiency, and a dynamic dual-branch mutual information weighting mechanism to achieve personalized alignment between recommendations and explanations. Extensive experiments on real-world datasets demonstrate that our method consistently improves recommendation accuracy and explanation quality, at a lower computational cost.

cs.IR↗
arXiv2026

CQ-CiM: Hardware-Aware Embedding Shaping for Robust CiM-Based Retrieval

Deploying Retrieval-Augmented Generation (RAG) on edge devices is in high demand, but is hindered by the latency of massive data movement and computation on traditional architectures. Compute-in-Memory (CiM) architectures address this bottleneck by performing vector search directly within their crossbar structure. However, CiM's adoption for RAG is limited by a fundamental ``representation gap,'' as high-precision, high-dimension embeddings are incompatible with CiM's low-precision, low-dimension array constraints. This gap is compounded by the diversity of CiM implementations (e.g., SRAM, ReRAM, FeFET), each with unique designs (e.g., 2-bit cells, 512x512 arrays). Consequently, RAG data must be naively reshaped to fit each target implementation. Current data shaping methods handle dimension and precision disjointly, which degrades data fidelity. This not only negates the advantages of CiM for RAG but also confuses hardware designers, making it unclear if a failure is due to the circuit design or the degraded input data. As a result, CiM adoption remains limited. In this paper, we introduce CQ-CiM, a unified, hardware-aware data shaping framework that jointly learns Compression and Quantization to produce CiM-compatible low-bit embeddings for diverse CiM designs. To the best of our knowledge, this is the first work to shape data for comprehensive CiM usage on RAG.

cs.ET↗
arXiv2025

Driving Through Uncertainty: Risk-Averse Control with LLM Commonsense for Autonomous Driving under Perception Deficits

Partial perception deficits can compromise autonomous vehicle safety by disrupting environmental understanding. Existing protocols typically default to entirely risk-avoidant actions such as immediate stops, which are detrimental to navigation goals and lack flexibility for rare driving scenarios. Yet, in cases of minor risk, halting the vehicle may be unnecessary, and more adaptive responses are preferable. In this paper, we propose LLM-RCO, a risk-averse framework leveraging large language models (LLMs) to integrate human-like driving commonsense into autonomous systems facing perception deficits. LLM-RCO features four key modules interacting with the dynamic driving environment: hazard inference, short-term motion planner, action condition verifier, and safety constraint generator, enabling proactive and context-aware actions in such challenging conditions. To enhance the driving decision-making of LLMs, we construct DriveLM-Deficit, a dataset of 53,895 video clips featuring deficits of safety-critical objects, annotated for LLM fine-tuning in hazard detection and motion planning. Extensive experiments in adverse driving conditions with the CARLA simulator demonstrate that LLM-RCO promotes proactive maneuvers over purely risk-averse actions in perception deficit scenarios, underscoring its value for boosting autonomous driving resilience against perception loss challenges.

cs.RO↗
arXiv2025

Sub-Sequential Physics-Informed Learning with State Space Model

Physics-Informed Neural Networks (PINNs) are a kind of deep-learning-based numerical solvers for partial differential equations (PDEs). Existing PINNs often suffer from failure modes of being unable to propagate patterns of initial conditions. We discover that these failure modes are caused by the simplicity bias of neural networks and the mismatch between PDE's continuity and PINN's discrete sampling. We reveal that the State Space Model (SSM) can be a continuous-discrete articulation allowing initial condition propagation, and that simplicity bias can be eliminated by aligning a sequence of moderate granularity. Accordingly, we propose PINNMamba, a novel framework that introduces sub-sequence modeling with SSM. Experimental results show that PINNMamba can reduce errors by up to 86.3\% compared with state-of-the-art architecture. Our code is available at https://github.com/miniHuiHui/PINNMamba.

cs.LG↗
arXiv2025

Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data

We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks, respectively, in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks. Code, model, and demo are available at https://ntlm1686.github.io/raso.

cs.CV↗