arXiv subjects

Ling Liu

Publications and source records attributed to Ling Liu.

At least 55 records · Page 3Linked to original sources

arXiv2025

A Neurosymbolic Agent System for Compositional Visual Reasoning

The advancement in large language models (LLMs) and large vision models has fueled the rapid progress in multi-modal vision-language reasoning capabilities. However, existing vision-language models (VLMs) remain challenged by compositional visual reasoning. This paper presents VLAgent, a neuro-symbolic approach to developing a Vision-Language Agent system for efficient compositional visual reasoning with three novel features. First, VLAgent develops an interpretable visualization-enhanced two-stage neuro-symbolic reasoning system. The first stage is managed by a front-end engine that generates a structured visual reasoning plan (symbolic program script) for each compositional visual reasoning task by utilizing a pre-trained LLM powered with few-shot chain-of-thought in-context learning. The second stage is managed by a high-performance back-end engine. It transforms the planning script into executable code based on visual input (image or video) and the combination of neural models and symbolic functions and then performs a sequence of actions for the compositional visual reason task. Second, to ensure and enhance the quality of mapping the logic plan to a sequence of executable instructions, VLAgent introduces the SS-parser, which examines the syntax and semantic correctness of the planning script, detects and repairs the logic errors found in the LLM-generated logic plan before generating the executable program. Third, VLAgent introduces the execution verifier in critical reasoning steps to validate and refine its compositional reasoning results in a stepwise manner, for example, ensemble methods for critical visual reasoning and caption analysis for low-confidence compositional reasoning. Extensive experiments on six visual benchmarks compared to a dozen SoTA visual reasoning models show that VLAgent outperforms existing representative approaches to compositional visual reasoning.

cs.CV↗

arXiv2025

Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning

Safety aligned Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- a few harmful data mixed in the fine-tuning dataset can break the LLMs's safety alignment. While several defenses have been proposed, our evaluation shows that existing defenses fail \textit{when some specific training hyper-parameters are chosen} -- a large learning rate or a large number of training epochs in the fine-tuning stage can easily invalidate the defense. To this end, we propose Antidote, a post-fine-tuning stage solution, which remains \textbf{\textit{agnostic to the training hyper-parameters in the fine-tuning stage}}. Antidote relies on the philosophy that by removing the harmful parameters, the harmful model can be recovered from the harmful behaviors, regardless of how those harmful parameters are formed in the fine-tuning stage. With this philosophy, we introduce a one-shot pruning stage after harmful fine-tuning to remove the harmful weights that are responsible for the generation of harmful content. Despite its embarrassing simplicity, empirical results show that Antidote can reduce harmful score while maintaining accuracy on downstream tasks. Code is available at https://github.com/git-disl/Antidote.

cs.AI↗

arXiv2025

Adversarial Attention Perturbations for Large Object Detection Transformers

Adversarial perturbations are useful tools for exposing vulnerabilities in neural networks. Existing adversarial perturbation methods for object detection are either limited to attacking CNN-based detectors or weak against transformer-based detectors. This paper presents an Attention-Focused Offensive Gradient (AFOG) attack against object detection transformers. By design, AFOG is neural-architecture agnostic and effective for attacking both large transformer-based object detectors and conventional CNN-based detectors with a unified adversarial attention framework. This paper makes three original contributions. First, AFOG utilizes a learnable attention mechanism that focuses perturbations on vulnerable image regions in multi-box detection tasks, increasing performance over non-attention baselines by up to 30.6%. Second, AFOG's attack loss is formulated by integrating two types of feature loss through learnable attention updates with iterative injection of adversarial perturbations. Finally, AFOG is an efficient and stealthy adversarial perturbation method. It probes the weak spots of detection transformers by adding strategically generated and visually imperceptible perturbations which can cause well-trained object detection models to fail. Extensive experiments conducted with twelve large detection transformers on COCO demonstrate the efficacy of AFOG. Our empirical results also show that AFOG outperforms existing attacks on transformer-based and CNN-based object detectors by up to 83% with superior speed and imperceptibility. Code is available at https://github.com/zacharyyahn/AFOG.

cs.CV↗

arXiv2025

When Graph Contrastive Learning Backfires: Spectral Vulnerability and Defense in Recommendation

Graph Contrastive Learning (GCL) has demonstrated substantial promise in enhancing the robustness and generalization of recommender systems, particularly by enabling models to leverage large-scale unlabeled data for improved representation learning. However, in this paper, we reveal an unexpected vulnerability: the integration of GCL inadvertently increases the susceptibility of a recommender to targeted promotion attacks. Through both theoretical investigation and empirical validation, we identify the root cause as the spectral smoothing effect induced by contrastive optimization, which disperses item embeddings across the representation space and unintentionally enhances the exposure of target items. Building on this insight, we introduce CLeaR, a bi-level optimization attack method that deliberately amplifies spectral smoothness, enabling a systematic investigation of the susceptibility of GCL-based recommendation models to targeted promotion attacks. Our findings highlight the urgent need for robust countermeasures; in response, we further propose SIM, a spectral irregularity mitigation framework designed to accurately detect and suppress targeted items without compromising model performance. Extensive experiments on multiple benchmark datasets demonstrate that, compared to existing targeted promotion attacks, GCL-based recommendation models exhibit greater susceptibility when evaluated with CLeaR, while SIM effectively mitigates these vulnerabilities.

cs.IR↗

arXiv2025

Pixel-Sensitive and Robust Steganography Based on Polar Codes

Steganography is an information hiding technique for covert communication. The core issue in steganography design is the rate-distortion coding problem. Polar codes, which have been proven to achieve the rate-distortion bound for any binary symmetric source, are utilized to design a steganographic scheme that can reach the embedding capacity for the Distortion-Limited Sender problem in certain cases. In adaptive steganography, for attack scenarios where each noise element can have different intensities, existing steganographic coding methods fail to resist such attacks. In this paper, we propose a pixel-sensitive and robust steganographic scheme based on polar codes. Our steganographic scheme not only matches the adaptive distortion well but is also robust against sophisticated noise attacks. Futher, it is proven that our scheme achieves the embedding capacity in certain cases. Experimentally, a steganographic scheme can be designed and implemented with a secret message error rate at the $10^{-5}$ level when the attack noise is known to both the sender and the receiver. This demonstrates its significant robustness.

cs.CR↗

arXiv2025

Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable

Safety alignment is an important procedure before the official deployment of a Large Language Model (LLM). While safety alignment has been extensively studied for LLM, there is still a large research gap for Large Reasoning Models (LRMs) that equip with improved reasoning capability. We in this paper systematically examine a simplified pipeline for producing safety aligned LRMs. With our evaluation of various LRMs, we deliver two main findings: i) Safety alignment can be done upon the LRM to restore its safety capability. ii) Safety alignment leads to a degradation of the reasoning capability of LRMs. The two findings show that there exists a trade-off between reasoning and safety capability with the sequential LRM production pipeline. The discovered trade-off, which we name Safety Tax, should shed light on future endeavors of safety research on LRMs. As a by-product, we curate a dataset called DirectRefusal, which might serve as an alternative dataset for safety alignment. Our source code is available at https://github.com/git-disl/Safety-Tax.

cs.CR↗

arXiv2025

From Intention To Implementation: Automating Biomedical Research via LLMs

Conventional biomedical research is increasingly labor-intensive due to the exponential growth of scientific literature and datasets. Artificial intelligence (AI), particularly Large Language Models (LLMs), has the potential to revolutionize this process by automating various steps. Still, significant challenges remain, including the need for multidisciplinary expertise, logicality of experimental design, and performance measurements. This paper introduces BioResearcher, the first end-to-end automated system designed to streamline the entire biomedical research process involving dry lab experiments. BioResearcher employs a modular multi-agent architecture, integrating specialized agents for search, literature processing, experimental design, and programming. By decomposing complex tasks into logically related sub-tasks and utilizing a hierarchical learning approach, BioResearcher effectively addresses the challenges of multidisciplinary requirements and logical complexity. Furthermore, BioResearcher incorporates an LLM-based reviewer for in-process quality control and introduces novel evaluation metrics to assess the quality and automation of experimental protocols. BioResearcher successfully achieves an average execution success rate of 63.07% across eight previously unmet research objectives. The generated protocols, on average, outperform typical agent systems by 22.0% on five quality metrics. The system demonstrates significant potential to reduce researchers' workloads and accelerate biomedical discoveries, paving the way for future innovations in automated research systems.

cs.MA↗

arXiv2025

Determination of melting temperature of hexagonal ice using Lee-Yang phase transition theory

Lee-Yang phase transition theory is a milestone in statistical physics. Its applications in realistic systems, however, had been substantially hindered by availability of practical schemes to calculate the Lee-Yang zeros. In this manuscript, we extend the scheme we have designed earlier [Phys. Rev. E 109, 024118 (2024)] and report simulation results for the melting temperature (T) of ice Ih under ambient pressure. The enhanced sampling technique is shown to be crucial for accessing Lee-Yang zeros accurately. The real and imaginary parts of our Lee-Yang edges demonstrate linear scaling of sizes, which can lead to a melting T of 248.15 K for the TIP4P/2005 potential in the thermodynamic limit. This result is in close quantitative agreement with previous coexistence simulations, achieved with cheaper computational costs and without prior knowledge of the phase transition. With these, we demonstrate the applicability of Lee-Yang phase transition theory in realistic molecular systems, and provide a feasible scheme for high-throughput calculations in determining the phase transition temperature.

cond-mat.stat-mech↗

arXiv2025

Residual Cross-Attention Transformer-Based Multi-User CSI Feedback with Deep Joint Source-Channel Coding

This letter proposes a deep-learning (DL)-based multi-user channel state information (CSI) feedback framework for massive multiple-input multiple-output systems, where the deep joint source-channel coding (DJSCC) is utilized to improve the CSI reconstruction accuracy. Specifically, we design a multi-user joint CSI feedback framework, whereby the CSI correlation of nearby users is utilized to reduce the feedback overhead. Under the framework, we propose a new residual cross-attention transformer architecture, which is deployed at the base station to further improve the CSI feedback performance. Moreover, to tackle the "cliff-effect" of conventional bit-level CSI feedback approaches, we integrated DJSCC into the multi-user CSI feedback, together with utilizing a two-stage training scheme to adapt to varying uplink noise levels. Experimental results demonstrate the superiority of our methods in CSI feedback performance, with low network complexity and better scalability.

cs.LG↗

arXiv2025

Adverseness vs. Equilibrium: Exploring Graph Adversarial Resilience through Dynamic Equilibrium

Adversarial attacks to graph analytics are gaining increased attention. To date, two lines of countermeasures have been proposed to resist various graph adversarial attacks from the perspectives of either graph per se or graph neural networks. Nevertheless, a fundamental question lies in whether there exists an intrinsic adversarial resilience state within a graph regime and how to find out such a critical state if exists. This paper contributes to tackle the above research questions from three unique perspectives: i) we regard the process of adversarial learning on graph as a complex multi-object dynamic system, and model the behavior of adversarial attack; ii) we propose a generalized theoretical framework to show the existence of critical adversarial resilience state; and iii) we develop a condensed one-dimensional function to capture the dynamic variation of graph regime under perturbations, and pinpoint the critical state through solving the equilibrium point of dynamic system. Multi-facet experiments are conducted to show our proposed approach can significantly outperform the state-of-the-art defense methods under five commonly-used real-world datasets and three representative attacks.

cs.LG↗

arXiv2025

Bridging Optical Sensing and Wearable Health Monitoring: A Functionalized Plasmonic Nanopillar for Non-Invasive Sweat Glucose Detection

Continuous glucose monitoring (CGM) is vital for diabetes care, but current systems rely on invasive implants or electrochemical sensors that often cause discomfort and skin irritation. Non-invasive alternatives remain limited by low sensitivity and poor compatibility with complex sweat environments, highlighting the urgent need for a comfortable and reliable solution. Here, we report the development of a wearable optical sensor watch that integrates surface plasmon resonance (SPR) technology with a functionalized silver-coated silicon nanowire (Ag/SiNW) substrate for real-time, non-invasive glucose monitoring in sweat. The nanostructured sensor is functionalized with 4-mercaptophenylboronic acid (4-MPBA), enabling selective glucose capture and optical signal transduction through both Raman scattering and SPR shift. The dual-mode detection strategy was systematically optimized, and a miniaturized SPR system operating at 638 nm was successfully integrated into a wearable watch format with wireless data transmission to a mobile application. This wearable device demonstrated excellent sensitivity (LOD down to 0.12 mM) and high selectivity in detecting glucose within physiological sweat concentration ranges. Human subject trials confirmed its applicability in real-life scenarios. This study offers a promising non-invasive alternative to traditional CGM and highlights the potential of integrating nanophotonic sensors with wearable platforms for continuous health monitoring and personalized medicine.

physics.optics↗

arXiv2025

Effects of appendages on the turbulence and flow noise of a submarine model using high-order scheme

This study employs high-fidelity numerical simulations to investigate the influence of appendages on the turbulent flow dynamics and far-field acoustic radiation of the SUBOFF submarine model at a Reynolds number of Re = 1.2*10^7. Utilizing a third-order numerical scheme combined with wall-modeled large eddy simulation (WMLES) and the Ffowcs Williams-Hawkings (FW-H) acoustic analogy, the hydrodynamic and acoustic behaviors of an appended SUBOFF configuration are compared to those of a bare hull. A computational grid of 103 million cells resolves the intricate flow interactions, while 648 hydrophones positioned 500 diameters from the model capture far-field acoustic signatures. Key results reveal that appendages significantly amplify hydrodynamic and acoustic disturbances. Flow separations and vortex shedding at appendage junctions elevate pressure-induced drag contributions, contrasting the viscous-dominated drag of the bare hull. The sail-hull interaction intensifies local surface pressure fluctuations, increasing power spectral density (PSD) amplitudes by up to an order of magnitude. In the far field, the appended SUBOFF generates sound pressure levels approximately 20 dB higher than the bare hull, with distinct dipole directivity patterns and peak noise levels (85.10 dB) observed on the central plane. Appendages also disrupt wake symmetry, introducing complex vortical structures such as horseshoe and necklace vortices. These findings demonstrate the critical influence of appendages on hydrodynamic and acoustic behavior, filling a gap in turbulence noise research for complex underwater geometries and providing a vital foundation for the noise reduction optimization of advanced underwater vehicles.

physics.flu-dyn↗

arXiv2025

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

Harmful fine-tuning attack poses serious safety concerns for large language models' fine-tuning-as-a-service. While existing defenses have been proposed to mitigate the issue, their performances are still far away from satisfactory, and the root cause of the problem has not been fully recovered. To this end, we in this paper show that harmful perturbation over the model weights could be a probable cause of alignment-broken. In order to attenuate the negative impact of harmful perturbation, we propose an alignment-stage solution, dubbed Booster. Technically, along with the original alignment loss, we append a loss regularizer in the alignment stage's optimization. The regularizer ensures that the model's harmful loss reduction after the simulated harmful perturbation is attenuated, thereby mitigating the subsequent fine-tuning risk. Empirical results show that Booster can effectively reduce the harmful score of the fine-tuned models while maintaining the performance of downstream tasks. Our code is available at https://github.com/git-disl/Booster.

cs.CL↗

arXiv2025

A Method to Decipher "Genome" from Interatomic Cohesion in the Exploration for a "Central Dogma" Replacement in Material Science

In the ball-stick model, interatomic cohesions are considered "sticks". But enormous details and features of the "sticks" are usually oversimplified as indexed quantities or equivocated as geometry characteristics. These indexed quantities or geometry characteristics not only limit the explanatory capability to a few chemical/physical aspects but also eliminate generativity for expected resemblance. And these limitations can be related to the information loss during the conversion. Herein, inspired by the central dogma, a framework is introduced to compact interatomic cohesions into a detailed residue-by-residue "genome" with matched encoding/decoding tools. The framework fuses the quantum mechanical aspects, auto feature extraction, nanostructures and/or simulations, and generative models. As a proof of concept, the realization introduced in this work adopted bosonic/fermionic features, an autoencoder with image recognition processes, Density Functional Theory simulations, and a thiolate-protected gold nanocluster dataset. After repetitive modeling, validating, and analysis based on 26,528 simulated interatomic images, the interatomic cohesion can be almost losslessly encoded into an 8-value-genome, and the genome encoder-decoder pair is also obtained. The model is then automatically extended into a generative model which converts any arbitrary 8-value-genome to a bond image.

cond-mat.mes-hall↗

arXiv2025

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation

Recent research shows that Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- models lose their safety alignment ability after fine-tuning on a few harmful samples. For risk mitigation, a guardrail is typically used to filter out harmful samples before fine-tuning. By designing a new red-teaming method, we in this paper show that purely relying on the moderation guardrail for data filtration is not reliable. Our proposed attack method, dubbed Virus, easily bypasses the guardrail moderation by slightly modifying the harmful data. Experimental results show that the harmful data optimized by Virus is not detectable by the guardrail with up to 100\% leakage ratio, and can simultaneously achieve superior attack performance. Finally, the key message we want to convey through this paper is that: \textbf{it is reckless to consider guardrail moderation as a clutch at straws towards harmful fine-tuning attack}, as it cannot solve the inherent safety issue of the pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus

cs.CR↗

arXiv2025

Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge

The effectiveness of automatic evaluation of generative models is typically measured by comparing the labels generated via automation with labels by humans using correlation metrics. However, metrics like Krippendorff's $α$ and Randolph's $κ$ were originally designed to measure the reliability of human labeling, thus make assumptions about typical human labeling behavior, and these assumptions may not be applicable to machine generated labels. In this paper, we show how *relying on a single aggregate correlation score* can obscure fundamental differences between human labels and those from automatic evaluation, including LLM-as-a-Judge. Specifically, we demonstrate that when the proportion of samples with variation or uncertainty in human assigned labels is relatively high, machine labels (generated by automatic evaluation methods) may superficially appear to have similar or better correlation with the human majority label compared to the human-to-human (HH) correlation. This can create the illusion that labels from automatic evaluation approximates the human majority label. However, as the proportion of samples with consistent human labels increases, the correlation between machine and human labels fall well below HH correlation. Based on these findings, we first propose stratifying data by human label uncertainty to provide a more robust analysis of automatic evaluation performance. Second, recognizing that uncertainty and variation are inherent in perception-based human evaluations, such as those involving attitudes or preferences, we introduce a new metric - binned Jensen-Shannon Divergence for perception for such scenarios to better measure the effectiveness of automatic evaluations. We present visualization techniques -- perception charts, to contextualize correlation measures appropriately. We have open-sourced at https://github.com/amazon-science/BeyondCorrelation.

cs.HC↗

arXiv2025

Construction of Simultaneously Good Polar Codes and Polar Lattices

In this work, we investigate the simultaneous goodness of polar codes and polar lattices. The simultaneous goodness of a lattice or a code means that it is optimal for both channel coding and source coding simultaneously. The existence of such kind of lattices was proven by using random lattice ensembles. Our work provides an explicit construction based on the polarization technique.

cs.IT↗

arXiv2025

On the quantization goodness of polar lattices

In this work, we prove that polar lattices, when tailored for lossy compression, are quantization-good in the sense that their normalized second moments approach $\frac{1}{2πe}$ as the dimension of lattices increases. It has been predicted by Zamir et al. \cite{ZamirQZ96} that the Entropy Coded Dithered Quantization (ECDQ) system using quantization-good lattices can achieve the rate-distortion bound of i.i.d. Gaussian sources. In our previous work \cite{LingQZ}, we established that polar lattices are indeed capable of attaining the same objective. It is reasonable to conjecture that polar lattices also demonstrate quantization goodness in the context of lossy compression. This study confirms this hypothesis.

cs.IT↗