arXiv subjects

Lang Gao

Publications and source records attributed to Lang Gao.

27 records · Page 2Linked to original sources

arXiv2025

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

The rise of Large Audio Language Models (LAMs) brings both potential and risks, as their audio outputs may contain harmful or unethical content. However, current research lacks a systematic, quantitative evaluation of LAM safety especially against jailbreak attacks, which are challenging due to the temporal and semantic nature of speech. To bridge this gap, we introduce AJailBench, the first benchmark specifically designed to evaluate jailbreak vulnerabilities in LAMs. We begin by constructing AJailBench-Base, a dataset of 1,495 adversarial audio prompts spanning 10 policy-violating categories, converted from textual jailbreak attacks using realistic text to speech synthesis. Using this dataset, we evaluate several state-of-the-art LAMs and reveal that none exhibit consistent robustness across attacks. To further strengthen jailbreak testing and simulate more realistic attack conditions, we propose a method to generate dynamic adversarial variants. Our Audio Perturbation Toolkit (APT) applies targeted distortions across time, frequency, and amplitude domains. To preserve the original jailbreak intent, we enforce a semantic consistency constraint and employ Bayesian optimization to efficiently search for perturbations that are both subtle and highly effective. This results in AJailBench-APT, an extended dataset of optimized adversarial audio samples. Our findings demonstrate that even small, semantically preserved perturbations can significantly reduce the safety performance of leading LAMs, underscoring the need for more robust and semantically aware defense mechanisms.

cs.SD↗

arXiv2025

Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs

Bias in Large Language Models (LLMs) significantly undermines their reliability and fairness. We focus on a common form of bias: when two reference concepts in the model's concept space, such as sentiment polarities (e.g., "positive" and "negative"), are asymmetrically correlated with a third, target concept, such as a reviewing aspect, the model exhibits unintended bias. For instance, the understanding of "food" should not skew toward any particular sentiment. Existing bias evaluation methods assess behavioral differences of LLMs by constructing labeled data for different social groups and measuring model responses across them, a process that requires substantial human effort and captures only a limited set of social concepts. To overcome these limitations, we propose BiasLens, a test-set-free bias analysis framework based on the structure of the model's vector space. BiasLens combines Concept Activation Vectors (CAVs) with Sparse Autoencoders (SAEs) to extract interpretable concept representations, and quantifies bias by measuring the variation in representational similarity between the target concept and each of the reference concepts. Even without labeled data, BiasLens shows strong agreement with traditional bias evaluation metrics (Spearman correlation r > 0.85). Moreover, BiasLens reveals forms of bias that are difficult to detect using existing methods. For example, in simulated clinical scenarios, a patient's insurance status can cause the LLM to produce biased diagnostic assessments. Overall, BiasLens offers a scalable, interpretable, and efficient paradigm for bias discovery, paving the way for improving fairness and transparency in LLMs.

cs.CL↗

arXiv2025

Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia

Human readers can efficiently comprehend scrambled words, a phenomenon known as Typoglycemia, primarily by relying on word form; if word form alone is insufficient, they further utilize contextual cues for interpretation. While advanced large language models (LLMs) exhibit similar abilities, the underlying mechanisms remain unclear. To investigate this, we conduct controlled experiments to analyze the roles of word form and contextual information in semantic reconstruction and examine LLM attention patterns. Specifically, we first propose SemRecScore, a reliable metric to quantify the degree of semantic reconstruction, and validate its effectiveness. Using this metric, we study how word form and contextual information influence LLMs' semantic reconstruction ability, identifying word form as the core factor in this process. Furthermore, we analyze how LLMs utilize word form and find that they rely on specialized attention heads to extract and process word form information, with this mechanism remaining stable across varying levels of word scrambling. This distinction between LLMs' fixed attention patterns primarily focused on word form and human readers' adaptive strategy in balancing word form and contextual information provides insights into enhancing LLM performance by incorporating human-like, context-aware mechanisms.

cs.CL↗

arXiv2024

VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models

Large Language Models (LLMs) have training corpora containing large amounts of program code, greatly improving the model's code comprehension and generation capabilities. However, sound comprehensive research on detecting program vulnerabilities, a more specific task related to code, and evaluating the performance of LLMs in this more specialized scenario is still lacking. To address common challenges in vulnerability analysis, our study introduces a new benchmark, VulDetectBench, specifically designed to assess the vulnerability detection capabilities of LLMs. The benchmark comprehensively evaluates LLM's ability to identify, classify, and locate vulnerabilities through five tasks of increasing difficulty. We evaluate the performance of 17 models (both open- and closed-source) and find that while existing models can achieve over 80% accuracy on tasks related to vulnerability identification and classification, they still fall short on specific, more detailed vulnerability analysis tasks, with less than 30% accuracy, making it difficult to provide valuable auxiliary information for professional vulnerability mining. Our benchmark effectively evaluates the capabilities of various LLMs at different levels in the specific task of vulnerability detection, providing a foundation for future research and improvements in this critical area of code security. VulDetectBench is publicly available at https://github.com/Sweetaroo/VulDetectBench.

cs.CR↗

arXiv2024

An electro-optically tunable arrayed waveguide grating fabricated on thin film lithium niobate

We design and fabricate an 8-channel thin film lithium niobate (TFLN) arrayed-waveguide grating (AWG) and demonstrate the electro-optical tunability of the device. The monolithically integrated microelectrodes are designed for waveguides phase modulation and wavelength tunning. Experiments show that the fabricated electro-optically controlled TFLN AWG has a channel spacing of 200 GHz and a wavelength tuning efficiency of 10 pm/V.

physics.optics↗

arXiv2024

Compact low-half-wave-voltage thin film lithium niobate electro-optic phase modulator fabricated by photolithography assisted chemo-mechanical etching

This paper presents a compact dual-arm thin film lithium niobate (TFLN) electro-optic phase modulator fabricated using the photolithography-assisted chemo-mechanical etching (PLACE) technique. The design of the device allows for complete utilization of the microwave electric field, doubling the modulation efficiency compared to single-arm modulators in theory. With a half-wave voltage of approximately 3 V and a modulation length of 1 cm, the device outperforms conventional phase modulators. Furthermore, the phase modulator exhibits low sensitivity to optical wavelengths in the range of 1510-1600 nm and offers a low insertion loss of 2.8 dB. The capability to generate multiple sideband signals for optical frequency comb applications is also demonstrated, producing 29 sideband signals at an input microwave power of 2 W.

physics.optics↗

arXiv2023

Thin Film Lithium Niobate Electro-optic Isolator Fabricated by photolithography assisted chemo-mechanical etching (PLACE)

We report a thin-film lithium niobate electro-optic isolator fabricated by photolithography-assisted chemo-mechanical etching in this work. The device demonstrates 39.50 dB isolation when subjected to a 24 GHz microwave of 25.5 dBm on its electrodes. The measured isolation remains consistently above 30 dB within the 1510 nm to 1600 nm wavelength range. The overall device insertion loss, specifically the fiber-to-fiber insert loss, has been measured to be 2.6 dB, which is attributed to our highly efficient spot size converter and the low propagation loss observed in the fabricated waveguides.

physics.optics↗

arXiv2023

On-chip coherent beam combination of waveguide amplifiers on Er$^{3+}$-doped thin film lithium niobate

We demonstrate on-chip coherent beam combination of two waveguide amplifiers on Er$^{3+}$-doped thin film lithium niobate (Er: TFLN) platform. Our device is built based on an electro-optic modulator fabricated on Er: TFLN. The output power of the coherently combined amplifiers is measured as high as 12.9 mW, surpassing that of previous single waveguide amplifiers based on Er$^{3+}$-doped thin film lithium niobate platform.

physics.optics↗

arXiv2022

High-production-rate fabrication of low-loss lithium niobate electro-optic modulators using photolithography assisted chemo-mechanical etching (PLACE)

Integrated thin-film lithium niobate (LN) electro-optic (EO) modulators of broad bandwidth, low insertion loss, low cost and high production rate are essential elements in contemporary inter-connection industries and disruptive applications. Here, we demonstrated the design and fabri-cation of a high performance thin-film LN EO modulator using photolithography assisted chemo-mechanical etching (PLACE) technology. Our device shows a 3-dB bandwidth over 50 GHz, along with a comparable low half wave voltage-length product of 2.16 Vcm. We obtain a fiber-to-fiber insertion loss of 2.6 dB.

physics.app-ph↗