arXiv subjects

Zhiqiang Gao

Publications and source records attributed to Zhiqiang Gao.

At least 37 records · Page 2Linked to original sources

arXiv2025

Escaping Optimization Stagnation: Taking Steps Beyond Task Arithmetic via Difference Vectors

Current methods for editing pre-trained models face significant challenges, primarily high computational costs and limited scalability. Task arithmetic has recently emerged as a promising solution, using simple arithmetic operations-addition and negation-based on task vectors which are the differences between fine-tuned and pre-trained model weights, to efficiently modify model behavior. However, the full potential of task arithmetic remains underexplored, primarily due to limited mechanisms for overcoming optimization stagnation. To address this challenge, we introduce the notion of difference vector, a generalized form of task vectors derived from the historical movements during optimization. Using difference vectors as directed perturbations, we propose the Difference Vector-based Anisotropic Scaling Iterative algorithm (DV-BASI) to enable a continuous optimization process for task arithmetic methods without relying on any additional modules or components. Notably, by leveraging escapability and directional advantages of difference vectors, the average performance on different tasks of the multi-task model merged by DV-BASI may even outperform models individually fine-tuned. Based on this observation, we extend the application of difference vectors to a feasible fine-tuning method for single-task models. On the practical side, DV-BASI allows expressive searching directions with few learnable parameters and forms a scalable framework. We also integrate DV-BASI with task arithmetic methods and advanced optimization techniques to achieve state-of-the-art performance on both supervised and unsupervised evaluation protocols.

cs.LG↗

arXiv2025

From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery

Artificial intelligence (AI) is reshaping scientific discovery, evolving from specialized computational tools into autonomous research partners. We position Agentic Science as a pivotal stage within the broader AI for Science paradigm, where AI systems progress from partial assistance to full scientific agency. Enabled by large language models (LLMs), multimodal systems, and integrated research platforms, agentic AI shows capabilities in hypothesis generation, experimental design, execution, analysis, and iterative refinement -- behaviors once regarded as uniquely human. This survey provides a domain-oriented review of autonomous scientific discovery across life sciences, chemistry, materials science, and physics. We unify three previously fragmented perspectives -- process-oriented, autonomy-oriented, and mechanism-oriented -- through a comprehensive framework that connects foundational capabilities, core processes, and domain-specific realizations. Building on this framework, we (i) trace the evolution of AI for Science, (ii) identify five core capabilities underpinning scientific agency, (iii) model discovery as a dynamic four-stage workflow, (iv) review applications across the above domains, and (v) synthesize key challenges and future opportunities. This work establishes a domain-oriented synthesis of autonomous scientific discovery and positions Agentic Science as a structured paradigm for advancing AI-driven research.

cs.LG↗

arXiv2025

SafeBehavior: Simulating Human-Like Multistage Reasoning to Mitigate Jailbreak Attacks in Large Language Models

Large Language Models (LLMs) have achieved impressive performance across diverse natural language processing tasks, but their growing power also amplifies potential risks such as jailbreak attacks that circumvent built-in safety mechanisms. Existing defenses including input paraphrasing, multi step evaluation, and safety expert models often suffer from high computational costs, limited generalization, or rigid workflows that fail to detect subtle malicious intent embedded in complex contexts. Inspired by cognitive science findings on human decision making, we propose SafeBehavior, a novel hierarchical jailbreak defense mechanism that simulates the adaptive multistage reasoning process of humans. SafeBehavior decomposes safety evaluation into three stages: intention inference to detect obvious input risks, self introspection to assess generated responses and assign confidence based judgments, and self revision to adaptively rewrite uncertain outputs while preserving user intent and enforcing safety constraints. We extensively evaluate SafeBehavior against five representative jailbreak attack types including optimization based, contextual manipulation, and prompt based attacks and compare it with seven state of the art defense baselines. Experimental results show that SafeBehavior significantly improves robustness and adaptability across diverse threat scenarios, offering an efficient and human inspired approach to safeguarding LLMs against jailbreak attempts.

cs.AI↗

arXiv2025

Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing

Peptide sequencing-the process of identifying amino acid sequences from mass spectrometry data-is a fundamental task in proteomics. Non-Autoregressive Transformers (NATs) have proven highly effective for this task, outperforming traditional methods. Unlike autoregressive models, which generate tokens sequentially, NATs predict all positions simultaneously, leveraging bidirectional context through unmasked self-attention. However, existing NAT approaches often rely on Connectionist Temporal Classification (CTC) loss, which presents significant optimization challenges due to CTC's complexity and increases the risk of training failures. To address these issues, we propose an improved non-autoregressive peptide sequencing model that incorporates a structured protein sequence curriculum learning strategy. This approach adjusts protein's learning difficulty based on the model's estimated protein generational capabilities through a sampling process, progressively learning peptide generation from simple to complex sequences. Additionally, we introduce a self-refining inference-time module that iteratively enhances predictions using learned NAT token embeddings, improving sequence accuracy at a fine-grained level. Our curriculum learning strategy reduces NAT training failures frequency by more than 90% based on sampled training over various data distributions. Evaluations on nine benchmark species demonstrate that our approach outperforms all previous methods across multiple metrics and species.

q-bio.BM↗

arXiv2025

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing

De novo peptide sequencing is a critical task in proteomics. However, the performance of current deep learning-based methods is limited by the inherent complexity of mass spectrometry data and the heterogeneous distribution of noise signals, leading to data-specific biases. We present RankNovo, the first deep reranking framework that enhances de novo peptide sequencing by leveraging the complementary strengths of multiple sequencing models. RankNovo employs a list-wise reranking approach, modeling candidate peptides as multiple sequence alignments and utilizing axial attention to extract informative features across candidates. Additionally, we introduce two new metrics, PMD (Peptide Mass Deviation) and RMD (residual Mass Deviation), which offer delicate supervision by quantifying mass differences between peptides at both the sequence and residue levels. Extensive experiments demonstrate that RankNovo not only surpasses its base models used to generate training candidates for reranking pre-training, but also sets a new state-of-the-art benchmark. Moreover, RankNovo exhibits strong zero-shot generalization to unseen models whose generations were not exposed during training, highlighting its robustness and potential as a universal reranking framework for peptide sequencing. Our work presents a novel reranking strategy that fundamentally challenges existing single-model paradigms and advances the frontier of accurate de novo sequencing. Our source code is provided on GitHub.

cs.LG↗

arXiv2025

PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Models

Large language models (LLMs) have rapidly advanced and are increasingly capable of tackling complex scientific problems, including those in physics. Despite this progress, current LLMs often fail to emulate the concise, principle-based reasoning characteristic of human experts, instead generating lengthy and opaque solutions. This discrepancy highlights a crucial gap in their ability to apply core physical principles for efficient and interpretable problem solving. To systematically investigate this limitation, we introduce PhySense, a novel principle-based physics reasoning benchmark designed to be easily solvable by experts using guiding principles, yet deceptively difficult for LLMs without principle-first reasoning. Our evaluation across multiple state-of-the-art LLMs and prompt types reveals a consistent failure to align with expert-like reasoning paths, providing insights for developing AI systems with efficient, robust and interpretable principle-based scientific reasoning.

cs.LG↗

arXiv2025

Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation

Deep learning has revolutionized medical image segmentation, yet its full potential remains constrained by the paucity of annotated datasets. While diffusion models have emerged as a promising approach for generating synthetic image-mask pairs to augment these datasets, they paradoxically suffer from the same data scarcity challenges they aim to mitigate. Traditional mask-only models frequently yield low-fidelity images due to their inability to adequately capture morphological intricacies, which can critically compromise the robustness and reliability of segmentation models. To alleviate this limitation, we introduce Siamese-Diffusion, a novel dual-component model comprising Mask-Diffusion and Image-Diffusion. During training, a Noise Consistency Loss is introduced between these components to enhance the morphological fidelity of Mask-Diffusion in the parameter space. During sampling, only Mask-Diffusion is used, ensuring diversity and scalability. Comprehensive experiments demonstrate the superiority of our method. Siamese-Diffusion boosts SANet's mDice and mIoU by 3.6% and 4.4% on the Polyps, while UNet improves by 1.52% and 1.64% on the ISIC2018. Code is available at GitHub.

cs.CV↗

arXiv2025

Magnetic ground state discrimination of a Polyradical Nanog-raphene using Nickelocene-Functionalized Tips

Molecular magnets are a promising class of materials with exciting properties and applications. However, a profound understanding and application of such materials depends on the accurate detection of their electronic and magnetic properties. Despite the availability of experimental techniques that can sense the magnetic signal, the exact determination of the spin ground states and spatial distribution of exchange interaction of strongly correlated single-molecule magnets remains challenging. Here, we demonstrate that scanning probe microscopy with a nickelocene-functionalized probe can distinguish between nearly degenerate multireference ground states of single-molecule π-magnets and map their spatial distribution of the exchange interaction. This method expands the already outstanding imaging capabilities of scanning probe microscopy for characterizing the chemical and electronic structures of individual molecules, paving the way for the study of strongly correlated molecular magnets with unprecedented spatial resolution.

cond-mat.mtrl-sci↗

arXiv2025

Discrete Wavelet Transform-Based Capsule Network for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a crucial technique for remote sensing to build large-scale earth monitoring systems. HSI contains much more information than traditional visual images for identifying the categories of land covers. One recent feasible solution for HSI is to leverage CapsNets for capturing spectral-spatial information. However, these methods require high computational requirements due to the full connection architecture between stacked capsule layers. To solve this problem, a DWT-CapsNet is proposed to identify partial but important connections in CapsNet for a effective and efficient HSI classification. Specifically, we integrate a tailored attention mechanism into a Discrete Wavelet Transform (DWT)-based downsampling layer, alleviating the information loss problem of conventional downsampling operation in feature extractors. Moreover, we propose a novel multi-scale routing algorithm that prunes a large proportion of connections in CapsNet. A capsule pyramid fusion mechanism is designed to aggregate the spectral-spatial relationships in multiple levels of granularity, and then a self-attention mechanism is further conducted in a partially and locally connected architecture to emphasize the meaningful relationships. As shown in the experimental results, our method achieves state-of-the-art accuracy while keeping lower computational demand regarding running time, flops, and the number of parameters, rendering it an appealing choice for practical implementation in HSI classification.

cs.CV↗

arXiv2024

NatLan: Native Language Prompting Facilitates Knowledge Elicitation Through Language Trigger Provision and Domain Trigger Retention

Multilingual large language models (MLLMs) do not perform as well when answering questions in non-dominant languages as they do in their dominant languages. Although existing translate-then-answer methods alleviate this issue, the mechanisms behind their effectiveness remain unclear. In this study, we analogize the dominant language of MLLMs to the native language of humans and use two human cognitive features: the Language Trigger (LT) and the Domain Trigger (DT), to interpret the mechanisms behind translate-then-answer methods. This reveals that while sufficient LTs are provided by these methods, there remains a deficiency in DT retention. To mitigate this issue, we propose Native Language Prompting (NatLan), employing a Multi-MLLM collaboration strategy and introducing an additional role-enhanced domain-specific MLLM with stronger multilingual understanding capabilities as the translator. Across five language QA benchmarks, NatLan achieves up to a 31.28% improvement in accuracy and, compared to existing state-of-the-art methods, provides comparable or greater retention of DTs in up to 87% of cases. Our code is available at https://github.com/AnonyNLP/NatLan.

cs.CL↗

arXiv2024

Disturbance Rejection-Guarded Learning for Vibration Suppression of Two-Inertia Systems

Model uncertainty presents significant challenges in vibration suppression of multi-inertia systems, as these systems often rely on inaccurate nominal mathematical models due to system identification errors or unmodeled dynamics. An observer, such as an extended state observer (ESO), can estimate the discrepancy between the inaccurate nominal model and the true model, thus improving control performance via disturbance rejection. The conventional observer design is memoryless in the sense that once its estimated disturbance is obtained and sent to the controller, the datum is discarded. In this research, we propose a seamless integration of ESO and machine learning. On one hand, the machine learning model attempts to model the disturbance. With the assistance of prior information about the disturbance, the observer is expected to achieve faster convergence in disturbance estimation. On the other hand, machine learning benefits from an additional assurance layer provided by the ESO, as any imperfections in the machine learning model can be compensated for by the ESO. We validated the effectiveness of this novel learning-for-control paradigm through simulation and physical tests on two-inertial motion control systems used for vibration studies.

eess.SY↗

arXiv2023

ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing

De novo peptide sequencing from mass spectrometry (MS) data is a critical task in proteomics research. Traditional de novo algorithms have encountered a bottleneck in accuracy due to the inherent complexity of proteomics data. While deep learning-based methods have shown progress, they reduce the problem to a translation task, potentially overlooking critical nuances between spectra and peptides. In our research, we present ContraNovo, a pioneering algorithm that leverages contrastive learning to extract the relationship between spectra and peptides and incorporates the mass information into peptide decoding, aiming to address these intricacies more efficiently. Through rigorous evaluations on two benchmark datasets, ContraNovo consistently outshines contemporary state-of-the-art solutions, underscoring its promising potential in enhancing de novo peptide sequencing. The source code is available at https://github.com/BEAM-Labs/ContraNovo.

q-bio.QM↗

arXiv2023

Wasserstein-$1$ distance and nonuniform Berry-Esseen bound for a supercritical branching process in a random environment

Let $ (Z_{n})_{n\geq 0} $ be a supercritical branching process in an independent and identically distributed random environment. We establish an optimal convergence rate in the Wasserstein-$1$ distance for the process $ (Z_{n})_{n\geq 0} $, which completes a result of Grama et al. [Stochastic Process. Appl., 127(4), 1255-1281, 2017]. Moreover, an exponential nonuniform Berry-Esseen bound is also given. At last, some applications of the main results to the confidence interval estimation for the criticality parameter and the population size $Z_n$ are discussed.

math.PR↗

arXiv2023

Robust Control Barrier Functions for Safe Control Under Uncertainty Using Extended State Observer and Output Measurement

Control barrier functions-based quadratic programming (CBF-QP) is gaining popularity as an effective controller synthesis tool for safe control. However, the provable safety is established on an accurate dynamic model and access to all states. To address such a limitation, this paper proposes a novel design combining an extended state observer (ESO) with a CBF for safe control of a system with model uncertainty and external disturbances only using output measurement. Our approach provides a less conservative estimation error bound than other disturbance observer-based CBFs. Moreover, only output measurements are needed to estimate the disturbances instead of access to the full state. The bounds of state estimation error and disturbance estimation error are obtained in a unified manner and then used for robust safe control under uncertainty. We validate our approach's efficacy in simulations of an adaptive cruise control system and a Segway self-balancing scooter.

eess.SY↗

arXiv2023

A General Model-Based Extended State Observer with Built-In Zero Dynamics

A general model-based extended state observer (GMB-ESO) is proposed for single-input single-output linear time-invariant systems with a given state space model, where the total disturbance, a lump sum of model uncertainties and external disturbances, is defined as an extended state in the same manner as in the original formulation of ESO. The conditions for the existence of such an observer, however, are shown for the first time as 1) the original plant is observable; and 2) there is no invariant zero between the plant output and the total disturbance. Then, the finite-step convergence and error characteristics of GMB-ESO are shown by exploiting its inherent connection to the well-known unknown input observer (UIO). Furthermore, it is shown that, with the relative degree of the plant greater than one and the observer eigenvalues all placed at the origin, GMB-ESO produces the identical disturbance estimation as that of UIO. Finally, an improved GMB-ESO with built-in zero dynamics is proposed for those plants with zero dynamics, which is a problem that has not been addressed in all existing ESO designs.

eess.SY↗

arXiv2022

A Higher-Order Semantic Dependency Parser

Higher-order features bring significant accuracy gains in semantic dependency parsing. However, modeling higher-order features with exact inference is NP-hard. Graph neural networks (GNNs) have been demonstrated to be an effective tool for solving NP-hard problems with approximate inference in many graph learning tasks. Inspired by the success of GNNs, we investigate building a higher-order semantic dependency parser by applying GNNs. Instead of explicitly extracting higher-order features from intermediate parsing graphs, GNNs aggregate higher-order information concisely by stacking multiple GNN layers. Experimental results show that our model outperforms the previous state-of-the-art parser on the SemEval 2015 Task 18 English datasets.

cs.CL↗

arXiv2019

An AI-based, Multi-stage detection system of banking botnets

Banking Trojans, botnets are primary drivers of financially-motivated cybercrime. In this paper, we first analyzed how an APT-based banking botnet works step by step through the whole lifecycle. Specifically, we present a multi-stage system that detects malicious banking botnet activities which potentially target the organizations. The system leverages Cyber Data Lake as well as multiple artificial intelligence techniques at different stages. The evaluation results using public datasets showed that Deep Learning based detections were highly successful compared with baseline models.

cs.CR↗

arXiv2015

Exact convergence rates in central limit theorems for a branching random walk with a random environment in time

Chen [Ann. Appl. Probab. {\bf 11} (2001), 1242--1262] derived exact convergence rates in a central limit theorem and a local limit theorem for a supercritical branching Wiener process.We extend Chen's results to a branching random walk under weaker moment conditions. For the branching Wiener process, our results sharpen Chen's by relaxing the second moment condition used by Chen to a moment condition of the form $ \E X (\ln^+X )^{1+λ}< \infty$. In the rate functions that we find for a branching random walk, we figure out some new terms which didn't appear in Chen's work.The results are established in the more general framework, i.e. for a branching random walk with a random environment in time.The lack of the second moment condition for the offspring distribution and the fact that the exponential moment does not exist necessarily for the displacements make the proof delicate; the difficulty is overcome by a careful analysis of martingale convergence using a truncating argument. The analysis is significantly more awkward due to the appearance of the random environment.

math.PR↗