arXiv subjects

Bin Shi

Publications and source records attributed to Bin Shi.

At least 19 recordsLinked to original sources

arXiv2026

Cross-Relational Preference Learning for Better LLM Instruction Following

Large Language Models (LLMs) still exhibit limited capability in following complex instructions. While existing approaches often rely on preference learning to enhance this ability, they typically overlook the relationships between the permissible response spaces of different instructions, which restricts a model to align with subtle and diverse constraint variations. To address this, we propose Cross-Relational Preference Learning (CRPL), a novel framework for constructing preference data that explicitly models inter-instruction relationships through two key techniques: Cross-Relationship Perturbation and Cross-Region Pair Sampling. This enables the generation of more diverse preference data that captures a wide spectrum of constraint variations. Additionally, we introduce an atomic constraint-based verification mechanism to rigorously assess response satisfaction, ensuring high-quality preference pair construction. Extensive experiments across multiple preference learning methods (e.g., DPO, KTO), LLM backbones and four instruction-following benchmarks demonstrate that our approach achieves substantial improvements over prior baselines and exhibits strong generalization.

cs.AI↗

arXiv2026

The Most Dangerous Seed: Nonlinear Optimal Perturbations in Rayleigh-Taylor Instability

Long-term instabilities in astrophysical fluids are inherently nonlinear, where even small-amplitude perturbations can trigger dramatic instability. However, owing to the complex interactions among non-normal modes, the perturbation structures responsible for the greatest growth remain poorly understood. In this paper, we employ the nonlinear optimization method known as the conditional nonlinear optimal perturbation (CNOP) to identify the most dangerous initial velocity perturbation, i.e., the perturbation that maximizes the kinetic energy growth in the two-dimensional compressible Rayleigh-Taylor instability in astrophysical hydrodynamics. Compared with random perturbations, the optimal perturbation forms a coherent wave-packet structure localized around the density interface. We investigate its dependence on spatial resolution and optimization time horizon through two sets of numerical experiments. For a fixed optimization time, increasing the spatial resolution produces a more sharply localized wave packet, whereas for a fixed spatial resolution, increasing the optimization time causes the wave packet to become progressively more dispersed. Furthermore, we analyze the optimal perturbations in Fourier space using the fast Fourier transform (FFT), which provides a clearer characterization of the spectral distribution. Higher-resolution simulations concentrate most of the perturbation energy into only a few dominant modes, while longer optimization times distribute the energy over a broader range of modes. These results indicate that short optimization time horizons involve relatively weak modal interactions and remain closer to the linear regime, whereas longer optimization times enhance nonlinear modal interactions, broaden the spectral distribution, and reduce the predictive capability of linear stability theory.

astro-ph.GA↗

arXiv2026

Modern Theory of Gradient-Based Optimization

In this review, we offer a comprehensive survey of emerging techniques in gradient-based optimization, with a particular emphasis on the interplay between ordinary differential equation (ODE) perspectives and their extensions into discrete Lyapunov analysis. We begin by examining the acceleration mechanisms underlying Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) and Polyak's heavy-ball method, identifying the gradient-correction term as the primary driver of acceleration. This mechanistic insight is substantiated through high-resolution ODE modeling and the systematic construction of Lyapunov functions. We then synthesize recent advancements in convex optimization regarding NAG and its proximal generalization, the fast iterative shrinkage-thresholding algorithm (FISTA). Key topics include the accelerated convergence of gradient norms, underdamped acceleration, linear convergence under strong convexity, and novel Lyapunov frameworks for establishing convergence and monotonicity properties of generalized accelerated methods. Furthermore, we demonstrate how these ODE approximations and Lyapunov techniques can be extended to provide a unified framework for analyzing advanced optimization algorithms, including the alternating direction method of multipliers (ADMM), the primal-dual hybrid gradient (PDHG) method, and their respective accelerated variants. Finally, we discuss recent progress in minimax optimization and outline future directions for extending Lyapunov-based analysis to saddle-point problems.

math.OC↗

arXiv2026

Accelerated Implicit GDA Schemes: Theoretical Guarantees and Application to Proximal Augmented Lagrangian Methods

Convex optimization problems with linear equality constraints arise ubiquitously in scientific computing, machine learning, and control theory. Classical Krylov methods are effective but rely on problem-specific preconditioners and high memory. Conversely, gradient-based methods like the augmented Lagrangian method (ALM) avoid these issues yet suffer from slow outer iterations. Developing accelerated outer-iteration schemes, therefore, remains a critical research objective. In this study, we demonstrate that incorporating a proximal operation into the augmented Lagrangian framework yields the proximal ALM, where the outer iteration is equivalent to an implicit gradient descent-ascent (GDA) scheme. We further establish that this equivalence extends naturally to the setting of variable step sizes. Through Lyapunov analysis, we show that the underlying potential function must be shifted from the conventional objective gap to a variational inequality measure, signaling a shift in perspective from pure convex optimization to minimax optimization. Motivated by these observations, we first develop an implicit GDA scheme with variable step sizes based on a continuous-time ODE framework, which achieves an $o(1/k)$ last-iterate convergence rate for both the primal-dual objective gap and the gradient norm. Building upon a second-order ODE framework, we then propose a family of Nesterov-type implicit GDA schemes parameterized by $r \geq 0$, which achieves an $o(1/k^{r+1})$ last-iterate convergence rate for the primal-dual objective gap. Furthermore, specializing the second-order ODE formulation to the case $r=0$, we derive a corresponding explicit GDA scheme and prove an $o(1/k)$ last-iterate convergence rate for the primal-dual objective gap. Finally, we present several numerical experiments to validate these theoretical results and demonstrate the effectiveness of the proposed methods.

math.OC↗

arXiv2026

Beyond Soft Masks: Hard-Perturbation Mixup Explainer for Robust GNN Explainability

Graph Neural Networks (GNNs) have demonstrated remarkable performance across a range of applications involving graph-structured data, particularly in high-stakes domains. However, the opaque nature of their decision-making processes limits their trustworthiness and broader adoption. Existing post-hoc explanation methods aim to improve explainability by identifying subgraphs that influence GNN predictions and adopt mixup strategies to alleviate the out-of-distribution (OOD) issue caused by using subgraphs for prediction. Yet, these approaches typically rely on soft masks, which are inherently unable to fully eliminate label-irrelevant information, allowing redundant structures to leak into the mixup process and hindering the resolution of the OOD problem, thereby degrading explanation fidelity. In this work, we propose HPME, a Hard-Perturbation Mixup Explanation framework grounded in a generalized Graph Information Bottleneck, which leverages graph pooling to extract discrete explanatory subgraphs and to yield an information-capacity bound to thoroughly compress label-irrelevant components. Furthermore, we introduce a novel mixup strategy built upon structure-level replacement, generating in-distribution explanations to effectively mitigate the distribution shift. Extensive experiments on diverse tasks demonstrate that HPME achieves state-of-the-art performance in generating robust and interpretable explanations across both synthetic and real-world datasets.

cs.LG↗

arXiv2026

Generalist Graph Anomaly Detection via Prototype-Based Distillation

Driven by the pressing demand for graph anomaly detection (GAD) in high-stakes domains, the generalist GAD paradigm, which trains a single detector transferable across new graphs, has recently gained growing attention. However, existing methods often rely on scarce and costly annotations for training and sometimes even require few-shot support at inference, which limits their robustness to diverse and unseen anomaly patterns. To address this limitation, we introduce ProMoS, the first unsupervised generalist GAD framework, which detects anomalies by modeling the abundant normality in unlabeled data. ProMoS adopts a knowledge-distillation paradigm to distill normality priors from a frozen self-supervised graph neural network (GNN) teacher to a mixture-of-students model with shared global and lightweight personalized branches, enabling efficient and expressive normality modeling without learning from scratch. We further propose prototype-guided soft-label distillation to align teacher and student in a shared prototype space, enhancing cross-graph generalizability. During inference, ProMoS performs zero-shot anomaly detection on unseen graphs via distillation bias and prototype geometric deviation. Extensive experiments show the effectiveness and efficiency of ProMoS, charting a practical path toward label-free, zero-shot generalist GAD.

cs.LG↗

arXiv2026

TED: Related Party Transaction guided Tax Evasion Detection on Heterogeneous Graph

Tax evasion causes severe losses of government revenues and disturbs the economic order of fair competition. To help alleviate this problem, the latest tax evasion detection solutions utilize expert knowledge to extract features and then train classifiers to determine whether a company is suspected of tax evasion. However, existing solutions mainly focus on the statistical features of the company, but fail to exploit the rich interactive information in tax scenarios, which affect the detection performance. In this paper, we first model the tax scenario as a heterogeneous graph and study the tax evasion detection problem under the heterogeneous graph model. To improve the performance of tax evasion detection, a novel graph neural network model is proposed to extract the comprehensive information of heterogeneous graphs. Specifically, we use heterogeneous and complex related party transaction groups to filter low-level noise information. Moreover, a hierarchical attention mechanism is designed to capture the deeper structure and semantic information hidden in the related party transaction group. We apply our method to the real risk management system of the tax bureau, and evaluate it on two human-labeled real-world tax datasets. The results demonstrate that our method significantly outperforms the state-of-the-art in the tax evasion detection task.

cs.LG↗

arXiv2026

Learning Dynamic Graph Representations through Timespan View Contrasts

The rich information underlying graphs has inspired further investigation of unsupervised graph representation. Existing studies mainly depend on node features and topological properties within static graphs to create self-supervised signals, neglecting the temporal components carried by real-world graph data, such as timestamps of edges. To overcome this limitation, this paper explores how to model temporal evolution on dynamic graphs elegantly. Specifically, we introduce a new inductive bias, namely temporal translation invariance, which illustrates the tendency of the identical node to keep similar labels across different timespans. Based on this assumption, we develop a dynamic graph representation framework CLDG that encourages the node to maintain locally consistent temporal translation invariance through contrastive learning on different timespans. Except for standard CLDG which only considers explicit topological links, our further proposed CLDG++ additionally employs graph diffusion to uncover global contextual correlations between nodes, and designs a multi-scale contrastive learning objective composed of local-local, local-global, and global-global contrasts to enhance representation capabilities. Interestingly, by measuring the consistency between different timespans to shape anomaly indicators, CLDG and CLDG++ are seamlessly integrated with the task of spotting anomalies on dynamic graphs, which has broad applications in many high-impact domains, such as finance, cybersecurity, and healthcare. Experiments demonstrate that CLDG and CLDG++ both exhibit desirable performance in downstream tasks including node classification and dynamic graph anomaly detection. Moreover, CLDG significantly reduces time and space complexity by implicitly exploiting temporal cues instead of complicated sequence models.

cs.LG↗

arXiv2026

Numerical Construction of Elliptic Lower-Dimensional Quasi-Periodic Solutions with a Priori Bound

A numerical framework for constructing full-dimensional quasi-periodic solutions in nearly integrable systems was recently developed by Fu and Shi[2026]. Based on an alternating scheme, this approach effectively overcomes the secular drift in angle variables, a fundamental limitation of symplectic integrators. However, in many applications, such as the restricted three-body problem, lower-dimensional quasi-periodic solutions hold greater significance. The construction of these solutions is considerably more challenging due to the presence of normal frequencies, leading to intricate resonance phenomena. Beyond the subspace resonance, one must also account for the first and second Melnikov conditions to eliminate small divisors. In this study, we extend the proposed alternating numerical scheme to compute the elliptic lower-dimensional quasi-periodic solutions. Numerical experiments are presented for the H\'{e}non-Heiles model and the Fermi--Pasta--Ulam (FPU) model, demonstrating the effectiveness of the proposed method. Furthermore, we emphasize that the perturbation is not merely a polynomial with real coefficients but is a real-valued function. As a result, the associated perturbation operator exhibits Gevrey decay without possessing a Hankel structure. Meanwhile, we further simplify the multi-scale analysis by exploiting the resolvent identity, showing that the global inverse can be expressed linearly in terms of local inverses via the gluing procedure. This representation reveals a regime-dependent interaction structure: weak interactions dominate at short range, while strong interactions emerge at long range. This balance ensures that the Gevrey decay of the inverse remains uniformly controlled. Moreover, within this linear representation, the inversion conditions provide a clearer characterization of the localization properties.

math.NA↗

arXiv2026

Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning

Fine-tuning large language models for domain-specific tasks such as medical text summarization demands substantial computational resources. Parameter-efficient fine-tuning (PEFT) methods offer promising alternatives by updating only a small fraction of parameters. This paper compares three adaptation approaches-Low-Rank Adaptation (LoRA), Prompt Tuning, and Full Fine-Tuning-across the Flan-T5 model family on the PubMed medical summarization dataset. Through experiments with multiple random seeds, we demonstrate that LoRA consistently outperforms full fine-tuning, achieving 43.52 +/- 0.18 ROUGE-1 on Flan-T5-Large with only 0.6% trainable parameters compared to 40.67 +/- 0.21 for full fine-tuning. Sensitivity analyses examine the impact of LoRA rank and prompt token count. Our findings suggest the low-rank constraint provides beneficial regularization, challenging assumptions about the necessity of full parameter updates. Code is available at https://github.com/eracoding/llm-medical-summarization

cs.CL↗

arXiv2026

Hide and Find: A Distributed Adversarial Attack on Federated Graph Learning

Federated Graph Learning (FedGL) is vulnerable to malicious attacks, yet developing a truly effective and stealthy attack method remains a significant challenge. Existing attack methods suffer from low attack success rates, high computational costs, and are easily identified and smoothed by defense algorithms. To address these challenges, we propose \textbf{FedShift}, a novel two-stage "Hide and Find" distributed adversarial attack. In the first stage, before FedGL begins, we inject a learnable and hidden "shifter" into part of the training data, which subtly pushes poisoned graph representations toward a target class's decision boundary without crossing it, ensuring attack stealthiness during training. In the second stage, after FedGL is complete, we leverage the global model information and use the hidden shifter as an optimization starting point to efficiently find the adversarial perturbations. During the final attack, we aggregate these perturbations from multiple malicious clients to form the final effective adversarial sample and trigger the attack. Extensive experiments on six large-scale datasets demonstrate that our method achieves the highest attack effectiveness compared to existing advanced attack methods. In particular, our attack can effectively evade 3 mainstream robust federated learning defense algorithms and converges with a time cost reduction of over 90\%, highlighting its exceptional stealthiness, robustness, and efficiency.

cs.LG↗

arXiv2026

Mitigating Instance Entanglement in Instance-Dependent Partial Label Learning

Partial label learning is a prominent weakly supervised classification task, where each training instance is ambiguously labeled with a set of candidate labels. In real-world scenarios, candidate labels are often influenced by instance features, leading to the emergence of instance-dependent PLL (ID-PLL), a setting that more accurately reflects this relationship. A significant challenge in ID-PLL is instance entanglement, where instances from similar classes share overlapping features and candidate labels, resulting in increased class confusion. To address this issue, we propose a novel Class-specific Augmentation based Disentanglement (CAD) framework, which tackles instance entanglement by both intra- and inter-class regulations. For intra-class regulation, CAD amplifies class-specific features to generate class-wise augmentations and aligns same-class augmentations across instances. For inter-class regulation, CAD introduces a weighted penalty loss function that applies stronger penalties to more ambiguous labels, encouraging larger inter-class distances. By jointly applying intra- and inter-class regulations, CAD improves the clarity of class boundaries and reduces class confusion caused by entanglement. Extensive experimental results demonstrate the effectiveness of CAD in mitigating the entanglement problem and enhancing ID-PLL performance. The code is available at https://github.com/RyanZhaoIc/CAD.git.

cs.CV↗

arXiv2026

Numerical Construction of Quasi-Periodic Solutions Beyond Symplectic Integrators

Symplectic integrators are the established standard for long-term simulations of nearly-integrable Hamiltonian systems due to their preservation of geometric structures. However, they suffer from an inherent limitation: secular phase-shift errors. While the qualitative ''shape'' of invariant tori is preserved, the numerical solution gradually drifts along the torus, leading to a phase-lag accumulation that degrades long-term positional accuracy. Inspired by the Craig-Wayne-Bourgain (CWB) scheme, originally developed as an analytical tool for infinite-dimensional systems, we introduce a numerical operator that incorporates frequency updates into a dimension-enlarged Newton iteration to compute quasi-periodic solutions. Unlike conventional time-stepping integrators, our alternating numerical procedure eliminates phase-lag accumulation by directly solving for instantaneous positions and phase angles. Theoretically, provided sufficient computational resources, the phase error can be reduced arbitrarily, remaining independent of the total integration time. Our algorithm translates the Nash-Moser iteration into a practical numerical framework, marking a significant departure from traditional Kolmogorov-Arnold-Moser (KAM) theory. While KAM provides rigorous existence proofs, its requirement for global Diophantine conditions and the total exclusion of resonant sets render it numerically inaccessible. By employing a ''step-by-step'' exclusion process and incrementally enlarging the dimension, our algorithm resolves irrationality conditions locally. This approach demonstrates that the ''numerical irrationality problem'' is not an intrinsic barrier to computation, offering a constructive, executable alternative to the non-executable nature of global KAM-based methods.

math.NA↗

arXiv2025

Exploring the Power of Diffusion Large Language Models for Software Engineering: An Empirical Investigation

Autoregressive Large Language Models (AR-LLMs) are widely used in software engineering (SE) but face limitations in processing code structure information and suffer from high inference latency. Diffusion LLMs (DLLMs) offer a promising alternative with global bidirectional encoding and decoupled generation steps. This work presents the first comprehensive evaluation of DLLMs across the software development lifecycle, including code generation, defect detection, and program repair. On a large-scale benchmark of 52,937 tasks, 7Bparameter DLLMs outperform AR-LLMs with a 30% average accuracy improvement achieving a 113% gain on cross-file repair, while maintaining superior efficiency and reduced latency. Our results establish DLLMs as a superior paradigm for SE tasks.

cs.SE↗

arXiv2025

Court of LLMs: Evidence-Augmented Generation via Multi-LLM Collaboration for Text-Attributed Graph Anomaly Detection

The natural combination of intricate topological structures and rich textual information in text-attributed graphs (TAGs) opens up a novel perspective for graph anomaly detection (GAD). However, existing GAD methods primarily focus on designing complex optimization objectives within the graph domain, overlooking the complementary value of the textual modality, whose features are often encoded by shallow embedding techniques, such as bag-of-words or skip-gram, so that semantic context related to anomalies may be missed. To unleash the enormous potential of textual modality, large language models (LLMs) have emerged as promising alternatives due to their strong semantic understanding and reasoning capabilities. Nevertheless, their application to TAG anomaly detection remains nascent, and they struggle to encode high-order structural information inherent in graphs due to input length constraints. For high-quality anomaly detection in TAGs, we propose CoLL, a novel framework that combines LLMs and graph neural networks (GNNs) to leverage their complementary strengths. CoLL employs multi-LLM collaboration for evidence-augmented generation to capture anomaly-relevant contexts while delivering human-readable rationales for detected anomalies. Moreover, CoLL integrates a GNN equipped with a gating mechanism to adaptively fuse textual features with evidence while preserving high-order topological information. Extensive experiments demonstrate the superiority of CoLL, achieving an average improvement of 13.37% in AP. This study opens a new avenue for incorporating LLMs in advancing GAD.

cs.LG↗

arXiv2025

Text-Attributed Graph Anomaly Detection via Multi-Scale Cross- and Uni-Modal Contrastive Learning

The widespread application of graph data in various high-risk scenarios has increased attention to graph anomaly detection (GAD). Faced with real-world graphs that often carry node descriptions in the form of raw text sequences, termed text-attributed graphs (TAGs), existing graph anomaly detection pipelines typically involve shallow embedding techniques to encode such textual information into features, and then rely on complex self-supervised tasks within the graph domain to detect anomalies. However, this text encoding process is separated from the anomaly detection training objective in the graph domain, making it difficult to ensure that the extracted textual features focus on GAD-relevant information, seriously constraining the detection capability. How to seamlessly integrate raw text and graph topology to unleash the vast potential of cross-modal data in TAGs for anomaly detection poses a challenging issue. This paper presents a novel end-to-end paradigm for text-attributed graph anomaly detection, named CMUCL. We simultaneously model data from both text and graph structures, and jointly train text and graph encoders by leveraging cross-modal and uni-modal multi-scale consistency to uncover potential anomaly-related information. Accordingly, we design an anomaly score estimator based on inconsistency mining to derive node-specific anomaly scores. Considering the lack of benchmark datasets tailored for anomaly detection on TAGs, we release 8 datasets to facilitate future research. Extensive evaluations show that CMUCL significantly advances in text-attributed graph anomaly detection, delivering an 11.13% increase in average accuracy (AP) over the suboptimal.

cs.LG↗

arXiv2025

Revisiting Graph Contrastive Learning on Anomaly Detection: A Structural Imbalance Perspective

The superiority of graph contrastive learning (GCL) has prompted its application to anomaly detection tasks for more powerful risk warning systems. Unfortunately, existing GCL-based models tend to excessively prioritize overall detection performance while neglecting robustness to structural imbalance, which can be problematic for many real-world networks following power-law degree distributions. Particularly, GCL-based methods may fail to capture tail anomalies (abnormal nodes with low degrees). This raises concerns about the security and robustness of current anomaly detection algorithms and therefore hinders their applicability in a variety of realistic high-risk scenarios. To the best of our knowledge, research on the robustness of graph anomaly detection to structural imbalance has received little scrutiny. To address the above issues, this paper presents a novel GCL-based framework named AD-GCL. It devises the neighbor pruning strategy to filter noisy edges for head nodes and facilitate the detection of genuine tail nodes by aligning from head nodes to forged tail nodes. Moreover, AD-GCL actively explores potential neighbors to enlarge the receptive field of tail nodes through anomaly-guided neighbor completion. We further introduce intra- and inter-view consistency loss of the original and augmentation graph for enhanced representation. The performance evaluation of the whole, head, and tail nodes on multiple datasets validates the comprehensive superiority of the proposed AD-GCL in detecting both head anomalies and tail anomalies.

cs.LG↗

arXiv2025

Feature Bank Enhancement for Distance-based Out-of-Distribution Detection

Out-of-distribution (OOD) detection is critical to ensuring the reliability of deep learning applications and has attracted significant attention in recent years. A rich body of literature has emerged to develop efficient score functions that assign high scores to in-distribution (ID) samples and low scores to OOD samples, thereby helping distinguish OOD samples. Among these methods, distance-based score functions are widely used because of their efficiency and ease of use. However, deep learning often leads to a biased distribution of data features, and extreme features are inevitable. These extreme features make the distance-based methods tend to assign too low scores to ID samples. This limits the OOD detection capabilities of such methods. To address this issue, we propose a simple yet effective method, Feature Bank Enhancement (FBE), that uses statistical characteristics from dataset to identify and constrain extreme features to the separation boundaries, therapy making the distance between samples inside and outside the distribution farther. We conducted experiments on large-scale ImageNet-1k and CIFAR-10 respectively, and the results show that our method achieves state-of-the-art performance on both benchmark. Additionally, theoretical analysis and supplementary experiments are conducted to provide more insights into our method.

cs.LG↗