arXiv subjects

Yuan Gao

Publications and source records attributed to Yuan Gao.

At least 235 records · Page 13Linked to original sources

arXiv2025

Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient

Recent Large-Language Models (LLMs) pruning methods typically operate at the post-training phase without the expensive weight finetuning, however, their pruning criteria often rely on heuristically hand-crafted metrics, potentially leading to suboptimal performance. We instead propose a novel optimization-based structural pruning that learns the pruning masks in a probabilistic space directly by optimizing the loss of the pruned model. To preserve efficiency, our method eliminates the back-propagation through the LLM per se during optimization, requiring only the forward pass of the LLM. We achieve this by learning an underlying Bernoulli distribution to sample binary pruning masks, where we decouple the Bernoulli parameters from LLM loss, facilitating efficient optimization via policy gradient estimator without back-propagation. Thus, our method can 1) support global and heterogeneous pruning (i.e., automatically determine different redundancy for different layers), and 2) optionally initialize with a metric-based method (for our Bernoulli distributions). Extensive experiments conducted on LLaMA, LLaMA-2, LLaMA-3, Vicuna, and Mistral models using the C4 and WikiText2 datasets demonstrate the promising performance of our method in efficiency and effectiveness. Code is available at https://github.com/ethanygao/backprop-free_LLM_pruning.

cs.LG↗

arXiv2025

LearnAFE: Circuit-Algorithm Co-design Framework for Learnable Audio Analog Front-End

This paper presents a circuit-algorithm co-design framework for learnable analog front-end (AFE) in audio signal classification. Designing AFE and backend classifiers separately is a common practice but non-ideal, as shown in this paper. Instead, this paper proposes a joint optimization of the backend classifier with the AFE's transfer function to achieve system-level optimum. More specifically, the transfer function parameters of an analog bandpass filter (BPF) bank are tuned in a signal-to-noise ratio (SNR)-aware training loop for the classifier. Using a co-design loss function LBPF, this work shows superior optimization of both the filter bank and the classifier. Implemented in open-source SKY130 130nm CMOS process, the optimized design achieved 90.5%-94.2% accuracy for 10-keyword classification task across a wide range of input signal SNR from 5 dB to 20 dB, with only 22k classifier parameters. Compared to conventional approach, the proposed audio AFE achieves 8.7% and 12.9% reduction in power and capacitor area respectively.

eess.AS↗

arXiv2025

CKD-EHR:Clinical Knowledge Distillation for Electronic Health Records

Electronic Health Records (EHR)-based disease prediction models have demonstrated significant clinical value in promoting precision medicine and enabling early intervention. However, existing large language models face two major challenges: insufficient representation of medical knowledge and low efficiency in clinical deployment. To address these challenges, this study proposes the CKD-EHR (Clinical Knowledge Distillation for EHR) framework, which achieves efficient and accurate disease risk prediction through knowledge distillation techniques. Specifically, the large language model Qwen2.5-7B is first fine-tuned on medical knowledge-enhanced data to serve as the teacher model.It then generates interpretable soft labels through a multi-granularity attention distillation mechanism. Finally, the distilled knowledge is transferred to a lightweight BERT student model. Experimental results show that on the MIMIC-III dataset, CKD-EHR significantly outperforms the baseline model:diagnostic accuracy is increased by 9%, F1-score is improved by 27%, and a 22.2 times inference speedup is achieved. This innovative solution not only greatly improves resource utilization efficiency but also significantly enhances the accuracy and timeliness of diagnosis, providing a practical technical approach for resource optimization in clinical settings. The code and data for this research are available athttps://github.com/209506702/CKD_EHR.

cs.CL↗

arXiv2025

Limited-Angle CBCT Reconstruction via Geometry-Integrated Cycle-domain Denoising Diffusion Probabilistic Models

Cone-beam CT (CBCT) is widely used in clinical radiotherapy for image-guided treatment, improving setup accuracy, adaptive planning, and motion management. However, slow gantry rotation limits performance by introducing motion artifacts, blurring, and increased dose. This work aims to develop a clinically feasible method for reconstructing high-quality CBCT volumes from consecutive limited-angle acquisitions, addressing imaging challenges in time- or dose-constrained settings. We propose a limited-angle (LA) geometry-integrated cycle-domain (LA-GICD) framework for CBCT reconstruction, comprising two denoising diffusion probabilistic models (DDPMs) connected via analytic cone-beam forward and back projectors. A Projection-DDPM completes missing projections, followed by back-projection, and an Image-DDPM refines the volume. This dual-domain design leverages complementary priors from projection and image spaces to achieve high-quality reconstructions from limited-angle (<= 90 degrees) scans. Performance was evaluated against full-angle reconstruction. Four board-certified medical physicists conducted assessments. A total of 78 planning CTs in common CBCT geometries were used for training and evaluation. The method achieved a mean absolute error of 35.5 HU, SSIM of 0.84, and PSNR of 29.8 dB, with visibly reduced artifacts and improved soft-tissue clarity. LA-GICD's geometry-aware dual-domain learning, embedded in analytic forward/backward operators, enabled artifact-free, high-contrast reconstructions from a single 90-degree scan, reducing acquisition time and dose four-fold. LA-GICD improves limited-angle CBCT reconstruction with strong data fidelity and anatomical realism. It offers a practical solution for short-arc acquisitions, enhancing CBCT use in radiotherapy by providing clinically applicable images with reduced scan time and dose for more accurate, personalized treatments.

cs.CV↗

arXiv2025

Analysis of Anonymous User Interaction Relationships and Prediction of Advertising Feedback Based on Graph Neural Network

While online advertising is highly dependent on implicit interaction networks of anonymous users for engagement inference, and for the selection and optimization of delivery strategies, existing graph models seldom can capture the multi-scale temporal, semantic and higher-order dependency features of these interaction networks, thus it's hard to describe the complicated patterns of the anonymous behavior. In this paper, we propose Decoupled Temporal-Hierarchical Graph Neural Network (DTH-GNN), which achieves three main contributions. Above all, we introduce temporal edge decomposition, which divides each interaction into three types of channels: short-term burst, diurnal cycle and long-range memory, and conducts feature extraction using the convolution kernel of parallel dilated residuals; Furthermore, our model builds a hierarchical heterogeneous aggregation, where user-user, user-advertisement, advertisement-advertisement subgraphs are combined through the meta-path conditional Transformer encoder, where the noise structure is dynamically tamped down via the synergy of cross-channel self-attention and gating relationship selector. Thirdly, the contrast regularity of feedback perception is formulated, the consistency of various time slices is maximized, the entropy of control exposure information with dual-view target is maximized, the global prototype of dual-momentum queue distillation is presented, and the strategy gradient layer with light weight is combined with delaying transformation signal to fine-tune the node representation for benefit-oriented. The AUC of DTH-GNN improved by 8.2% and the logarithmic loss improved by 5.7% in comparison with the best baseline model.

cs.IR↗

arXiv2025

Abstract categorical residues and Calabi-Yau structures

Inspired by the simple fact that a compact n-dimensional manifold-with-boundary which satisfies Poincaré-Lefschetz duality of dimension n has a boundary which itself satisfies Poincaré duality of dimension n, we show that the categorical formal punctured neighborhood of infinity, a canonical categorical construction associated to every $A_{\infty}$ category, has a weak proper Calabi-Yau structure of dimension n-1 whenever the original $A_{\infty}$ category admits a weak smooth Calabi-Yau structure of dimension n. Applications include proper Calabi-Yau structures on Rabinowitz Fukaya category of a Liouville manifold and Orlov's singularity category of a proper singular Gorenstein scheme of finite type.

math.SG↗

arXiv2025

A Large Convolutional Neural Network for Clinical Target and Multi-organ Segmentation in Gynecologic Brachytherapy with Multi-stage Learning

Purpose: Accurate segmentation of clinical target volumes (CTV) and organs-at-risk is crucial for optimizing gynecologic brachytherapy (GYN-BT) treatment planning. However, anatomical variability, low soft-tissue contrast in CT imaging, and limited annotated datasets pose significant challenges. This study presents GynBTNet, a novel multi-stage learning framework designed to enhance segmentation performance through self-supervised pretraining and hierarchical fine-tuning strategies. Methods: GynBTNet employs a three-stage training strategy: (1) self-supervised pretraining on large-scale CT datasets using sparse submanifold convolution to capture robust anatomical representations, (2) supervised fine-tuning on a comprehensive multi-organ segmentation dataset to refine feature extraction, and (3) task-specific fine-tuning on a dedicated GYN-BT dataset to optimize segmentation performance for clinical applications. The model was evaluated against state-of-the-art methods using the Dice Similarity Coefficient (DSC), 95th percentile Hausdorff Distance (HD95), and Average Surface Distance (ASD). Results: Our GynBTNet achieved superior segmentation performance, significantly outperforming nnU-Net and Swin-UNETR. Notably, it yielded a DSC of 0.837 +/- 0.068 for CTV, 0.940 +/- 0.052 for the bladder, 0.842 +/- 0.070 for the rectum, and 0.871 +/- 0.047 for the uterus, with reduced HD95 and ASD compared to baseline models. Self-supervised pretraining led to consistent performance improvements, particularly for structures with complex boundaries. However, segmentation of the sigmoid colon remained challenging, likely due to anatomical ambiguities and inter-patient variability. Statistical significance analysis confirmed that GynBTNet's improvements were significant compared to baseline models.

cs.CV↗

arXiv2025

ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI

Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks and lack features critical for scaling generalizable robotics and sim2real. We introduce and open source ManiSkill3, the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more. Simulation with rendering on ManiSkill3 can run 10-1000x faster with 2-3x less GPU memory usage than other platforms, achieving up to 30,000+ FPS in benchmarked environments due to minimal python/pytorch overhead in the system, simulation on the GPU, and the use of the SAPIEN parallel rendering system. Tasks that used to take hours to train can now take minutes. We further provide the most comprehensive range of GPU parallelized environments/tasks spanning 12 distinct domains including but not limited to mobile manipulation for tasks such as drawing, humanoids, and dextrous manipulation in realistic scenes designed by artists or real-world digital twins. In addition, millions of demonstration frames are provided from motion planning, RL, and teleoperation. ManiSkill3 also provides a comprehensive set of baselines that span popular RL and learning-from-demonstrations algorithms.

cs.RO↗

arXiv2025

GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers

Graph Transformers (GTs) have demonstrated remarkable performance in graph representation learning over popular graph neural networks (GNNs). However, self--attention, the core module of GTs, preserves only low-frequency signals in graph features, leading to ineffectiveness in capturing other important signals like high-frequency ones. Some recent GT models help alleviate this issue, but their flexibility and expressiveness are still limited since the filters they learn are fixed on predefined graph spectrum or spectral order. To tackle this challenge, we propose a Graph Fourier Kolmogorov-Arnold Transformer (GrokFormer), a novel GT model that learns highly expressive spectral filters with adaptive graph spectrum and spectral order through a Fourier series modeling over learnable activation functions. We demonstrate theoretically and empirically that the proposed GrokFormer filter offers better expressiveness than other spectral methods. Comprehensive experiments on 10 real-world node classification datasets across various domains, scales, and graph properties, as well as 5 graph classification datasets, show that GrokFormer outperforms state-of-the-art GTs and GNNs. Our code is available at https://github.com/GGA23/GrokFormer

cs.LG↗

arXiv2025

A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

For personalized marketing, a new challenge of how to effectively algorithm the A/B testing to maximize user response is urgently to be overcome. In this paper, we present a new approach, the RL-LLM-AB test framework, for using reinforcement learning strategy optimization combined with LLM to automate and personalize A/B tests. The RL-LLM-AB test is built upon the pre-trained instruction-tuned language model. It first generates A/B versions of candidate content variants using a Prompt-Conditioned Generator, and then dynamically embeds and fuses the user portrait and the context of the current query with the multi-modal perception module to constitute the current interaction state. The content version is then selected in real-time through the policy optimization module with an Actor-Critic structure, and long-term revenue is estimated according to real-time feedback (such as click-through rate and conversion rate). Furthermore, a Memory-Augmented Reward Estimator is embedded into the framework to capture long-term user preference drift, which helps to generalize policy across multiple users and content contexts. Numerical results demonstrate the superiority of our proposed RL-LLM-ABTest over existing A/B testing methods, including classical A/B testing, Contextual Bandits, and benchmark reinforcement learning approaches on real-world marketing data.

cs.IR↗

arXiv2025

Neural Inverse Scattering with Score-based Regularization

Inverse scattering is a fundamental challenge in many imaging applications, ranging from microscopy to remote sensing. Solving this problem often requires jointly estimating two unknowns -- the image and the scattering field inside the object -- necessitating effective image prior to regularize the inference. In this paper, we propose a regularized neural field (NF) approach which integrates the denoising score function used in score-based generative models. The neural field formulation offers convenient flexibility to performing joint estimation, while the denoising score function imposes the rich structural prior of images. Our results on three high-contrast simulated objects show that the proposed approach yields a better imaging quality compared to the state-of-the-art NF approach, where regularization is based on total variation.

eess.IV↗

arXiv2025

DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding

In this paper, we introduce DINO-X, which is a unified object-centric vision model developed by IDEA Research with the best open-world object detection performance to date. DINO-X employs the same Transformer-based encoder-decoder architecture as Grounding DINO 1.5 to pursue an object-level representation for open-world object understanding. To make long-tailed object detection easy, DINO-X extends its input options to support text prompt, visual prompt, and customized prompt. With such flexible prompt options, we develop a universal object prompt to support prompt-free open-world detection, making it possible to detect anything in an image without requiring users to provide any prompt. To enhance the model's core grounding capability, we have constructed a large-scale dataset with over 100 million high-quality grounding samples, referred to as Grounding-100M, for advancing the model's open-vocabulary detection performance. Pre-training on such a large-scale grounding dataset leads to a foundational object-level representation, which enables DINO-X to integrate multiple perception heads to simultaneously support multiple object perception and understanding tasks, including detection, segmentation, pose estimation, object captioning, object-based QA, etc. Experimental results demonstrate the superior performance of DINO-X. Specifically, the DINO-X Pro model achieves 56.0 AP, 59.8 AP, and 52.4 AP on the COCO, LVIS-minival, and LVIS-val zero-shot object detection benchmarks, respectively. Notably, it scores 63.3 AP and 56.5 AP on the rare classes of LVIS-minival and LVIS-val benchmarks, improving the previous SOTA performance by 5.8 AP and 5.0 AP. Such a result underscores its significantly improved capacity for recognizing long-tailed objects.

cs.CV↗

arXiv2025

Reality-Infused Deep Learning for Angle-resolved Quasi-optical Fourier Surfaces

Optical Fourier surfaces (OFSs), featuring sinusoidally profiled diffractive elements, manipulate light through patterned nanostructures and incident angle modulation. Compared to altering structural parameters, tuning elevation and azimuth angles offers greater design flexibility for light field control. However, angle-resolved responses of OFSs are often complex due to diverse mode excitations and couplings, complicating the alignment between simulations and practical fabrication. Here, we present a reality-infused deep learning framework, empowered by angle-resolved measurements, to enable real-time and accurate predictions of angular dispersion in quasi-OFSs. This approach captures critical features, including nanofabrication and measurement imperfections, which conventional simulation-based methods typically overlook. Our framework significantly accelerates the design process while achieving predictive performance highly consistent with experimental observations across broad angular and spectral ranges. Our study supports valuable insights into the development of OFS-based devices, and represents a paradigm shift from simulation-driven to reality-infused methods, paving the way for advancements in optical design applications.

physics.optics↗

arXiv2025

AI-Powered Agile Analog Circuit Design and Optimization

Artificial intelligence (AI) techniques are transforming analog circuit design by automating device-level tuning and enabling system-level co-optimization. This paper integrates two approaches: (1) AI-assisted transistor sizing using Multi-Objective Bayesian Optimization (MOBO) for direct circuit parameter optimization, demonstrated on a linearly tunable transconductor; and (2) AI-integrated circuit transfer function modeling for system-level optimization in a keyword spotting (KWS) application, demonstrated by optimizing an analog bandpass filter within a machine learning training loop. The combined insights highlight how AI can improve analog performance, reduce design iteration effort, and jointly optimize analog components and application-level metrics.

cs.AR↗

arXiv2025

Personalized Risks and Regulatory Strategies of Large Language Models in Digital Advertising

Although large language models have demonstrated the potential for personalized advertising recommendations in experimental environments, in actual operations, how advertising recommendation systems can be combined with measures such as user privacy protection and data security is still an area worthy of in-depth discussion. To this end, this paper studies the personalized risks and regulatory strategies of large language models in digital advertising. This study first outlines the principles of Large Language Model (LLM), especially the self-attention mechanism based on the Transformer architecture, and how to enable the model to understand and generate natural language text. Then, the BERT (Bidirectional Encoder Representations from Transformers) model and the attention mechanism are combined to construct an algorithmic model for personalized advertising recommendations and user factor risk protection. The specific steps include: data collection and preprocessing, feature selection and construction, using large language models such as BERT for advertising semantic embedding, and ad recommendations based on user portraits. Then, local model training and data encryption are used to ensure the security of user privacy and avoid the leakage of personal data. This paper designs an experiment for personalized advertising recommendation based on a large language model of BERT and verifies it with real user data. The experimental results show that BERT-based advertising push can effectively improve the click-through rate and conversion rate of advertisements. At the same time, through local model training and privacy protection mechanisms, the risk of user privacy leakage can be reduced to a certain extent.

cs.CL↗

arXiv2025

Hyper-parameter Optimization for Wireless Network Traffic Prediction Models with A Novel Meta-Learning Framework

This paper proposes a novel meta-learning based hyper-parameter optimization framework for wireless network traffic prediction (NTP) models. The primary objective is to accumulate and leverage the acquired hyper-parameter optimization experience, enabling the rapid determination of optimal hyper-parameters for new tasks. In this paper, an attention-based deep neural network (ADNN) is employed as the base-learner to address specific NTP tasks. The meta-learner is an innovative framework that integrates meta-learning with the k-nearest neighbor algorithm (KNN), genetic algorithm (GA), and gated residual network (GRN). Specifically, KNN is utilized to identify a set of candidate hyper-parameter selection strategies for a new task, which then serves as the initial population for GA, while a GRN-based chromosome screening module accelerates the validation of offspring chromosomes, ultimately determining the optimal hyper-parameters. Experimental results demonstrate that, compared to traditional methods such as Bayesian optimization (BO), GA, and particle swarm optimization (PSO), the proposed framework determines optimal hyper-parameters more rapidly, significantly reduces optimization time, and enhances the performance of the base-learner. It achieves an optimal balance between optimization efficiency and prediction accuracy.

cs.NI↗

arXiv2025

Intelligent Mode-Locked Single-Cavity Dual-Comb Laser Utilizing Time-Stretch Dispersive Fourier Transform Spectroscopy

As dual combs play a significant role in numerous high-precision measurements, their efficient generation has been widely researched. Although the single-cavity dual-comb generation can avoid the complex active stabilization methods, achieving and maintaining stable dual-comb mode locking within a single cavity remains a critical challenge. To break through this constraint, a two-part evaluation criterion containing a fitness function and a CNN-Transformer network is employed to achieve mode locking and classify the dual-comb mode-locked state. Simulated time-stretch dispersive Fourier transform (DFT) spectra are used as datasets, which simplifies the optimization process and does not rely on specific experimental data. A developed evolutionary algorithm (EA) for paddle-based motorized polarization controllers (MPCs) is proposed, enabling the intelligent attainment of dual-comb mode-locked states. A real-time library stores fitness and MPC angles, facilitating mode-locked state achievement within 2 seconds. Finally, long term running of dual-comb mode locking is ensured by a random collision algorithm utilizing an evaluation criterion of weak soliton peaks.

physics.optics↗

arXiv2025

AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization

Preference alignment through Direct Preference Optimization (DPO) has demonstrated significant effectiveness in aligning multimodal large language models (MLLMs) with human preferences. However, existing methods focus primarily on language preferences while neglecting the critical visual context. In this paper, we propose an Adaptive Vision-enhanced Preference optimization (AdaViP) that addresses these limitations through two key innovations: (1) vision-based preference pair construction, which integrates multiple visual foundation models to strategically remove key visual elements from the image, enhancing MLLMs' sensitivity to visual details; and (2) adaptive preference optimization that dynamically balances vision- and language-based preferences for more accurate alignment. Extensive evaluations across different benchmarks demonstrate our effectiveness. Notably, our AdaViP-7B achieves 93.7% and 96.4% reductions in response-level and mentioned-level hallucination respectively on the Object HalBench, significantly outperforming current state-of-the-art methods.

cs.CV↗