Searcharxiv⌕ Search

arXiv subjects

Wei Sun

Publications and source records attributed to Wei Sun.

At least 145 records · Page 8Linked to original sources
arXiv2025

AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Images

The rapid development of text-to-image (T2I) generation approaches has attracted extensive interest in evaluating the quality of generated images, leading to the development of various quality assessment methods for general-purpose T2I outputs. However, existing image quality assessment (IQA) methods are limited to providing global quality scores, failing to deliver fine-grained perceptual evaluations for structurally complex subjects like humans, which is a critical challenge considering the frequent anatomical and textural distortions in AI-generated human images (AGHIs). To address this gap, we introduce AGHI-QA, the first large-scale benchmark specifically designed for quality assessment of AGHIs. The dataset comprises 4,000 images generated from 400 carefully crafted text prompts using 10 state of-the-art T2I models. We conduct a systematic subjective study to collect multidimensional annotations, including perceptual quality scores, text-image correspondence scores, visible and distorted body part labels. Based on AGHI-QA, we evaluate the strengths and weaknesses of current T2I methods in generating human images from multiple dimensions. Furthermore, we propose AGHI-Assessor, a novel quality metric that integrates the large multimodal model (LMM) with domain-specific human features for precise quality prediction and identification of visible and distorted body parts in AGHIs. Extensive experimental results demonstrate that AGHI-Assessor showcases state-of-the-art performance, significantly outperforming existing IQA methods in multidimensional quality assessment and surpassing leading LMMs in detecting structural distortions in AGHIs.

cs.CV↗
arXiv2025

NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Results

This paper presents a review for the NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement. The challenge comprises two tracks: (i) Efficient Video Quality Assessment (KVQ), and (ii) Diffusion-based Image Super-Resolution (KwaiSR). Track 1 aims to advance the development of lightweight and efficient video quality assessment (VQA) models, with an emphasis on eliminating reliance on model ensembles, redundant weights, and other computationally expensive components in the previous IQA/VQA competitions. Track 2 introduces a new short-form UGC dataset tailored for single image super-resolution, i.e., the KwaiSR dataset. It consists of 1,800 synthetically generated S-UGC image pairs and 1,900 real-world S-UGC images, which are split into training, validation, and test sets using a ratio of 8:1:1. The primary objective of the challenge is to drive research that benefits the user experience of short-form UGC platforms such as Kwai and TikTok. This challenge attracted 266 participants and received 18 valid final submissions with corresponding fact sheets, significantly contributing to the progress of short-form UGC VQA and image superresolution. The project is publicly available at https://github.com/lixinustc/KVQE- ChallengeCVPR-NTIRE2025.

eess.IV↗
arXiv2025

CAGS: Open-Vocabulary 3D Scene Understanding with Context-Aware Gaussian Splatting

Open-vocabulary 3D scene understanding is crucial for applications requiring natural language-driven spatial interpretation, such as robotics and augmented reality. While 3D Gaussian Splatting (3DGS) offers a powerful representation for scene reconstruction, integrating it with open-vocabulary frameworks reveals a key challenge: cross-view granularity inconsistency. This issue, stemming from 2D segmentation methods like SAM, results in inconsistent object segmentations across views (e.g., a "coffee set" segmented as a single entity in one view but as "cup + coffee + spoon" in another). Existing 3DGS-based methods often rely on isolated per-Gaussian feature learning, neglecting the spatial context needed for cohesive object reasoning, leading to fragmented representations. We propose Context-Aware Gaussian Splatting (CAGS), a novel framework that addresses this challenge by incorporating spatial context into 3DGS. CAGS constructs local graphs to propagate contextual features across Gaussians, reducing noise from inconsistent granularity, employs mask-centric contrastive learning to smooth SAM-derived features across views, and leverages a precomputation strategy to reduce computational cost by precomputing neighborhood relationships, enabling efficient training in large-scale scenes. By integrating spatial context, CAGS significantly improves 3D instance segmentation and reduces fragmentation errors on datasets like LERF-OVS and ScanNet, enabling robust language-guided 3D scene understanding.

cs.CV↗
arXiv2025

MonoDiff9D: Monocular Category-Level 9D Object Pose Estimation via Diffusion Model

Object pose estimation is a core means for robots to understand and interact with their environment. For this task, monocular category-level methods are attractive as they require only a single RGB camera. However, current methods rely on shape priors or CAD models of the intra-class known objects. We propose a diffusion-based monocular category-level 9D object pose generation method, MonoDiff9D. Our motivation is to leverage the probabilistic nature of diffusion models to alleviate the need for shape priors, CAD models, or depth sensors for intra-class unknown object pose estimation. We first estimate coarse depth via DINOv2 from the monocular image in a zero-shot manner and convert it into a point cloud. We then fuse the global features of the point cloud with the input image and use the fused features along with the encoded time step to condition MonoDiff9D. Finally, we design a transformer-based denoiser to recover the object pose from Gaussian noise. Extensive experiments on two popular benchmark datasets show that MonoDiff9D achieves state-of-the-art monocular category-level 9D object pose estimation accuracy without the need for shape priors or CAD models at any stage. Our code will be made public at https://github.com/CNJianLiu/MonoDiff9D.

cs.CV↗
arXiv2025

A high-resolution microresonator-frequency-comb spectrometer

Spectral analysis is one of the most powerful technologies for studying and understanding matter. As the devices for spectral analysis, spectrometers are widely used in material detection, isotope analysis, trace gas detection, and the study of atomic and molecular hyperfine structures. While high resolution, wide bandwidth and fast speed are essential factors, they are always trade-offs for conventional spectrometers. Here, we present a soliton-microcomb-based spectrometer that overcomes these challenges by integrating dissipative Kerr solitons (DKSs) with double-sideband modulation and parallelized detection. Leveraging a high-quality silicon nitride microresonator, we generate a broadband, fully stabilized soliton microcomb and employ radio-frequency-modulated double sidebands to scan the optical spectrum with the resolution constrained only by the comb-line linewidth. By projecting the comb lines onto a two-dimensional charge-coupled device (CCD) via a virtually imaged phased array (VIPA)-grating system, we enable parallel processing of all spectral components, circumventing sequential scanning delays. The resulting spectrometer achieves 200-kHz resolution across a 4-THz bandwidth with minutes-level processing time while maintaining robustness against environmental fluctuations. Being promising for miniaturization, this work bridges the gap between laboratory-grade performance and field-deployable practicality, unlocking new possibilities for spectroscopy in astronomy, metrology, and integrated photonics.

physics.optics↗
arXiv2025

Triply charmed baryons mass decomposition from lattice QCD

We present the first calculation of the connected scalar matrix element and the momentum fraction of charm quark within the$\frac{3}{2}^{+}$ and $\frac{3}{2}^{-}$triply charmed baryons on lattice QCD. The results are based on overlap valence fermions on two ensembles of $N_f=2+1$ domain wall fermion configurations with two lattice spacings. The corresponding sea quark pion masses are $300$ MeV and $278$ MeV. The separated contributions to the triply charmed baryon mass are derived through the decomposition of the QCD energy-momentum tensor. The contribution of the connected charm quark matrix element to the triply charmed baryon is about 3/2 times that of the charmonium. And it contributes almost 70% of the total mass. The mass splitting of $\frac{3}{2}^{+}$ and $\frac{3}{2}^{-}$triply charmed baryons is mainly from $\langle H_{E}\rangle$ of the QCD energy-momentum tensor. A mass decomposition from the quark model is also studied for comparison.

hep-lat↗
arXiv2025

PANDA: Parkinson's Assistance and Notification Driving Aid

Parkinson's Disease (PD) significantly impacts driving abilities, often leading to early driving cessation or accidents due to reduced motor control and increasing reaction times. To diminish the impact of these symptoms, we developed PANDA (Parkinson's Assistance and Notification Driving Aid), a multi-modality real-time alert system designed to monitor driving patterns continuously and provide immediate alerts for irregular driving behaviors, enhancing driver safety of individuals with PD. The system was developed through a participatory design process with 9 people with PD and 13 non-PD individuals using a driving simulator, which allowed us to identify critical design characteristics and collect detailed data on driving behavior. A user study involving individuals with PD evaluated the effectiveness of PANDA, exploring optimal strategies for delivering alerts and ensuring they are timely and helpful. Our findings demonstrate that PANDA has the potential to enhance the driving safety of individuals with PD, offering a valuable tool for maintaining independence and confidence behind the wheel.

cs.HC↗
arXiv2025

Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs

With the rising interest in research on Large Multi-modal Models (LMMs) for video understanding, many studies have emphasized general video comprehension capabilities, neglecting the systematic exploration into video quality understanding. To address this oversight, we introduce Q-Bench-Video in this paper, a new benchmark specifically designed to evaluate LMMs' proficiency in discerning video quality. a) To ensure video source diversity, Q-Bench-Video encompasses videos from natural scenes, AI-generated Content (AIGC), and Computer Graphics (CG). b) Building on the traditional multiple-choice questions format with the Yes-or-No and What-How categories, we include Open-ended questions to better evaluate complex scenarios. Additionally, we incorporate the video pair quality comparison question to enhance comprehensiveness. c) Beyond the traditional Technical, Aesthetic, and Temporal distortions, we have expanded our evaluation aspects to include the dimension of AIGC distortions, which addresses the increasing demand for video generation. Finally, we collect a total of 2,378 question-answer pairs and test them on 12 open-source & 5 proprietary LMMs. Our findings indicate that while LMMs have a foundational understanding of video quality, their performance remains incomplete and imprecise, with a notable discrepancy compared to human performance. Through Q-Bench-Video, we seek to catalyze community interest, stimulate further research, and unlock the untapped potential of LMMs to close the gap in video quality understanding.

cs.CV↗
arXiv2025

A chip-based optoelectronic-oscillator frequency comb

Microresonator-based Kerr frequency combs ("Kerr microcombs") constitute chip-scale frequency combs of broad spectral bandwidth and repetition rate ranging from gigahertz to terahertz. An appealing application exploiting microcombs' coherence and large repetition rate is microwave and millimeter-wave generation. Latest endeavor applying two-point optical frequency division (OFD) on photonic-chip-based microcombs has created microwaves with exceptionally low phase noise. Nevertheless, microcomb-based OFD still requires extensive active locking, additional lasers, and external RF or microwave sources, as well as sophisticated initiation. Here we demonstrate a simple and entirely passive (no active locking) architecture, which incorporates an optoelectronic oscillator (OEO) and symphonizes a coherent microcomb and a low-noise microwave spontaneously. Our OEO microcomb leverages state-of-the-art integrated chip devices including a high-power DFB laser, a broadband silicon Mach-Zehnder modulator, an ultralow-loss silicon nitride microresonator, and a high-speed photodetector. Each can be manufactured in large volume with low cost and high yield using established CMOS and III-V foundries. Our system synergizes a microcomb of 10.7 GHz repetition rate and an X-band microwave with phase noise of $-$97/$-$126/$-$130 dBc/Hz at 1/10/100 kHz Fourier frequency offset, yet does not demand active locking, additional lasers, and external RF or microwave sources. With potential to be fully integrated, our OEO microcomb can become an invaluable technology and building block for microwave photonics, radio-over-fiber, and optical communication.

physics.optics↗
arXiv2025

Ferroelectric Chirality-Driven Direction-Tunable and Spin-Invertible Corner States in 2D MOF-Based Magnetic Second-Order Topological Insulators

Despite the rapid progress in predicting 2D magnetic second-order topological insulators (SOTIs), effective strategies for manipulating their spin-polarized corner states remain largely unexplored. The interplay between ferroelectricity, chirality, magnetism, and topology presents an untapped opportunity for controlling these corner states. Here, we propose a novel approach for tuning spin-polarized corner states in 2D magnetic SOTIs by inducing ferroelectric chirality in 2D metal-organic frameworks (MOFs) with intrinsic structural flexibility. Through symmetry analysis, we strategically replace pyrazine (pyz) ligands with 2-pyrazinolate (2-pyzol) ligands in the 2D MOF Cr(pyz)2, leading to the emergence of a new 2D magnetic SOTI, Cr(2-pyzol)2, which facilitates ferroelectric chirality controlled spin-polarized corner states in both spin channels. Through first-principle calculations, we demonstrate that Cr(2-pyzol)2 belongs to ferroelectric chiral systems, and its corner states can be directionally tuned in real space and spin-inverted in spin space upon ferroelectric chirality switching. Our work represents the first attempt to simultaneously manipulate corner states in both real space and spin space, offering a new strategy for integrating ferroelectric chirality into 2D MOF-based magnetic SOTIs.

cond-mat.mtrl-sci↗
arXiv2025

Linear type conditional specifications for multivariate count variables

This paper investigates conditional specifications for multivariate count variables. Recently, the spatial count data literature has proposed several conditional models such that the conditional expectations are linear in the conditioning variables. These models are much easier to estimate than existing spatial count models based on Gaussian random field. However, whether or not such conditional specifications are compatible have not been addressed. We investigate two large families of conditional models, that are the compound autoregressive model and the random coefficient integer autoregressive model. We characterize all the solutions to these two families of models at arbitrary dimensions, and find that only a handful of them admit non-trivial solutions. We then show that if we focus on the linearity condition of the conditional expectations only, a considerable larger family of solutions can be obtained. This suggests that for spatial count data modeling, semi-parametric type specifications that impose the conditional expectation structure is preferable.

math.ST↗
arXiv2025

API Pack: A Massive Multi-Programming Language Dataset for API Call Generation

We introduce API Pack, a massive multi-programming language dataset containing over one million instruction-API calls for improving the API call generation capabilities of large language models. Our evaluation highlights three key findings: First, fine-tuning on API Pack enables open-source models to outperform GPT-3.5 and GPT-4 in generating code for entirely new API calls. We show this by fine-tuning CodeLlama-13B on 20,000 Python instances from API Pack. Second, fine-tuning on a large dataset in one language, combined with smaller datasets from others, improves API generation accuracy across multiple languages. Third, we confirm the benefits of larger datasets for API generalization, as increasing fine-tuning data to one million instances enhances generalization to new APIs. To support further research, we open-source the API Pack dataset, trained model, and code at https://github.com/zguo0525/API-Pack.

cs.CL↗
arXiv2025

Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Models

Multimodal Large Language Models (LLMs) are pivotal in revolutionizing customer support and operations by integrating multiple modalities such as text, images, and audio. Federated Prompt Learning (FPL) is a recently proposed approach that combines pre-trained multimodal LLMs such as vision-language models with federated learning to create personalized, privacy-preserving AI systems. However, balancing the competing goals of personalization, generalization, and privacy remains a significant challenge. Over-personalization can lead to overfitting, reducing generalizability, while stringent privacy measures, such as differential privacy, can hinder both personalization and generalization. In this paper, we propose a Differentially Private Federated Prompt Learning (DP-FPL) approach to tackle this challenge by leveraging a low-rank factorization scheme to capture generalization while maintaining a residual term that preserves expressiveness for personalization. To ensure privacy, we introduce a novel method where we apply local differential privacy to the two low-rank components of the local prompt, and global differential privacy to the global prompt. Our approach mitigates the impact of privacy noise on the model performance while balancing the tradeoff between personalization and generalization. Extensive experiments demonstrate the effectiveness of our approach over other benchmarks.

cs.LG↗
arXiv2025

Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving

Understanding world dynamics is crucial for planning in autonomous driving. Recent methods attempt to achieve this by learning a 3D occupancy world model that forecasts future surrounding scenes based on current observation. However, 3D occupancy labels are still required to produce promising results. Considering the high annotation cost for 3D outdoor scenes, we propose a semi-supervised vision-centric 3D occupancy world model, PreWorld, to leverage the potential of 2D labels through a novel two-stage training paradigm: the self-supervised pre-training stage and the fully-supervised fine-tuning stage. Specifically, during the pre-training stage, we utilize an attribute projection head to generate different attribute fields of a scene (e.g., RGB, density, semantic), thus enabling temporal supervision from 2D labels via volume rendering techniques. Furthermore, we introduce a simple yet effective state-conditioned forecasting module to recursively forecast future occupancy and ego trajectory in a direct manner. Extensive experiments on the nuScenes dataset validate the effectiveness and scalability of our method, and demonstrate that PreWorld achieves competitive performance across 3D occupancy prediction, 4D occupancy forecasting and motion planning tasks.

cs.CV↗
arXiv2025

A-Bench: Are LMMs Masters at Evaluating AI-generated Images?

How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models. Given the high costs and extensive time commitments required for user studies, many researchers have turned towards employing large multi-modal models (LMMs) as AIGI evaluators, the precision and validity of which are still questionable. Furthermore, traditional benchmarks often utilize mostly natural-captured content rather than AIGIs to test the abilities of LMMs, leading to a noticeable gap for AIGIs. Therefore, we introduce A-Bench in this paper, a benchmark designed to diagnose whether LMMs are masters at evaluating AIGIs. Specifically, A-Bench is organized under two key principles: 1) Emphasizing both high-level semantic understanding and low-level visual quality perception to address the intricate demands of AIGIs. 2) Various generative models are utilized for AIGI creation, and various LMMs are employed for evaluation, which ensures a comprehensive validation scope. Ultimately, 2,864 AIGIs from 16 text-to-image models are sampled, each paired with question-answers annotated by human experts, and tested across 18 leading LMMs. We hope that A-Bench will significantly enhance the evaluation process and promote the generation quality for AIGIs. The benchmark is available at https://github.com/Q-Future/A-Bench.

cs.CV↗
arXiv2025

Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation

Nine-degrees-of-freedom (9-DoF) object pose and size estimation is crucial for enabling augmented reality and robotic manipulation. Category-level methods have received extensive research attention due to their potential for generalization to intra-class unknown objects. However, these methods require manual collection and labeling of large-scale real-world training data. To address this problem, we introduce a diffusion-based paradigm for domain-generalized category-level 9-DoF object pose estimation. Our motivation is to leverage the latent generalization ability of the diffusion model to address the domain generalization challenge in object pose estimation. This entails training the model exclusively on rendered synthetic data to achieve generalization to real-world scenes. We propose an effective diffusion model to redefine 9-DoF object pose estimation from a generative perspective. Our model does not require any 3D shape priors during training or inference. By employing the Denoising Diffusion Implicit Model, we demonstrate that the reverse diffusion process can be executed in as few as 3 steps, achieving near real-time performance. Finally, we design a robotic grasping system comprising both hardware and software components. Through comprehensive experiments on two benchmark datasets and the real-world robotic system, we show that our method achieves state-of-the-art domain generalization performance. Our code will be made public at https://github.com/CNJianLiu/Diff9D.

cs.CV↗
arXiv2025

Do We Really Need to Design New Byzantine-robust Aggregation Rules?

Federated learning (FL) allows multiple clients to collaboratively train a global machine learning model through a server, without exchanging their private training data. However, the decentralized aspect of FL makes it susceptible to poisoning attacks, where malicious clients can manipulate the global model by sending altered local model updates. To counter these attacks, a variety of aggregation rules designed to be resilient to Byzantine failures have been introduced. Nonetheless, these methods can still be vulnerable to sophisticated attacks or depend on unrealistic assumptions about the server. In this paper, we demonstrate that there is no need to design new Byzantine-robust aggregation rules; instead, FL can be secured by enhancing the robustness of well-established aggregation rules. To this end, we present FoundationFL, a novel defense mechanism against poisoning attacks. FoundationFL involves the server generating synthetic updates after receiving local model updates from clients. It then applies existing Byzantine-robust foundational aggregation rules, such as Trimmed-mean or Median, to combine clients' model updates with the synthetic ones. We theoretically establish the convergence performance of FoundationFL under Byzantine settings. Comprehensive experiments across several real-world datasets validate the efficiency of our FoundationFL method.

cs.CR↗