arXiv subjects

Pai Peng

Publications and source records attributed to Pai Peng.

At least 37 records · Page 2Linked to original sources

arXiv2025

Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs

Multimodal Large Language Models (MLLMs) have achieved impressive advances, yet object hallucination remains a persistent challenge. Existing methods, based on the flawed assumption that omission and fabrication hallucinations share a common cause, often reduce omissions only to trigger more fabrications. In this work, we overturn this view by demonstrating that omission hallucinations arise from insufficient confidence when mapping perceived visual features to linguistic expressions, whereas fabrication hallucinations result from spurious associations within the cross-modal representation space due to statistical biases in the training corpus. Building on findings from visual attention intervention experiments, we propose the Visual-Semantic Attention Potential Field, a conceptual framework that reveals how the model constructs visual evidence to infer the presence or absence of objects. Leveraging this insight, we introduce Visual Potential Field Calibration (VPFC), a plug-and-play hallucination mitigation method that effectively reduces omission hallucinations without introducing additional fabrication hallucinations. Our findings reveal a critical oversight in current object hallucination research and chart new directions for developing more robust and balanced hallucination mitigation strategies.

cs.CV↗

arXiv2025

Broad-temperature-range ultrafast terahertz excitation of collective dynamics in polar skyrmions

Ultrafast coherent control of electric dipoles using strong terahertz (THz) pulses provides a means to discover hidden phases of materials and potentially leads to applications in high-speed electro-optic devices. The effectiveness of this means, albeit demonstrated in architype (incipient) ferroelectric systems such as SrTiO3, hinges on a spectral overlapping between their soft phonon modes within the excitation bandwidth of THz pulses. Generally this can only induce an appreciable coupling close to the phase transition temperatures, where the lattice phonons substantially soften. Because of their emergent subterahertz collective dynamics, a fundamentally distinct and effective THz coupling mechanism can be envisaged in topological polar structures recently discovered in PbTiO3/SrTiO3 superlattices. Here, we show that polar skyrmions can be coherently driven into a hidden phase with transient macroscopic polarization, as probed based on THz field-induced second harmonic generation and optical Kerr effects. Such an ultrafast THz-driven phase transition is found to sustain across a broad temperature range of 4-470 K, in accordance with the equilibrium stability field of the skyrmions. Supplemented by dynamical phase-field simulations, we identify the spatial symmetries and relaxation behaviors of the excited collective modes, thereby revealing their correlation with the emergence of the polar phases. Our results unveil the exotic dynamical properties of topological polar structures, which could be technologically exploited given their remarkable flexibility in structure design and tunability under external fields.

cond-mat.mtrl-sci↗

arXiv2025

Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction

We present GDFusion, a temporal fusion method for vision-based 3D semantic occupancy prediction (VisionOcc). GDFusion opens up the underexplored aspects of temporal fusion within the VisionOcc framework, focusing on both temporal cues and fusion strategies. It systematically examines the entire VisionOcc pipeline, identifying three fundamental yet previously overlooked temporal cues: scene-level consistency, motion calibration, and geometric complementation. These cues capture diverse facets of temporal evolution and make distinct contributions across various modules in the VisionOcc framework. To effectively fuse temporal signals across heterogeneous representations, we propose a novel fusion strategy by reinterpreting the formulation of vanilla RNNs. This reinterpretation leverages gradient descent on features to unify the integration of diverse temporal information, seamlessly embedding the proposed temporal cues into the network. Extensive experiments on nuScenes demonstrate that GDFusion significantly outperforms established baselines. Notably, on Occ3D benchmark, it achieves 1.4\%-4.8\% mIoU improvements and reduces memory consumption by 27\%-72\%.

cs.CV↗

arXiv2025

You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving

Outdoor LiDAR point cloud 3D instance segmentation is a crucial task in autonomous driving. However, it requires laborious human efforts to annotate the point cloud for training a segmentation model. To address this challenge, we propose a YoCo framework, which generates 3D pseudo labels using minimal coarse click annotations in the bird's eye view plane. It is a significant challenge to produce high-quality pseudo labels from sparse annotations. Our YoCo framework first leverages vision foundation models combined with geometric constraints from point clouds to enhance pseudo label generation. Second, a temporal and spatial-based label updating module is designed to generate reliable updated labels. It leverages predictions from adjacent frames and utilizes the inherent density variation of point clouds (dense near, sparse far). Finally, to further improve label quality, an IoU-guided enhancement module is proposed, replacing pseudo labels with high-confidence and high-IoU predictions. Experiments on the Waymo dataset demonstrate YoCo's effectiveness and generality, achieving state-of-the-art performance among weakly supervised methods and surpassing fully supervised Cylinder3D. Additionally, the YoCo is suitable for various networks, achieving performance comparable to fully supervised methods with minimal fine-tuning using only 0.8% of the fully labeled data, significantly reducing annotation costs.

cs.CV↗

arXiv2025

Int2Planner: An Intention-based Multi-modal Motion Planner for Integrated Prediction and Planning

Motion planning is a critical module in autonomous driving, with the primary challenge of uncertainty caused by interactions with other participants. As most previous methods treat prediction and planning as separate tasks, it is difficult to model these interactions. Furthermore, since the route path navigates ego vehicles to a predefined destination, it provides relatively stable intentions for ego vehicles and helps constrain uncertainty. On this basis, we construct Int2Planner, an \textbf{Int}ention-based \textbf{Int}egrated motion \textbf{Planner} achieves multi-modal planning and prediction. Instead of static intention points, Int2Planner utilizes route intention points for ego vehicles and generates corresponding planning trajectories for each intention point to facilitate multi-modal planning. The experiments on the private dataset and the public nuPlan benchmark show the effectiveness of route intention points, and Int2Planner achieves state-of-the-art performance. We also deploy it in real-world vehicles and have conducted autonomous driving for hundreds of kilometers in urban areas. It further verifies that Int2Planner can continuously interact with the traffic environment. Code will be avaliable at https://github.com/cxlz/Int2Planner.

cs.RO↗

arXiv2025

Generative Planning with 3D-vision Language Pre-training for End-to-End Autonomous Driving

Autonomous driving is a challenging task that requires perceiving and understanding the surrounding environment for safe trajectory planning. While existing vision-based end-to-end models have achieved promising results, these methods are still facing the challenges of vision understanding, decision reasoning and scene generalization. To solve these issues, a generative planning with 3D-vision language pre-training model named GPVL is proposed for end-to-end autonomous driving. The proposed paradigm has two significant aspects. On one hand, a 3D-vision language pre-training module is designed to bridge the gap between visual perception and linguistic understanding in the bird's eye view. On the other hand, a cross-modal language model is introduced to generate holistic driving decisions and fine-grained trajectories with perception and navigation information in an auto-regressive manner. Experiments on the challenging nuScenes dataset demonstrate that the proposed scheme achieves excellent performances compared with state-of-the-art methods. Besides, the proposed GPVL presents strong generalization ability and real-time potential when handling high-level commands in various scenarios. It is believed that the effective, robust and efficient performance of GPVL is crucial for the practical application of future autonomous driving systems. Code is available at https://github.com/ltp1995/GPVL

cs.CV↗

arXiv2025

Maximum likelihood estimation in the sparse Rasch model

The Rasch model has been widely used to analyse item response data in psychometrics and educational assessments. When the number of individuals and items are large, it may be impractical to provide all possible responses. It is desirable to study sparse item response experiments. Here, we propose to use the Erdős\textendash Rényi random sampling design, where an individual responds to an item with low probability $p$. We prove the uniform consistency of the maximum likelihood estimator %by developing a leave-one-out method for the Rasch model when both the number of individuals, $r$, and the number of items, $t$, approach infinity. Sampling probability $p$ can be as small as $\max\{\log r/r, \log t/t\}$ up to a constant factor, which is a fundamental requirement to guarantee the connection of the sampling graph by the theory of the Erdős\textendash Rényi graph. The key technique behind this significant advancement is a powerful leave-one-out method for the Rasch model. We further establish the asymptotical normality of the MLE by using a simple matrix to approximate the inverse of the Fisher information matrix. The theoretical results are corroborated by simulation studies and an analysis of a large item-response dataset.

math.ST↗

arXiv2024

Reprint: a randomized extrapolation based on principal components for data augmentation

Data scarcity and data imbalance have attracted a lot of attention in many fields. Data augmentation, explored as an effective approach to tackle them, can improve the robustness and efficiency of classification models by generating new samples. This paper presents REPRINT, a simple and effective hidden-space data augmentation method for imbalanced data classification. Given hidden-space representations of samples in each class, REPRINT extrapolates, in a randomized fashion, augmented examples for target class by using subspaces spanned by principal components to summarize distribution structure of both source and target class. Consequently, the examples generated would diversify the target while maintaining the original geometry of target distribution. Besides, this method involves a label refinement component which allows to synthesize new soft labels for augmented examples. Compared with different NLP data augmentation approaches under a range of data imbalanced scenarios on four text classification benchmark, REPRINT shows prominent improvements. Moreover, through comprehensive ablation studies, we show that label refinement is better than label-preserving for augmented examples, and that our method suggests stable and consistent improvements in terms of suitable choices of principal components. Moreover, REPRINT is appealing for its easy-to-use since it contains only one hyperparameter determining the dimension of subspace and requires low computational resource.

cs.CL↗

arXiv2024

Spectroscopy and modeling of $^{171}$Yb Rydberg states for high-fidelity two-qubit gates

Highly excited Rydberg states and their interactions play an important role in quantum computing and simulation. These properties can be predicted accurately for alkali atoms with simple Rydberg level structures. However, an extension of these methods to more complex atoms such as alkaline-earth atoms has not been demonstrated or experimentally validated. Here, we present multichannel quantum defect (MQDT) models for highly excited $^{174}$Yb and $^{171}$Yb Rydberg states with $L \leq 2$. The models are developed using a combination of existing literature data and new, high-precision laser and microwave spectroscopy in an atomic beam, and validated by detailed comparison with experimentally measured Stark shifts and magnetic moments. We then use these models to compute interaction potentials between two Yb atoms, and find excellent agreement with direct measurements in an optical tweezer array. From the computed interaction potential, we identify an anomalous Förster resonance that likely degraded the fidelity of previous entangling gates in $^{171}$Yb using $F=3/2$ Rydberg states. We then identify a more suitable $F=1/2$ state, and achieve a state-of-the-art controlled-Z gate fidelity of $F=0.994(1)$, with the remaining error fully explained by known sources. This work establishes a solid foundation for the continued development of quantum computing, simulation and entanglement-enhanced metrology with Yb neutral atom arrays.

physics.atom-ph↗

arXiv2024

Lines of Bound States in the Continuum in a Phononic Crystal Slab

We demonstrate that bound states in the continuum (BICs) form continuous lines along high-symmetry directions of momentum space in a simple phononic crystal slab. Contrary to common sense, these BICs are symmetry-protected (SP) BICs not only at the center of the Brillouin zone (gamma point) but also off the gamma point. We utilize numerical simulations, a group theory method, and a mode expansion method to comprehensively understand the formation of the BICs. It is revealed that these BICs correspond to phase singularity lines of far-field radiation, and the phase winding number can be defined as a topological index. This makes the BICs topologically protected and robust to any parameter variation that maintains periodicity and rotational symmetry. Finally, the generation of the BICs lines is experimentally demonstrated.

cond-mat.mtrl-sci↗

arXiv2024

Emergent Universal Quench Dynamics in Randomly Interacting Spin Models

Universality often emerges in low-energy equilibrium physics of quantum many-body systems, despite their microscopic complexity and variety. Recently, there has been a growing interest in studying far-from-equilibrium dynamics of quantum many-body systems. Such dynamics usually involves highly excited states beyond the traditional low-energy theory description. Whether universal behaviors can also emerge in such non-equilibrium dynamics is a central issue at the frontier of quantum dynamics. Here we report the experimental observation of universal dynamics by monitoring the spin depolarization process in a solid-state NMR system described by an ensemble of randomly interacting spins. The spin depolarization can be related to temporal spin-spin correlation functions at high temperatures. We discover a remarkable phenomenon that these correlation functions obey a universal functional form. This experimental fact helps us identify the dominant interacting processes in the spin depolarization dynamics that lead to this universality. Our observation demonstrates the existence of universality even in non-equilibrium dynamics at high temperatures, thereby complementing the well-established universality in low-energy physics.

cond-mat.str-el↗

arXiv2024

Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection

Adversarial training methods commonly generate independent initial perturbation for adversarial samples from a simple uniform distribution, and obtain the training batch for the classifier without selection. In this work, we propose a simple yet effective training framework called Batch-in-Batch (BB) to enhance models robustness. It involves specifically a joint construction of initial values that could simultaneously generates $m$ sets of perturbations from the original batch set to provide more diversity for adversarial samples; and also includes various sample selection strategies that enable the trained models to have smoother losses and avoid overconfident outputs. Through extensive experiments on three benchmark datasets (CIFAR-10, SVHN, CIFAR-100) with two networks (PreActResNet18 and WideResNet28-10) that are used in both the single-step (Noise-Fast Gradient Sign Method, N-FGSM) and multi-step (Projected Gradient Descent, PGD-10) adversarial training, we show that models trained within the BB framework consistently have higher adversarial accuracy across various adversarial settings, notably achieving over a 13% improvement on the SVHN dataset with an attack radius of 8/255 compared to the N-FGSM baseline model. Furthermore, experimental analysis of the efficiency of both the proposed initial perturbation method and sample selection strategies validates our insights. Finally, we show that our framework is cost-effective in terms of computational resources, even with a relatively large value of $m$.

cs.LG↗

arXiv2024

Open-Vocabulary Object Detection via Neighboring Region Attention Alignment

The nature of diversity in real-world environments necessitates neural network models to expand from closed category settings to accommodate novel emerging categories. In this paper, we study the open-vocabulary object detection (OVD), which facilitates the detection of novel object classes under the supervision of only base annotations and open-vocabulary knowledge. However, we find that the inadequacy of neighboring relationships between regions during the alignment process inevitably constrains the performance on recent distillation-based OVD strategies. To this end, we propose Neighboring Region Attention Alignment (NRAA), which performs alignment within the attention mechanism of a set of neighboring regions to boost the open-vocabulary inference. Specifically, for a given proposal region, we randomly explore the neighboring boxes and conduct our proposed neighboring region attention (NRA) mechanism to extract relationship information. Then, this interaction information is seamlessly provided into the distillation procedure to assist the alignment between the detector and the pre-trained vision-language models (VLMs). Extensive experiments validate that our proposed model exhibits superior performance on open-vocabulary benchmarks.

cs.CV↗

arXiv2023

MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Driving

Instance segmentation is a fundamental research in computer vision, especially in autonomous driving. However, manual mask annotation for instance segmentation is quite time-consuming and costly. To address this problem, some prior works attempt to apply weakly supervised manner by exploring 2D or 3D boxes. However, no one has ever successfully segmented 2D and 3D instances simultaneously by only using 2D box annotations, which could further reduce the annotation cost by an order of magnitude. Thus, we propose a novel framework called Multimodal Weakly Supervised Instance Segmentation (MWSIS), which incorporates various fine-grained label generation and correction modules for both 2D and 3D modalities to improve the quality of pseudo labels, along with a new multimodal cross-supervision approach, named Consistency Sparse Cross-modal Supervision (CSCS), to reduce the inconsistency of multimodal predictions by response distillation. Particularly, transferring the 3D backbone to downstream tasks not only improves the performance of the 3D detectors, but also outperforms fully supervised instance segmentation with only 5% fully supervised annotations. On the Waymo dataset, the proposed framework demonstrates significant improvements over the baseline, especially achieving 2.59% mAP and 12.75% mAP increases for 2D and 3D instance segmentation tasks, respectively. The code is available at https://github.com/jiangxb98/mwsis-plugin.

cs.CV↗

arXiv2023

Frame Change Technique for Phase Transient Cancellation

The precise control of complex quantum mechanical systems can unlock applications ranging from quantum simulation to quantum computation. Controlling strongly interacting many-body systems often relies on Floquet Hamiltonian engineering that is achieved by fast switching between Hamiltonian primitives via external control. For example, in our solid-state NMR system, we perform quantum simulation by modulating the natural Hamiltonian with control pulses. As the Floquet heating errors scale with the interpulse delay, $δt$, it is favorable to keep $δt$ as short as possible, forcing our control pulses to be short duration and high power. Additionally, high-power pulses help to minimize undesirable evolution from occurring during the duration of the pulse. However, such pulses introduce an appreciable phase-transient control error, a form of unitary error. In this work, we detail our ability to diagnose the error, calibrate its magnitude, and correct it for $π/2$-pulses of arbitrary phase. We demonstrate the improvements gained by correcting for the phase transient error, using a method which we call the ``frame-change technique'', in a variety of experimental settings of interest. Given that the correction mechanism adds no real control overhead, we recommend that any resonance probe be checked for these phase transient control errors, and correct them using the frame-change technique.

quant-ph↗

arXiv2023

Locate before Answering: Answer Guided Question Localization for Video Question Answering

Video question answering (VideoQA) is an essential task in vision-language understanding, which has attracted numerous research attention recently. Nevertheless, existing works mostly achieve promising performances on short videos of duration within 15 seconds. For VideoQA on minute-level long-term videos, those methods are likely to fail because of lacking the ability to deal with noise and redundancy caused by scene changes and multiple actions in the video. Considering the fact that the question often remains concentrated in a short temporal range, we propose to first locate the question to a segment in the video and then infer the answer using the located segment only. Under this scheme, we propose "Locate before Answering" (LocAns), a novel approach that integrates a question locator and an answer predictor into an end-to-end model. During the training phase, the available answer label not only serves as the supervision signal of the answer predictor, but also is used to generate pseudo temporal labels for the question locator. Moreover, we design a decoupled alternative training strategy to update the two modules separately. In the experiments, LocAns achieves state-of-the-art performance on two modern long-term VideoQA datasets NExT-QA and ActivityNet-QA, and its qualitative examples show the reliable performance of the question localization.

cs.CV↗

arXiv2023

A scaled local gate controller for optically addressed qubits

Scalable classical controllers are a key component of future fault-tolerant quantum computers. Neutral atom quantum computers leverage commercially available optoelectronic devices for generating large-scale tweezer arrays and performing parallel readout, but implementing massively parallel, locally-addressed gate operations is an open challenge. In this work, we demonstrate an optical modulator system based on off-the-shelf components, which can generate a two-dimensional array of over 10,000 focused spots with uniform frequency and amplitude, and switching them on and off individually in arbitrary configurations at rates of up to 43 kHz. Through careful control of aberrations, the modulator achieves an extinction ratio of 46 dB, and nearest-neighbor crosstalk of $-44$ dB with a beam spacing of 4.6 waists. The underlying components can operate at wavelengths from the UV to the NIR, and sustain high laser intensities. This approach is suitable for local addressing of gates with low cross-talk error rates in any optically addressed qubit platform, including neutral atoms, trapped ions, or solid-state atomic defects.

physics.atom-ph↗

arXiv2023

Complete mode conversion for elastic waves reflected by elastic metamaterial slab with double hexapole resonances

In this study, we investigate the phenomenon of mode conversion in elastic bulk waves using coupled hexapole resonances. A metamaterial slab is proposed enabling the complete conversion between longitudinal and transverse modes. Each unit of the elastic metamaterial slab comprises a pair of scatterers, and their relative direction is oriented at an oblique angle. The interaction between the coupled hexapoles and the background results in oblique displacements, which are responsible for the mode conversion. Moreover, this conversion exhibits a broader frequency range compared to the quadrupole resonance. This innovative design significantly broadens the range of possibilities for developing mode-converting metamaterials.

physics.app-ph↗