arXiv subjects

Yupei Wang

Publications and source records attributed to Yupei Wang.

At least 19 recordsLinked to original sources

arXiv2025

MM-DETR: An Efficient Multimodal Detection Transformer with Mamba-Driven Dual-Granularity Fusion and Frequency-Aware Modality Adapters

Multimodal remote sensing object detection aims to achieve more accurate and robust perception under challenging conditions by fusing complementary information from different modalities. However, existing approaches that rely on attention-based or deformable convolution fusion blocks still struggle to balance performance and lightweight design. Beyond fusion complexity, extracting modality features with shared backbones yields suboptimal representations due to insufficient modality-specific modeling, whereas dual-stream architectures nearly double the parameter count, ultimately limiting practical deployment. To this end, we propose MM-DETR, a lightweight and efficient framework for multimodal object detection. Specifically, we propose a Mamba-based dual granularity fusion encoder that reformulates global interaction as channel-wise dynamic gating and leverages a 1D selective scan for efficient cross-modal modeling with linear complexity. Following this design, we further reinterpret multimodal fusion as a modality completion problem. A region-aware 2D selective scanning completion branch is introduced to recover modality-specific cues, supporting fine-grained fusion along a bidirectional pyramid pathway with minimal overhead. To further reduce parameter redundancy while retaining strong feature extraction capability, a lightweight frequency-aware modality adapter is inserted into the shared backbone. This adapter employs a spatial-frequency co-expert structure to capture modality-specific cues, while a pixel-wise router dynamically balances expert contributions for efficient spatial-frequency fusion. Extensive experiments conducted on four multimodal benchmark datasets demonstrate the effectiveness and generalization capability of the proposed method.

cs.CV↗

arXiv2025

Does Visual Grounding Enhance the Understanding of Embodied Knowledge in Large Language Models?

Despite significant progress in multimodal language models (LMs), it remains unclear whether visual grounding enhances their understanding of embodied knowledge compared to text-only models. To address this question, we propose a novel embodied knowledge understanding benchmark based on the perceptual theory from psychology, encompassing visual, auditory, tactile, gustatory, olfactory external senses, and interoception. The benchmark assesses the models' perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. By comparing 30 state-of-the-art LMs, we surprisingly find that vision-language models (VLMs) do not outperform text-only models in either task. Moreover, the models perform significantly worse in the visual dimension compared to other sensory dimensions. Further analysis reveals that the vector representations are easily influenced by word form and frequency, and the models struggle to answer questions involving spatial perception and reasoning. Our findings underscore the need for more effective integration of embodied knowledge in LMs to enhance their understanding of the physical world.

cs.CL↗

arXiv2025

VFM-Guided Semi-Supervised Detection Transformer under Source-Free Constraints for Remote Sensing Object Detection

Unsupervised domain adaptation methods have been widely explored to bridge domain gaps. However, in real-world remote-sensing scenarios, privacy and transmission constraints often preclude access to source domain data, which limits their practical applicability. Recently, Source-Free Object Detection (SFOD) has emerged as a promising alternative, aiming at cross-domain adaptation without relying on source data, primarily through a self-training paradigm. Despite its potential, SFOD frequently suffers from training collapse caused by noisy pseudo-labels, especially in remote sensing imagery with dense objects and complex backgrounds. Considering that limited target domain annotations are often feasible in practice, we propose a Vision foundation-Guided DEtection TRansformer (VG-DETR), built upon a semi-supervised framework for SFOD in remote sensing images. VG-DETR integrates a Vision Foundation Model (VFM) into the training pipeline in a "free lunch" manner, leveraging a small amount of labeled target data to mitigate pseudo-label noise while improving the detector's feature-extraction capability. Specifically, we introduce a VFM-guided pseudo-label mining strategy that leverages the VFM's semantic priors to further assess the reliability of the generated pseudo-labels. By recovering potentially correct predictions from low-confidence outputs, our strategy improves pseudo-label quality and quantity. In addition, a dual-level VFM-guided alignment method is proposed, which aligns detector features with VFM embeddings at both the instance and image levels. Through contrastive learning among fine-grained prototypes and similarity matching between feature maps, this dual-level alignment further enhances the robustness of feature representations against domain gaps. Extensive experiments demonstrate that VG-DETR achieves superior performance in source-free remote sensing detection tasks.

cs.CV↗

arXiv2025

Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection

Single-source domain generalization (SDG) in object detection aims to develop a detector using only source domain data that generalizes well to unseen target domains. Existing methods are primarily CNN-based and improve robustness through data augmentation combined with feature alignment. However, these methods are limited, as augmentation is only effective when the synthetic distribution approximates that of unseen domains, thus failing to ensure generalization across diverse scenarios. While DEtection TRansformer (DETR) has shown strong generalization in domain adaptation due to global context modeling, its potential for SDG remains underexplored. To this end, we propose Style-Adaptive DEtection TRansformer (SA-DETR), a DETR-based detector tailored for SDG. SA-DETR introduces an online domain style adapter that projects the style representation of unseen domains into the source domain via a dynamic memory bank. This bank self-organizes into diverse style prototypes and is continuously updated under a test-time adaptation framework, enabling effective style rectification. Additionally, we design an object-aware contrastive learning module to promote extraction of domain-invariant features. By applying gating masks that constrain contrastive learning in both spatial and semantic dimensions, this module facilitates instance-level cross-domain contrast and enhances generalization. Extensive experiments across five distinct weather scenarios demonstrate that SA-DETR consistently outperforms existing methods in both detection accuracy and domain generalization capability.

cs.CV↗

arXiv2025

Decoupled Global-Local Alignment for Improving Compositional Understanding

Contrastive Language-Image Pre-training (CLIP) has achieved success on multiple downstream tasks by aligning image and text modalities. However, the nature of global contrastive learning limits CLIP's ability to comprehend compositional concepts, such as relations and attributes. Although recent studies employ global hard negative samples to improve compositional understanding, these methods significantly compromise the model's inherent general capabilities by forcibly distancing textual negative samples from images in the embedding space. To overcome this limitation, we introduce a Decoupled Global-Local Alignment (DeGLA) framework that improves compositional understanding while substantially mitigating losses in general capabilities. To optimize the retention of the model's inherent capabilities, we incorporate a self-distillation mechanism within the global alignment process, aligning the learnable image-text encoder with a frozen teacher model derived from an exponential moving average. Under the constraint of self-distillation, it effectively mitigates the catastrophic forgetting of pretrained knowledge during fine-tuning. To improve compositional understanding, we first leverage the in-context learning capability of Large Language Models (LLMs) to construct about 2M high-quality negative captions across five types. Subsequently, we propose the Image-Grounded Contrast (IGC) loss and Text-Grounded Contrast (TGC) loss to enhance vision-language compositionally. Extensive experimental results demonstrate the effectiveness of the DeGLA framework. Compared to previous state-of-the-art methods, DeGLA achieves an average enhancement of 3.5% across the VALSE, SugarCrepe, and ARO benchmarks. Concurrently, it obtains an average performance improvement of 13.0% on zero-shot classification tasks across eleven datasets. Our code will be released at https://github.com/xiaoxing2001/DeGLA

cs.CV↗

arXiv2025

Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation

Parameter-Efficient Fine-Tuning (PEFT) is a technique that allows us to adapt powerful Foundation Models (FMs) to diverse downstream tasks while preserving and unleashing their inherent capabilities. However, we have observed that existing PEFT methods, which are often designed with natural imagery in mind, struggle when applied to Remote Sensing (RS) scenarios. This is primarily due to their inability to handle artifact influences, a problem particularly severe in RS image features. To tackle this challenge, we introduce Earth-Adapter, the first PEFT method specifically designed for RS artifacts conquering. Earth-Adapter introduces a novel Mixture of Frequency Adaptation process that combines a Mixture of Adapter (MoA) with Discrete Fourier Transformation (DFT). By utilizing DFT, Earth-Adapter can decompose features into different frequency components, precisely separating artifacts from original features. The MoA then dynamically assigns weights to each adapter expert, allowing for the combination of features across various frequency domains. These simple-yet-effective approaches enable Earth-Adapter to more efficiently overcome the disturbances caused by artifacts than previous PEFT methods, significantly enhancing the FMs' performance on RS scenarios. Experiments on Domain Adaptation (DA), and Domain Generalization (DG) semantic segmentation benchmarks showcase the Earth-Adapter's effectiveness. Compared with baseline Rein, Earth-Adapter significantly improves 9.0% mIoU in DA and 3.1% mIoU in DG benchmarks. Our code will be released at https://github.com/VisionXLab/Earth-Adapter.

cs.CV↗

arXiv2025

Heterogeneous-free narrow linewidth semiconductor laser with optical injection locking

Narrow linewidth lasers are indispensable for coherent optical systems, including communications, metrology, and sensing. Although compact semiconductor lasers with narrow linewidths and low noise have been demonstrated, their spectral purity typically relies on hybrid or heterogeneous external cavity feedback. Here, we present a theoretical and experimental demonstration of a heterogeneous free optical injection locking (HF OIL) semiconductor laser. By integrating a topological interface state extended (TISE) laser with a micro ring resonator (MRR) on an AlGaInAs multiple quantum well platform,we achieve monolithic photon injection and phase locking, thereby reducing the optical linewidth. We fabricated and characterized a 1550 nm sidewall HF OIL laser, achieving stable single mode operation over a broad current range (65 to 300 mA) and a side mode suppression ratio (SMSR) over 50 dB. Under injection locking, the devices Voigt fitted linewidth narrowed from over 1.7 MHz (free running) to 4.2 kHz, representing a three order of magnitude improvement over conventional distributed feedback lasers. The intrinsic linewidth of 1.4 kHz is measured by correlated delayed self-heterodyne frequency noise power spectrum density (FN PSD) method. Moreover, the HF OIL laser demonstrated high phase stability and the ability to transition from a random phased to a phase locked state. These results underscore the potential of HF-OIL lasers in advancing coherent optical communications and phase encoders in quantum key distribution (QKD) systems.

physics.optics↗

arXiv2024

Narrow Linewidth Laser Based on Extended Topological Interface States in One-Dimensional Photonic Crystals

Recent advances in topological one-dimensional photonic crystal concepts have enabled the development of robust light-emitting devices by incorporating a topological interface state (TIS) at the cavity center. In this study, we theoretically and experimentally demonstrate a one-dimensional TIS-extended photonic crystal (1D-TISE-PC) structure. By integrating a linearly dispersive zero-index one-dimensional photonic crystal structure with a four-phase shift sampled grating, photons propagate along the cavity without phase differences, enhancing the robustness to material variations and extending the TIS. Our findings indicate that extending the TIS promotes a more uniform photon distribution along the laser cavity and mitigates the spatial hole burning (SHB) effect. We fabricated and characterized a 1550 nm sidewall 1D-TISE-PC semiconductor laser, achieving stable single-mode operation across a wide current range from 60 to 420 mA, with a side-mode suppression ratio of 50 dB. The 1D-TISE-PC structure exhibited a linewidth narrowing effect to approximately 150 kHz Lorentzian linewidth. Utilizing reconstruction equivalent-chirp technology for the 4PS sampled grating enabled precise wavelength control in 1D-TISE-PC laser arrays, achieving a wavelength spacing of 0.796 nm +- 0.003 nm. We show that the TIS still exists in the TISE cavity and topological protection is preserved. Its mode extension characteristics mitigate the SHB so narrows the linewidth. We argue that the design simplicity and improvement of the fabrication tolerance make this architecture suitable for high-power and narrow-linewidth semiconductor lasers development.

physics.optics↗

arXiv2024

Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals

While current Automated Essay Scoring (AES) methods demonstrate high scoring agreement with human raters, their decision-making mechanisms are not fully understood. Our proposed method, using counterfactual intervention assisted by Large Language Models (LLMs), reveals that BERT-like models primarily focus on sentence-level features, whereas LLMs such as GPT-3.5, GPT-4 and Llama-3 are sensitive to conventions & accuracy, language complexity, and organization, indicating a more comprehensive rationale alignment with scoring rubrics. Moreover, LLMs can discern counterfactual interventions when giving feedback on essays. Our approach improves understanding of neural AES methods and can also apply to other domains seeking transparency in model-driven decisions.

cs.CL↗

arXiv2024

DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment

Object detectors frequently encounter significant performance degradation when confronted with domain gaps between collected data (source domain) and data from real-world applications (target domain). To address this task, numerous unsupervised domain adaptive detectors have been proposed, leveraging carefully designed feature alignment techniques. However, these techniques primarily align instance-level features in a class-agnostic manner, overlooking the differences between extracted features from different categories, which results in only limited improvement. Furthermore, the scope of current alignment modules is often restricted to a limited batch of images, failing to learn the entire dataset-level cues, thereby severely constraining the detector's generalization ability to the target domain. To this end, we introduce a strong DETR-based detector named Domain Adaptive detection TRansformer (DATR) for unsupervised domain adaptation of object detection. Firstly, we propose the Class-wise Prototypes Alignment (CPA) module, which effectively aligns cross-domain features in a class-aware manner by bridging the gap between object detection task and domain adaptation task. Then, the designed Dataset-level Alignment Scheme (DAS) explicitly guides the detector to achieve global representation and enhance inter-class distinguishability of instance-level features across the entire dataset, which spans both domains, by leveraging contrastive learning. Moreover, DATR incorporates a mean-teacher based self-training framework, utilizing pseudo-labels generated by the teacher model to further mitigate domain bias. Extensive experimental results demonstrate superior performance and generalization capabilities of our proposed DATR in multiple domain adaptation scenarios. Code is released at https://github.com/h751410234/DATR.

cs.CV↗

arXiv2023

Isotropic gap formation, localization, and waveguiding in mesoscale Yukawa-potential amorphous structures

Amorphous photonic structures are mesoscopic optical structures described by electrical permittivity distributions with underlying spatial randomness. They offer a unique platform for studying a broad set of electromagnetic phenomena, including transverse Anderson localization, enhanced wave transport, and suppressed diffusion in random media. Despite this, at a more practical level, there is insufficient work on both understanding the nature of optical transport and the conditions conducive to vector-wave localization in these planar structures, as well as their potential applications to photonic nanodevices. In this study, we fill this gap by investigating experimentally and theoretically the characteristics of optical transport in a class of amorphous photonic structures and by demonstrating their use to some basic waveguiding nanostructures. We demonstrate that these 2-D structures have unique isotropic and asymmetric band gaps for in-plane propagation, controlled from first principles by varying the scattering strength and whose properties are elucidated by establishing an analogy between photon and carrier transport in amorphous semiconductors. We further observe Urbach band tails in these random structures and uncover their relation to frequency- and disorder-dependent Anderson-like localized modes through the modified Ioffe-Regel criterion and their mean free path - localization length character. Finally, we illustrate that our amorphous structures can serve as a versatile platform in which photonic devices such as disorder-localized waveguides can be readily implemented.

physics.optics↗

arXiv2023

MS-Net: A Multi-modal Self-supervised Network for Fine-Grained Classification of Aircraft in SAR Images

Synthetic aperture radar (SAR) imaging technology is commonly used to provide 24-hour all-weather earth observation. However, it still has some drawbacks in SAR target classification, especially in fine-grained classification of aircraft: aircrafts in SAR images have large intra-class diversity and inter-class similarity; the number of effective samples is insufficient and it's hard to annotate. To address these issues, this article proposes a novel multi-modal self-supervised network (MS-Net) for fine-grained classification of aircraft. Firstly, in order to entirely exploit the potential of multi-modal information, a two-sided path feature extraction network (TSFE-N) is constructed to enhance the image feature of the target and obtain the domain knowledge feature of text mode. Secondly, a contrastive self-supervised learning (CSSL) framework is employed to effectively learn useful label-independent feature from unbalanced data, a similarity per-ception loss (SPloss) is proposed to avoid network overfitting. Finally, TSFE-N is used as the encoder of CSSL to obtain the classification results. Through a large number of experiments, our MS-Net can effectively reduce the difficulty of classifying similar types of aircrafts. In the case of no label, the proposed algorithm achieves an accuracy of 88.46% for 17 types of air-craft classification task, which has pioneering significance in the field of fine-grained classification of aircraft in SAR images.

cs.CV↗

arXiv2023

All-optical control of topological valley transport in graphene metasurfaces

We demonstrate that the influence of Kerr effect on valley-Hall topological transport in graphene metasurfaces can be used to implement an all-optical switch. In particular, by taking advantage of the large Kerr coefficient of graphene, the index of refraction of a topologically-protected graphene metasurface can be tuned via a pump beam, which results in an optically controllable frequency shift of the photonic bands of the metasurface. This spectral variation can in turn be readily employed to control and switch the propagation of an optical signal in certain waveguide modes of the graphene metasurface. Importantly, our theoretical and computational analysis reveals that the threshold pump power needed to optically switch ON/OFF the signal is strongly dependent on the group velocity of the pump mode, especially when the device is operated in the slow-light regime. This study could open up new routes towards active photonic nanodevices whose underlying functionality stems from their topological characteristics.

physics.optics↗

arXiv2022

Optically controllable coupling between edge and topological interface modes of graphene metasurfaces

Nonlinear topological photonics has been attracting increasing research interest, as it provides an exciting photonic platform that combines the advantages of active all-optical control offered by nonlinear optics with the unique features of topological photonic systems, such as topologically-protected defect-immune light propagation. In this paper, we demonstrate that topological interface modes and trivial edge modes of a specially designed graphene metasurface can be coupled in a tunable and optically controllable manner, thus providing an efficient approach to transfer optical power to topologically protected states. This is achieved in a pump-signal configuration, in which an optical pump propagating in a bulk mode of the metasurface is employed to tune the band structure of the photonic system and, consequently, the coupling coefficient and wave-vector mismatch between edge and topological interface modes. This tunable coupling mechanism is particularly efficient due to the large Kerr coefficient of graphene. Importantly, we demonstrate that the required pump power can be significantly reduced if the optical device is operated in the slow-light regime. We perform our analysis using both \textit{ab initio} full-wave simulations and a coupled-mode theory that captures the main physics of this active coupler and observe a good agreement between the two approaches. This work may lead to the design of active topological photonic devices with new or improved functionality.

physics.optics↗

arXiv2020

Topological valley plasmon transport in bilayer graphene metasurfaces for sensing applications

Topologically protected plasmonic modes located inside topological bandgaps are attracting increasing attention, chiefly due to their robustness against disorder-induced backscattering. Here, we introduce a bilayer graphene metasurface that possesses plasmonic topological valley interface modes when the mirror symmetry of the metasurface is broken by horizontally shifting the lattice of holes of the top layer of the two freestanding graphene layers in opposite directions. In this configuration, light propagation along the domain-wall interface of the bilayer graphene metasurface shows unidirectional features. Moreover, we have designed a molecular sensor based on the topological properties of this metasurface using the fact that the Fermi energy of graphene varies upon chemical doping. This effect induces strong variation of the transmission of the topological guided modes, which can be employed as the underlying working principle of gas sensing devices. Our work opens up new ways of developing robust integrated plasmonic devices for molecular sensing.

physics.optics↗

arXiv2019

SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

Recently, proposal-free instance segmentation has received increasing attention due to its concise and efficient pipeline. Generally, proposal-free methods generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. We argue that treating these two sub-tasks separately is suboptimal. In fact, employing multiple separate modules significantly reduces the potential for application. The mutual benefits between the two complementary sub-tasks are also unexplored. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on a pixel-pair affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. The affinity pyramid can also be jointly learned with the semantic class labeling and achieve mutual benefits. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition module is presented to sequentially generate instances from coarse to fine. Unlike previous time-consuming graph partition methods, this module achieves $5\times$ speedup and 9% relative improvement on Average-Precision (AP). Our approach achieves state-of-the-art results on the challenging Cityscapes dataset.

cs.CV↗

arXiv2018

Deep Crisp Boundaries: From Boundaries to Higher-level Tasks

Edge detection has made significant progress with the help of deep Convolutional Networks (ConvNet). These ConvNet based edge detectors have approached human level performance on standard benchmarks. We provide a systematical study of these detectors' outputs. We show that the detection results did not accurately localize edge pixels, which can be adversarial for tasks that require crisp edge inputs. As a remedy, we propose a novel refinement architecture to address the challenging problem of learning a crisp edge detector using ConvNet. Our method leverages a top-down backward refinement pathway, and progressively increases the resolution of feature maps to generate crisp edges. Our results achieve superior performance, surpassing human accuracy when using standard criteria on BSDS500, and largely outperforming state-of-the-art methods when using more strict criteria. More importantly, we demonstrate the benefit of crisp edge maps for several important applications in computer vision, including optical flow estimation, object proposal generation and semantic segmentation.

cs.CV↗

arXiv2017

Field-free perpendicular magnetization switching through domain wall motion in Pt/Co/Cr racetracks by spin orbit torques with the assistance of accompanying Joule heating effect

Heavy metal/ferromagnetic layers with perpendicular magnetic anisotropy (PMA) have potential applications for high-density information storage in racetrack memories and nonvolatile magnetic random access memories. Writing and erasing of information in these devices are carried out by domain wall (DW) motion and deterministic magnetization switching via electric current generated spin orbital torques (SOTs) with an assistance of in-plane bias field to break the symmetry. Improvements in energy efficiency could be obtained when the switching of perpendicular magnetization is controlled by an electric current generated SOTs without the in-plane bias fields. Here, we report on reversible electric-current-driven magnetization switching through DW motion in Pt/Co/Cr trilayers with PMA at room temperature due to the formation of homochiral Neel-type domain, in which an in-plane effective Dzyaloshinskii-Moriya interaction field exists. Fully deterministic magnetic magnetization switching in this trilayers is based on the enhancement of SOTs from a dedicated design of Pt/Co/Cr structures with two heavy metals Pt and Cr which show the opposite sign of spin Hall angles. We also demonstrated that the simultaneously accompanying Joule heating effect also plays a key role for field-free magnetization switching through the decrease of the propagation field.

physics.app-ph↗