arXiv subjects

Lei Tan

Publications and source records attributed to Lei Tan.

At least 37 records · Page 2Linked to original sources

arXiv2024

Knowing Where to Focus: Attention-Guided Alignment for Text-based Person Search

In the realm of Text-Based Person Search (TBPS), mainstream methods aim to explore more efficient interaction frameworks between text descriptions and visual data. However, recent approaches encounter two principal challenges. Firstly, the widely used random-based Masked Language Modeling (MLM) considers all the words in the text equally during training. However, massive semantically vacuous words ('with', 'the', etc.) be masked fail to contribute efficient interaction in the cross-modal MLM and hampers the representation alignment. Secondly, manual descriptions in TBPS datasets are tedious and inevitably contain several inaccuracies. To address these issues, we introduce an Attention-Guided Alignment (AGA) framework featuring two innovative components: Attention-Guided Mask (AGM) Modeling and Text Enrichment Module (TEM). AGM dynamically masks semantically meaningful words by aggregating the attention weight derived from the text encoding process, thereby cross-modal MLM can capture information related to the masked word from text context and images and align their representations. Meanwhile, TEM alleviates low-quality representations caused by repetitive and erroneous text descriptions by replacing those semantically meaningful words with MLM's prediction. It not only enriches text descriptions but also prevents overfitting. Extensive experiments across three challenging benchmarks demonstrate the effectiveness of our AGA, achieving new state-of-the-art results with Rank-1 accuracy reaching 78.36%, 67.31%, and 67.4% on CUHK-PEDES, ICFG-PEDES, and RSTPReid, respectively.

cs.CV↗

arXiv2024

Learning on Less: Constraining Pre-trained Model Learning for Generalizable Diffusion-Generated Image Detection

Diffusion Models enable realistic image generation, raising the risk of misinformation and eroding public trust. Currently, detecting images generated by unseen diffusion models remains challenging due to the limited generalization capabilities of existing methods. To address this issue, we rethink the effectiveness of pre-trained models trained on large-scale, real-world images. Our findings indicate that: 1) Pre-trained models can cluster the features of real images effectively. 2) Models with pre-trained weights can approximate an optimal generalization solution at a specific training step, but it is extremely unstable. Based on these facts, we propose a simple yet effective training method called Learning on Less (LoL). LoL utilizes a random masking mechanism to constrain the model's learning of the unique patterns specific to a certain type of diffusion model, allowing it to focus on less image content. This leverages the inherent strengths of pre-trained weights while enabling a more stable approach to optimal generalization, which results in the extraction of a universal feature that differentiates various diffusion-generated images from real images. Extensive experiments on the GenImage benchmark demonstrate the remarkable generalization capability of our proposed LoL. With just 1% training data, LoL significantly outperforms the current state-of-the-art, achieving a 13.6% improvement in average ACC across images generated by eight different models.

cs.CV↗

arXiv2024

Image Forgery Localization via Guided Noise and Multi-Scale Feature Aggregation

Image Forgery Localization (IFL) technology aims to detect and locate the forged areas in an image, which is very important in the field of digital forensics. However, existing IFL methods suffer from feature degradation during training using multi-layer convolutions or the self-attention mechanism, and perform poorly in detecting small forged regions and in robustness against post-processing. To tackle these, we propose a guided and multi-scale feature aggregated network for IFL. Spectifically, in order to comprehensively learn the noise feature under different types of forgery, we develop an effective noise extraction module in a guided way. Then, we design a Feature Aggregation Module (FAM) that uses dynamic convolution to adaptively aggregate RGB and noise features over multiple scales. Moreover, we propose an Atrous Residual Pyramid Module (ARPM) to enhance features representation and capture both global and local features using different receptive fields to improve the accuracy and robustness of forgery localization. Expensive experiments on 5 public datasets have shown that our proposed model outperforms several the state-of-the-art methods, specially on small region forged image.

cs.CV↗

arXiv2024

RLE: A Unified Perspective of Data Augmentation for Cross-Spectral Re-identification

This paper makes a step towards modeling the modality discrepancy in the cross-spectral re-identification task. Based on the Lambertain model, we observe that the non-linear modality discrepancy mainly comes from diverse linear transformations acting on the surface of different materials. From this view, we unify all data augmentation strategies for cross-spectral re-identification by mimicking such local linear transformations and categorizing them into moderate transformation and radical transformation. By extending the observation, we propose a Random Linear Enhancement (RLE) strategy which includes Moderate Random Linear Enhancement (MRLE) and Radical Random Linear Enhancement (RRLE) to push the boundaries of both types of transformation. Moderate Random Linear Enhancement is designed to provide diverse image transformations that satisfy the original linear correlations under constrained conditions, whereas Radical Random Linear Enhancement seeks to generate local linear transformations directly without relying on external information. The experimental results not only demonstrate the superiority and effectiveness of RLE but also confirm its great potential as a general-purpose data augmentation for cross-spectral re-identification. The code is available at \textcolor{magenta}{\url{https://github.com/stone96123/RLE}}.

cs.CV↗

arXiv2024

TMFNet: Two-Stream Multi-Channels Fusion Networks for Color Image Operation Chain Detection

Image operation chain detection techniques have gained increasing attention recently in the field of multimedia forensics. However, existing detection methods suffer from the generalization problem. Moreover, the channel correlation of color images that provides additional forensic evidence is often ignored. To solve these issues, in this article, we propose a novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner. Specifically, we first propose a novel deep residual architecture without pooling in the spatial artifact stream for learning the global features representation of multi-channel correlation. Then, a set of filters is designed to aggregate the correlation information of multi-channels while capturing the low-level features in the noise residual stream. Subsequently, the high-level features are extracted by the deep residual model. Finally, features from the two streams are fed into a fusion module, to effectively learn richer discriminative representations of the operation chain. Extensive experiments show that the proposed method achieves state-of-the-art generalization ability while maintaining robustness to JPEG compression. The source code used in these experiments will be released at https://github.com/LeiTan-98/TMFNet.

cs.CV↗

arXiv2024

PartFormer: Awakening Latent Diverse Representation from Vision Transformer for Object Re-Identification

Extracting robust feature representation is critical for object re-identification to accurately identify objects across non-overlapping cameras. Although having a strong representation ability, the Vision Transformer (ViT) tends to overfit on most distinct regions of training data, limiting its generalizability and attention to holistic object features. Meanwhile, due to the structural difference between CNN and ViT, fine-grained strategies that effectively address this issue in CNN do not continue to be successful in ViT. To address this issue, by observing the latent diverse representation hidden behind the multi-head attention, we present PartFormer, an innovative adaptation of ViT designed to overcome the granularity limitations in object Re-ID tasks. The PartFormer integrates a Head Disentangling Block (HDB) that awakens the diverse representation of multi-head self-attention without the typical loss of feature richness induced by concatenation and FFN layers post-attention. To avoid the homogenization of attention heads and promote robust part-based feature learning, two head diversity constraints are imposed: attention diversity constraint and correlation diversity constraint. These constraints enable the model to exploit diverse and discriminative feature representations from different attention heads. Comprehensive experiments on various object Re-ID benchmarks demonstrate the superiority of the PartFormer. Specifically, our framework significantly outperforms state-of-the-art by 2.4\% mAP scores on the most challenging MSMT17 dataset.

cs.CV↗

arXiv2024

Highly Scalable Quantum Router with Frequency-Independent Scattering Spectra

Optical quantum routers play a crucial role in quantum networks and have been extensively studied in both theory and experiment, leading to significant advancements in their performance. However, these routers impose stringent requirements for achieving desired routing results, as the incident photon frequency must be in strict resonance with one or several specific frequencies. To address this challenge, we propose an efficient quantum router scheme composed of semi-infinite coupled-resonator waveguide (CRW) and a giant atom. The single-channel router scheme enables stable output with 100% transfer rate over the entire energy band of the CRW. Leveraging this intriguing result, we further propose a multi-channel router scheme that possesses high stability and universality, while also being capable of performing various functionalities. The complete physical explanation of the underlying mechanism for this intriguing result is also presented. We hope that quantum router with output results unaffected by the frequency of the incoming information carriers presents a more reliable solution for the implementation of quantum networks.

quant-ph↗

arXiv2024

Tip-leakage-flow excited unsteadiness and associated control

Tip leakage flow in turbomachinery inherently generates intense unsteady features, named self-excited unsteadiness, which has not been well understood. A Zonalised LES (ZLES) is employed around a linear cascade, with wall-modelled Large Eddy Simulation (LES) forced to be active in the tip region. The simulation is well validated and demonstrated the advantages of effectively reducing the computational effort while maintaining an equivalent prediction accuracy in the region of interest. The time-averaged and spatial-spectral characteristics of tip leakage vortex (TLV) structures are systematically discussed. The self-excited unsteady processes of TLV include unsteady vortex separation inside the tip gap, the tip leakage jet-mainstream interaction, the primary tip leakage vortex (PTLV) wandering motion and the induced separation near endwall. The Spectral Proper Orthogonal Decomposition (SPOD) is used to examine the dominant frequencies and their coherent structures. It is found that these unsteady features change from a single high-frequency mode to a multiple lower-frequencies mode due to the PTLV breakdown. The SPOD and correlation analyses reveal that the self-excited unsteadiness is mainly induced by the interactions between unsteady vortex separation, tip leakage jet, and mainstream. The associated unsteady fluctuations are convected along the tip leakage jet trajectory, causing the wandering motion of PTLV core. Based on the cause of the investigated unsteadiness, a micro-offset tip design is proposed and validated for effectively suppressing this unsteadiness, and associated turbulence generation and hence pressure fluctuations. This work improves the understanding of tip-leakage-flow dynamics and informs the control of the associated unsteady fluid oscillation and noise.

physics.flu-dyn↗

arXiv2024

Entanglement enhancement of two giant atoms with multiple connection points in bidirectional-chiral quantum waveguide-QED system

We study the entanglement generation of two giant atoms within a one-dimensional bidirectional-chiral waveguide quantum electrodynamics (QED) system, where the initial state of the two giant atoms are $|e_a,g_b\rangle $. Here, each giant atom is coupled to the waveguide through three connection points, with the configurations divided into five types based on the arrangement of coupling points between the giant atoms and the waveguide: separate, fully braided, partially braided, fully nested, and partially nested. We explore the entanglement generation process within each configuration in both nonchiral and chiral coupling cases. It is demonstrated that entanglement can be controlled as needed by either adjusting the phase shift or selecting different configurations. For nonchiral coupling, the entanglement of each configuration exhibits steady state properties attributable to the presence of dark state. In addition, we find that steady-state entanglement can be obtained at more phase shifts in certain configurations by increasing the number of coupling points between the giant atoms and the bidirectional waveguide. In the case of chiral coupling, the entanglement is maximally enhanced compared to the one of nonchiral case. Especially in fully braided configuration, the concurrence reaches its peak value 1, which is robust to chirality. We further show the influence of atomic initial states on the evolution of interatomic entanglement. Our scheme can be used for entanglement generation in chiral quantum networks of giant-atom waveguide-QED systems, with potential applications in quantum networks and quantum communications.

quant-ph↗

arXiv2024

Attention Disturbance and Dual-Path Constraint Network for Occluded Person Re-identification

Occluded person re-identification (Re-ID) aims to address the potential occlusion problem when matching occluded or holistic pedestrians from different camera views. Many methods use the background as artificial occlusion and rely on attention networks to exclude noisy interference. However, the significant discrepancy between simple background occlusion and realistic occlusion can negatively impact the generalization of the network. To address this issue, we propose a novel transformer-based Attention Disturbance and Dual-Path Constraint Network (ADP) to enhance the generalization of attention networks. Firstly, to imitate real-world obstacles, we introduce an Attention Disturbance Mask (ADM) module that generates an offensive noise, which can distract attention like a realistic occluder, as a more complex form of occlusion. Secondly, to fully exploit these complex occluded images, we develop a Dual-Path Constraint Module (DPC) that can obtain preferable supervision information from holistic images through dual-path interaction. With our proposed method, the network can effectively circumvent a wide variety of occlusions using the basic ViT baseline. Comprehensive experimental evaluations conducted on person re-ID benchmarks demonstrate the superiority of ADP over state-of-the-art methods.

cs.CV↗

arXiv2024

Prompt Decoupling for Text-to-Image Person Re-identification

Text-to-image person re-identification (TIReID) aims to retrieve the target person from an image gallery via a textual description query. Recently, pre-trained vision-language models like CLIP have attracted significant attention and have been widely utilized for this task due to their robust capacity for semantic concept learning and rich multi-modal knowledge. However, recent CLIP-based TIReID methods commonly rely on direct fine-tuning of the entire network to adapt the CLIP model for the TIReID task. Although these methods show competitive performance on this topic, they are suboptimal as they necessitate simultaneous domain adaptation and task adaptation. To address this issue, we attempt to decouple these two processes during the training stage. Specifically, we introduce the prompt tuning strategy to enable domain adaptation and propose a two-stage training approach to disentangle domain adaptation from task adaptation. In the first stage, we freeze the two encoders from CLIP and solely focus on optimizing the prompts to alleviate domain gap between the original training data of CLIP and downstream tasks. In the second stage, we maintain the fixed prompts and fine-tune the CLIP model to prioritize capturing fine-grained information, which is more suitable for TIReID task. Finally, we evaluate the effectiveness of our method on three widely used datasets. Compared to the directly fine-tuned approach, our method achieves significant improvements.

cs.CV↗

arXiv2023

Scientific Preparation for CSST: Classification of Galaxy and Nebula/Star Cluster Based on Deep Learning

The Chinese Space Station Telescope (abbreviated as CSST) is a future advanced space telescope. Real-time identification of galaxy and nebula/star cluster (abbreviated as NSC) images is of great value during CSST survey. While recent research on celestial object recognition has progressed, the rapid and efficient identification of high-resolution local celestial images remains challenging. In this study, we conducted galaxy and NSC image classification research using deep learning methods based on data from the Hubble Space Telescope. We built a Local Celestial Image Dataset and designed a deep learning model named HR-CelestialNet for classifying images of the galaxy and NSC. HR-CelestialNet achieved an accuracy of 89.09% on the testing set, outperforming models such as AlexNet, VGGNet and ResNet, while demonstrating faster recognition speeds. Furthermore, we investigated the factors influencing CSST image quality and evaluated the generalization ability of HR-CelestialNet on the blurry image dataset, demonstrating its robustness to low image quality. The proposed method can enable real-time identification of celestial images during CSST survey mission.

cs.CV↗

arXiv2023

Key Issues in Wireless Transmission for NTN-Assisted Internet of Things

Non-terrestrial networks (NTNs) have become appealing resolutions for seamless coverage in the next-generation wireless transmission, where a large number of Internet of Things (IoT) devices diversely distributed can be efficiently served. The explosively growing number of IoT devices brings a new challenge for massive connection. The long-distance wireless signal propagation in NTNs leads to severe path loss and large latency, where the accurate acquisition of channel state information (CSI) is another challenge, especially for fast-moving non-terrestrial base stations (NTBSs). Moreover, the scarcity of on-board resources of NTBSs is also a challenge for resource allocation. To this end, we investigate three key issues, where the existing schemes and emerging resolutions for these three key issues have been comprehensively presented. The first issue is to enable the massive connection by designing random access to establish the wireless link and multiple access to transmit data streams. The second issue is to accurately acquire CSI in various channel conditions by channel estimation and beam training, where orthogonal time frequency space modulation and dynamic codebooks are on focus. The third issue is to efficiently allocate the wireless resources, including power allocation, spectrum sharing, beam hopping, and beamforming. At the end of this article, some future research topics are identified.

eess.SP↗

arXiv2023

Study on many-body phases in Jaynes-Cummings-Hubbard arrays

Disorder in one-dimensional (1D) many-body systems emerges abundant phases such as many-body localization (MBL), and thermalization. However, it remains unclear regarding their existence and behavior within hybrid quantum systems. Here, based on a simple bosonic-spin hybrid model, as known as the Jaynes-Cummings-Hubbard (JCH) array, we investigate the effect of disorder comparing to the phenomena in the clean system with the variation of atom-photon coupling strength. By using the level-spacing ratio, entanglement entropy, and the properties of observable diagonal and off-diagonal matrix elements, we find that strong disorder results in the appearance of MBL phase in the JCH model that strongly violate eigenstate thermalization hypothesis (ETH), while a conditional prethermal behavior can exist in weak disorder or weak coupling regime. The conditional prethermal dynamics is based on the choice of initial product states. This work systematically reveals abundant many-body phases in the 1D JCH model and clarifies the discrepancies in the thermalization properties of systems with and without disorder.

quant-ph↗

arXiv2023

Directional router and controllable non-reciprocity transmission based on phase and pathway coherence

A multi-channel quantum router with four nodal cavities is constructed by two coupled-resonator waveguides and four single cavities. We can achieve directional routing by adjusting the probability of photon exiting from the specified port to close to 100% based on multiple pathways between the photon from the incident port to the outgoing port in this hybrid system. Under the effect of phase difference between two classical light fields, the mutual interference between different pathways can be adjusted to destructive interference or constructive interference, which lays the foundation for the increase and decrease of the routing probability. The influence of different parameter values on single photon routing probability is also studied. By studying the analytic formula of probability amplitude, we get the physical mechanism of exiting ports being closed under certain parameter conditions and the phase relationship between the backward transmission and the original direction transmission of photons. Furthermore the non-reciprocal transmission and directional routing beyond chiral coupling can also be realized, which provides new possibilities for the study of quantum routers and new insights for the study of photon transmission characteristics.

quant-ph↗

arXiv2023

Anomalous non-Hermitian skin effect: the topological inequivalence of skin modes versus point gap

Non-Hermitian skin effect, the localization of an extensive number of eigenstates at the ends of the system, has greatly expanded the frontier of physical laws. It has long been believed that the present of skin modes is equivalent to the topologically nontrivial point gap of complex eigenvalues under periodic boundary conditions, and vice versa. However, we find that this concomitance can be broken, i.e., the skin modes can be present or absent whereas the point gap is topologically trivial or nontrivial, respectively, named anomalous non-Hermitian skin effect. This anomalous phenomenon arises when the unidirectional hopping amplitudes leading to the decoupling-like behaviors among subsystems are emergence. The emergence of the anomalous non-Hermitian skin effect is accompanied by the mutations of the open boundary energy spectrum, whose structure exhibits the multifold exceptional point and can not be recovered by continuum bands. Moreover, an experimental setup using circuits is proposed to simulate this novel quantum effect. Our results reveal the topologically inequivalent between skin modes and point gap. This new effect not only can give a deeper understanding of non-Bloch theory and the critical phenomenon in non-Hermitian systems, but may also inspire new applications such as in the sensors field.

cond-mat.mes-hall↗

arXiv2023

Spectral Aware Softmax for Visible-Infrared Person Re-Identification

Visible-infrared person re-identification (VI-ReID) aims to match specific pedestrian images from different modalities. Although suffering an extra modality discrepancy, existing methods still follow the softmax loss training paradigm, which is widely used in single-modality classification tasks. The softmax loss lacks an explicit penalty for the apparent modality gap, which adversely limits the performance upper bound of the VI-ReID task. In this paper, we propose the spectral-aware softmax (SA-Softmax) loss, which can fully explore the embedding space with the modality information and has clear interpretability. Specifically, SA-Softmax loss utilizes an asynchronous optimization strategy based on the modality prototype instead of the synchronous optimization based on the identity prototype in the original softmax loss. To encourage a high overlapping between two modalities, SA-Softmax optimizes each sample by the prototype from another spectrum. Based on the observation and analysis of SA-Softmax, we modify the SA-Softmax with the Feature Mask and Absolute-Similarity Term to alleviate the ambiguous optimization during model training. Extensive experimental evaluations conducted on RegDB and SYSU-MM01 demonstrate the superior performance of the SA-Softmax over the state-of-the-art methods in such a cross-modality condition.

cs.CV↗

arXiv2023

Exploring Invariant Representation for Visible-Infrared Person Re-Identification

Cross-spectral person re-identification, which aims to associate identities to pedestrians across different spectra, faces a main challenge of the modality discrepancy. In this paper, we address the problem from both image-level and feature-level in an end-to-end hybrid learning framework named robust feature mining network (RFM). In particular, we observe that the reflective intensity of the same surface in photos shot in different wavelengths could be transformed using a linear model. Besides, we show the variable linear factor across the different surfaces is the main culprit which initiates the modality discrepancy. We integrate such a reflection observation into an image-level data augmentation by proposing the linear transformation generator (LTG). Moreover, at the feature level, we introduce a cross-center loss to explore a more compact intra-class distribution and modality-aware spatial attention to take advantage of textured regions more efficiently. Experiment results on two standard cross-spectral person re-identification datasets, i.e., RegDB and SYSU-MM01, have demonstrated state-of-the-art performance.

cs.CV↗