arXiv subjects

Kaijie Wu

Publications and source records attributed to Kaijie Wu.

5 recordsLinked to original sources

arXiv2026

Illuminating Visual Identity in Universal Multimodal Embeddings

Universal Multimodal Embeddings (UMEs) aim to unify various modalities and tasks into a shared representation space. In recent years, this field has witnessed substantial progress driven by the development of Multimodal Large Language Models (MLLMs). However, a crucial capability, visual identity discrimination, remains underexplored in existing UME methods, despite its critical role in a wide range of tasks, including instance retrieval, re-identification, and identity preservation in AI-generated content. To bridge this gap, we propose a unified formulation for visual identity discrimination~(VisID) and introduce $\textbf{MVEB}$ ($\textbf{M}$ultimodal $\textbf{V}$isual Identity $\textbf{E}$mbedding $\textbf{B}$enchmark), a large-scale benchmark curated from both real-world and synthetic datasets to support evaluation and training. Furthermore, we present a simple yet effective learning framework that jointly optimizes general multimodal and visual identity representations through a carefully designed identity-aware sampling mechanism. Extensive experiments demonstrate that our approach successfully endows UMEs with strong identity discrimination capability and maintains competitive general multimodal performance. We believe this work not only illuminates a critical yet neglected capability, but also takes a step toward more holistic universal multimodal embeddings. Code and data are available at \href{https://chrisclear3.github.io/MVEB}{MVEB}.

cs.CV↗

arXiv2024

Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs

The hallucination problem in multimodal large language models (MLLMs) remains a common issue. Although image tokens occupy a majority of the input sequence of MLLMs, there is limited research to explore the relationship between image tokens and hallucinations. In this paper, we analyze the distribution of attention scores for image tokens across each layer and head of the model, revealing an intriguing and common phenomenon: most hallucinations are closely linked to the pattern of attention sinks in the self-attention matrix of image tokens, where shallow layers exhibit dense attention sinks and deeper layers show sparse attention sinks. We further analyze the attention heads of different layers and find that heads with high-density attention sink in the image part play a positive role in alleviating hallucinations. In this paper, we propose a training-free method named \textcolor{red}{\textbf{E}}nhancing \textcolor{red}{\textbf{A}}ttention \textcolor{red}{\textbf{H}}eads (EAH), an approach designed to enhance the convergence of image tokens attention sinks in the shallow layers. EAH identifies the attention head that shows the vision sink in a shallow layer and extracts its attention matrix. This attention map is then broadcast to other heads in the layer, thereby strengthening the layer to pay more attention to the image itself. With extensive experiments, EAH shows significant hallucination-mitigating performance on different MLLMs and metrics, proving its effectiveness and generality.

cs.CV↗

arXiv2022

Industrial Scene Text Detection with Refined Feature-attentive Network

Detecting the marking characters of industrial metal parts remains challenging due to low visual contrast, uneven illumination, corroded character structures, and cluttered background of metal part images. Affected by these factors, bounding boxes generated by most existing methods locate low-contrast text areas inaccurately. In this paper, we propose a refined feature-attentive network (RFN) to solve the inaccurate localization problem. Specifically, we design a parallel feature integration mechanism to construct an adaptive feature representation from multi-resolution features, which enhances the perception of multi-scale texts at each scale-specific level to generate a high-quality attention map. Then, an attentive refinement network is developed by the attention map to rectify the location deviation of candidate boxes. In addition, a re-scoring mechanism is designed to select text boxes with the best rectified location. Moreover, we construct two industrial scene text datasets, including a total of 102156 images and 1948809 text instances with various character structures and metal parts. Extensive experiments on our dataset and four public datasets demonstrate that our proposed method achieves the state-of-the-art performance.

cs.CV↗

arXiv2017

Towards Reverse Engineering Reversible Logic

Reversible logic has two main properties. First, the number of inputs is equal to the number of outputs. Second, it implements a one-to-one mapping; i.e., one can reconstruct the inputs from the outputs. These properties enable its applications in building quantum computing architectures. In this paper, we study reverse engineering of reversible logic circuits, including reverse engineering of non-reversible functions embedded into reversible circuits. We propose the number of embeddings of non-reversible functions into a reversible circuit as the security metric for reverse engineering. We analyze the security benefits of automatic synthesis of reversible circuits. We use our proposed security metric to show that the functional synthesis approaches yield reversible circuits that are more resilient to reverse engineering than the structural synthesis approaches. Finally, we propose scrambling of the inputs and outputs of a reversible circuit to thwart reverse engineering.

cs.CR↗

arXiv2017

On the Difficulty of Inserting Trojans in Reversible Computing Architectures

Fabrication-less design houses outsource their designs to 3rd party foundries to lower fabrication cost. However, this creates opportunities for a rogue in the foundry to introduce hardware Trojans, which stay inactive most of the time and cause unintended consequences to the system when triggered. Hardware Trojans in traditional CMOS-based circuits have been studied and Design-for-Trust (DFT) techniques have been proposed to detect them. Different from traditional circuits in many ways, reversible circuits implement one-to-one, bijective input/output mappings. We will investigate the security implications of reversible circuits with a particular focus on susceptibility to hardware Trojans. We will consider inherently reversible circuits and non-reversible functions embedded in reversible circuits.

cs.CR↗