arXiv subjects

Jiuxin Cao

Publications and source records attributed to Jiuxin Cao.

26 records · Page 2Linked to original sources

arXiv2023

Text as Image: Learning Transferable Adapter for Multi-Label Classification

Pre-trained vision-language models have notably accelerated progress of open-world concept recognition. Their impressive zero-shot ability has recently been transferred to multi-label image classification via prompt tuning, enabling to discover novel labels in an open-vocabulary manner. However, this paradigm suffers from non-trivial training costs, and becomes computationally prohibitive for a large number of candidate labels. To address this issue, we note that vision-language pre-training aligns images and texts in a unified embedding space, making it potential for an adapter network to identify labels in visual modality while be trained in text modality. To enhance such cross-modal transfer ability, a simple yet effective method termed random perturbation is proposed, which enables the adapter to search for potential visual embeddings by perturbing text embeddings with noise during training, resulting in better performance in visual modality. Furthermore, we introduce an effective approach to employ large language models for multi-label instruction-following text generation. In this way, a fully automated pipeline for visual label recognition is developed without relying on any manual data. Extensive experiments on public benchmarks show the superiority of our method in various multi-label classification tasks.

cs.CV↗

arXiv2023

Multi-hop Commonsense Knowledge Injection Framework for Zero-Shot Commonsense Question Answering

Commonsense question answering (QA) research requires machines to answer questions based on commonsense knowledge. However, this research requires expensive labor costs to annotate data as the basis of research, and models that rely on fine-tuning paradigms only apply to specific tasks, rather than learn a general commonsense reasoning ability. As a more robust method, zero-shot commonsense question answering shows a good prospect. The current zero-shot framework tries to convert triples in commonsense knowledge graphs (KGs) into QA-form samples as the pre-trained data source to incorporate commonsense knowledge into the model. However, this method ignores the multi-hop relationship in the KG, which is also an important central problem in commonsense reasoning. In this paper, we propose a novel multi-hop commonsense knowledge injection framework. Specifically, it explores multi-hop reasoning paradigm in KGs that conform to linguistic logic, and we further propose two multi-hop QA generation methods based on KGs. Then, we utilize contrastive learning to pre-train the model with the synthetic QA dataset to inject multi-hop commonsense knowledge. Extensive experiments on five commonsense question answering benchmarks demonstrate that our framework achieves state-of-art performance.

cs.CL↗

arXiv2023

No Place to Hide: Dual Deep Interaction Channel Network for Fake News Detection based on Data Augmentation

Online Social Network (OSN) has become a hotbed of fake news due to the low cost of information dissemination. Although the existing methods have made many attempts in news content and propagation structure, the detection of fake news is still facing two challenges: one is how to mine the unique key features and evolution patterns, and the other is how to tackle the problem of small samples to build the high-performance model. Different from popular methods which take full advantage of the propagation topology structure, in this paper, we propose a novel framework for fake news detection from perspectives of semantic, emotion and data enhancement, which excavates the emotional evolution patterns of news participants during the propagation process, and a dual deep interaction channel network of semantic and emotion is designed to obtain a more comprehensive and fine-grained news representation with the consideration of comments. Meanwhile, the framework introduces a data enhancement module to obtain more labeled data with high quality based on confidence which further improves the performance of the classification model. Experiments show that the proposed approach outperforms the state-of-the-art methods.

cs.CL↗

arXiv2022

A Comprehensive Survey and Taxonomy on Single Image Dehazing Based on Deep Learning

With the development of convolutional neural networks, hundreds of deep learning based dehazing methods have been proposed. In this paper, we provide a comprehensive survey on supervised, semi-supervised, and unsupervised single image dehazing. We first discuss the physical model, datasets, network modules, loss functions, and evaluation metrics that are commonly used. Then, the main contributions of various dehazing algorithms are categorized and summarized. Further, quantitative and qualitative experiments of various baseline methods are carried out. Finally, the unsolved issues and challenges that can inspire the future research are pointed out. A collection of useful dehazing materials is available at \url{https://github.com/Xiaofeng-life/AwesomeDehazing}.

cs.CV↗

arXiv2022

AlignVE: Visual Entailment Recognition Based on Alignment Relations

Visual entailment (VE) is to recognize whether the semantics of a hypothesis text can be inferred from the given premise image, which is one special task among recent emerged vision and language understanding tasks. Currently, most of the existing VE approaches are derived from the methods of visual question answering. They recognize visual entailment by quantifying the similarity between the hypothesis and premise in the content semantic features from multi modalities. Such approaches, however, ignore the VE's unique nature of relation inference between the premise and hypothesis. Therefore, in this paper, a new architecture called AlignVE is proposed to solve the visual entailment problem with a relation interaction method. It models the relation between the premise and hypothesis as an alignment matrix. Then it introduces a pooling operation to get feature vectors with a fixed size. Finally, it goes through the fully-connected layer and normalization layer to complete the classification. Experiments show that our alignment-based architecture reaches 72.45\% accuracy on SNLI-VE dataset, outperforming previous content-based models under the same settings.

cs.CV↗

arXiv2021

Understanding User Topic Preferences across Multiple Social Networks

In recent years, social networks have shown diversity in function and applications. People begin to use multiple online social networks simultaneously for different demands. The ability to uncover a user's latent topic and social network preference is critical for community detection, recommendation, and personalized service across social networks. Unfortunately, most current works focus on the single network, necessitating new technology and models to address this issue. This paper proposes a user preference discovery model on multiple social networks. Firstly, the global and local topic concepts are defined, then a latent semantic topic discovery method is used to obtain global and local topic word distributions, along with user topic and social network preferences. After that, the topic distribution characteristics of different social networks are examined, as well as the reasons why users choose one network over another to create a post. Next, a Gibbs sampling algorithm is adopted to obtain the model parameters. In the experiment, we collect data from Twitter, Instagram, and Tumblr websites to build a dataset of multiple social networks. Finally, we compare our research to previous works, and both qualitative and quantitative evaluation results have demonstrated the effectiveness.

cs.SI↗

arXiv2021

Heterogeneous Hypergraph Embedding for Graph Classification

Recently, graph neural networks have been widely used for network embedding because of their prominent performance in pairwise relationship learning. In the real world, a more natural and common situation is the coexistence of pairwise relationships and complex non-pairwise relationships, which is, however, rarely studied. In light of this, we propose a graph neural network-based representation learning framework for heterogeneous hypergraphs, an extension of conventional graphs, which can well characterize multiple non-pairwise relations. Our framework first projects the heterogeneous hypergraph into a series of snapshots and then we take the Wavelet basis to perform localized hypergraph convolution. Since the Wavelet basis is usually much sparser than the Fourier basis, we develop an efficient polynomial approximation to the basis to replace the time-consuming Laplacian decomposition. Extensive evaluations have been conducted and the experimental results show the superiority of our method. In addition to the standard tasks of network embedding evaluation such as node classification, we also apply our method to the task of spammers detection and the superior performance of our framework shows that relationships beyond pairwise are also advantageous in the spammer detection.

cs.SI↗

arXiv2019

Community Detection Across Multiple Social Networks based on Overlapping Users

With the rapid development of Internet technology, online social networks (OSNs) have got fast development and become increasingly popular. Meanwhile, the research works across multiple social networks attract more and more attention from researchers, and community detection is an important one across OSNs for online security problems, such as the user behavior analysis and abnormal community discovery. In this paper, a community detection method is proposed across multiple social networks based on overlapping users. First, the concept of overlapping users is defined, then an algorithm CMN NMF is designed to discover the stub communities from overlapping users based on the social relevance. After that, we extend each stub community in different social networks by adding the users with strong similarity, and in the end different communities are excavated out across networks. Experimental results show the advantage on effectiveness of our method over other methods under real data sets.

cs.SI↗