arXiv subjects

Ke Qin

Publications and source records attributed to Ke Qin.

33 records · Page 2Linked to original sources

arXiv2024

GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction

Large language models (LLMs) have been demonstrated to possess the capabilities to understand fundamental graph properties and address various graph reasoning tasks. Existing methods fine-tune LLMs to understand and execute graph reasoning tasks by specially designed task instructions. However, these Text-Instruction methods generally exhibit poor performance. Inspired by tool learning, researchers propose Tool-Instruction methods to solve various graph problems by special tool calling (e.g., function, API and model), achieving significant improvements in graph reasoning tasks. Nevertheless, current Tool-Instruction approaches focus on the tool information and ignore the graph structure information, which leads to significantly inferior performance on small-scale LLMs (less than 13B). To tackle this issue, we propose GraphTool-Instruction, an innovative Instruction-tuning approach that decomposes the graph reasoning task into three distinct subtasks (i.e., graph extraction, tool name identification and tool parameter extraction), and design specialized instructions for each subtask. Our GraphTool-Instruction can be used as a plug-and-play prompt for different LLMs without fine-tuning. Moreover, building on GraphTool-Instruction, we develop GTools, a dataset that includes twenty graph reasoning tasks, and create a graph reasoning LLM called GraphForge based on Llama3-8B. We conduct extensive experiments on twenty graph reasoning tasks with different graph types (e.g., graph size or graph direction), and we find that GraphTool-Instruction achieves SOTA compared to Text-Instruction and Tool-Instruction methods. Fine-tuned on GTools, GraphForge gets further improvement of over 30% compared to the Tool-Instruction enhanced GPT-3.5-turbo, and it performs comparably to the high-cost GPT-4o. Our codes and data are available at https://anonymous.4open.science/r/GraphTool-Instruction.

cs.LG↗

arXiv2024

Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation

Data-free knowledge distillation (DFKD) has emerged as a pivotal technique in the domain of model compression, substantially reducing the dependency on the original training data. Nonetheless, conventional DFKD methods that employ synthesized training data are prone to the limitations of inadequate diversity and discrepancies in distribution between the synthesized and original datasets. To address these challenges, this paper introduces an innovative approach to DFKD through diverse diffusion augmentation (DDA). Specifically, we revise the paradigm of common data synthesis in DFKD to a composite process through leveraging diffusion models subsequent to data synthesis for self-supervised augmentation, which generates a spectrum of data samples with similar distributions while retaining controlled variations. Furthermore, to mitigate excessive deviation in the embedding space, we introduce an image filtering technique grounded in cosine similarity to maintain fidelity during the knowledge distillation process. Comprehensive experiments conducted on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets showcase the superior performance of our method across various teacher-student network configurations, outperforming the contemporary state-of-the-art DFKD methods. Code will be available at:https://github.com/SLGSP/DDA.

cs.CV↗

arXiv2024

MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality

Recently, prompt learning has garnered considerable attention for its success in various Vision-Language (VL) tasks. However, existing prompt-based models are primarily focused on studying prompt generation and prompt strategies with complete modality settings, which does not accurately reflect real-world scenarios where partial modality information may be missing. In this paper, we present the first comprehensive investigation into prompt learning behavior when modalities are incomplete, revealing the high sensitivity of prompt-based models to missing modalities. To this end, we propose a novel Multi-step Adaptive Prompt Learning (MuAP) framework, aiming to generate multimodal prompts and perform multi-step prompt tuning, which adaptively learns knowledge by iteratively aligning modalities. Specifically, we generate multimodal prompts for each modality and devise prompt strategies to integrate them into the Transformer model. Subsequently, we sequentially perform prompt tuning from single-stage and alignment-stage, allowing each modality-prompt to be autonomously and adaptively learned, thereby mitigating the imbalance issue caused by only textual prompts that are learnable in previous works. Extensive experiments demonstrate the effectiveness of our MuAP and this model achieves significant improvements compared to the state-of-the-art on all benchmark datasets

cs.AI↗

arXiv2024

No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks

While the progression of Large Language Models (LLMs) has notably propelled financial analysis, their application has largely been confined to singular language realms, leaving untapped the potential of bilingual Chinese-English capacity. To bridge this chasm, we introduce ICE-PIXIU, seamlessly amalgamating the ICE-INTENT model and ICE-FLARE benchmark for bilingual financial analysis. ICE-PIXIU uniquely integrates a spectrum of Chinese tasks, alongside translated and original English datasets, enriching the breadth and depth of bilingual financial modeling. It provides unrestricted access to diverse model variants, a substantial compilation of diverse cross-lingual and multi-modal instruction data, and an evaluation benchmark with expert annotations, comprising 10 NLP tasks, 20 bilingual specific tasks, totaling 95k datasets. Our thorough evaluation emphasizes the advantages of incorporating these bilingual datasets, especially in translation tasks and utilizing original English data, enhancing both linguistic flexibility and analytical acuity in financial contexts. Notably, ICE-INTENT distinguishes itself by showcasing significant enhancements over conventional LLMs and existing financial LLMs in bilingual milieus, underscoring the profound impact of robust bilingual data on the accuracy and efficacy of financial NLP.

cs.CE↗

arXiv2024

An Alternative Channel to Black Hole Low-Mass X-ray Binaries: Dynamical Friction of Dark Matter?

Both the anomalous magnetic braking of Ap/Bp stars and the surrounding circumbinary disk models can account for the formation of black hole (BH) low-mass X-ray binaries (LMXBs), while the simulated effective temperatures of the donor stars are significantly higher than the observed values. Therefore, the formation of BH LMXBs is not still completely understood. In this work, we diagnose whether the dynamical friction between dark matter and the companion stars can drive BH binaries to evolve toward the observed BH LMXBs and alleviate the effective temperature problem. Assuming that there exists a density spike of dark matter around BH, the dynamical friction can produce an efficient angular momentum loss, driving BH binaries with an intermediate-mass companion star to evolve into BH LMXBs for a spike index higher than $γ= 1.58$. Our detailed stellar evolution models show that the calculated effective temperatures can match the observed value of most BH LMXBs for a spike index range of $γ= 1.7-2.1$. However, the simulated mass-transfer rates when $γ= 2.0$ and $2.1$ are too high to be consistent with the observed properties that BH LMXBs appears as soft X-ray transients. Therefore, the dynamical friction of dark matter can only alleviate the effective temperature problem of those BH LMXBs with a relatively short orbital period.

astro-ph.HE↗

arXiv2024

Towards Lifelong Scene Graph Generation with Knowledge-ware In-context Prompt Learning

Scene graph generation (SGG) endeavors to predict visual relationships between pairs of objects within an image. Prevailing SGG methods traditionally assume a one-off learning process for SGG. This conventional paradigm may necessitate repetitive training on all previously observed samples whenever new relationships emerge, mitigating the risk of forgetting previously acquired knowledge. This work seeks to address this pitfall inherent in a suite of prior relationship predictions. Motivated by the achievements of in-context learning in pretrained language models, our approach imbues the model with the capability to predict relationships and continuously acquire novel knowledge without succumbing to catastrophic forgetting. To achieve this goal, we introduce a novel and pragmatic framework for scene graph generation, namely Lifelong Scene Graph Generation (LSGG), where tasks, such as predicates, unfold in a streaming fashion. In this framework, the model is constrained to exclusive training on the present task, devoid of access to previously encountered training data, except for a limited number of exemplars, but the model is tasked with inferring all predicates it has encountered thus far. Rigorous experiments demonstrate the superiority of our proposed method over state-of-the-art SGG models in the context of LSGG across a diverse array of metrics. Besides, extensive experiments on the two mainstream benchmark datasets, VG and Open-Image(v6), show the superiority of our proposed model to a number of competitive SGG models in terms of continuous learning and conventional settings. Moreover, comprehensive ablation experiments demonstrate the effectiveness of each component in our model.

cs.CV↗

arXiv2023

Black Hole Ultracompact X-Ray Binaries as Galactic Low-frequency Gravitational Wave Sources: the He Star Channel

Black hole (BH) ultracompact X-ray binaries (UCXBs) are potential Galactic low-frequency gravitational wave (GW) sources. As an alternative channel, BH UCXBs can evolve from BH+He star binaries. In this work, we perform a detailed stellar evolution model for the formation and evolution of BH UCXBs evolving from the He star channel to diagnose their detectability as low-frequency GW sources. Our calculations found that some nascent BH+He star binaries after the common-envelope (CE) phase could evolve into UCXB-LISA sources with a maximum GW frequency of $\sim5~\rm mHz$, which can be detected in a distance of 10 kpc (or 100 kpc). Once BH+He star systems become UCXBs through mass transfer, they would emit X-ray luminosities of $\sim10^{38}~\rm erg\, s^{-1}$, making them ideal multimessenger objects. If the initial He-star masses are $\geq 0.7 M_{\odot}$, those systems are likely to experience two Roche lobe overflows, and the X-ray luminosity can reach a maximum of $3.5\times 10^{39}~\rm erg\, s^{-1}$ in the second mass-transfer stage. The initial He-star masses and initial orbital periods of progenitors of Galactic BH UCXB-LISA sources are in the range of 0.32-2.9 $M_{\odot}$ and 0.02-0.19 days, respectively. Nearly all BH+He star binaries in the above parameter space can evolve into GW sources whose chirp masses can be accurately measured. Employing a population synthesis simulation, we predict the birthrate and detection number of Galactic BH UCXB-LISA source evolving from the He star channel are $R=2.2\times10^{-6}~\rm yr^{-1}$ and 33 for an optimistic CE parameter, respectively.

astro-ph.HE↗

arXiv2023

Graph Entropy Minimization for Semi-supervised Node Classification

Node classifiers are required to comprehensively reduce prediction errors, training resources, and inference latency in the industry. However, most graph neural networks (GNN) concentrate only on one or two of them. The compromised aspects thus are the shortest boards on the bucket, hindering their practical deployments for industrial-level tasks. This work proposes a novel semi-supervised learning method termed Graph Entropy Minimization (GEM) to resolve the three issues simultaneously. GEM benefits its one-hop aggregation from massive uncategorized nodes, making its prediction accuracy comparable to GNNs with two or more hops message passing. It can be decomposed to support stochastic training with mini-batches of independent edge samples, achieving extremely fast sampling and space-saving training. While its one-hop aggregation is faster in inference than deep GNNs, GEM can be further accelerated to an extreme by deriving a non-hop classifier via online knowledge distillation. Thus, GEM can be a handy choice for latency-restricted and error-sensitive services running on resource-constraint hardware. Code is available at https://github.com/cf020031308/GEM.

cs.LG↗

arXiv2023

Black Hole Ultra-compact X-ray Binaries: Galactic Low-Frequency Gravitational Wave Sources

In the Galaxy, close binaries with compact objects are important low-frequency gravitational wave (GW) sources. As potential low-frequency GW sources, neutron star/white dwarf (WD) ultra-compact X-ray binaries (UCXBs) have been investigated extensively. Using the MESA code, we systematically explored the evolution of black hole (BH)-main sequence star (MS) binaries to diagnose whether their descendants can be detected by space-borne GW detectors. Our simulations show that BH-MS binaries with an initial orbital period less than the bifurcation period can evolve into BH UCXBs that can be detected by LISA. Such an evolutionary channel would form compact mass-transferring BH-WD systems rather than detached BH-WD systems. The calculated X-ray luminosities of BH UCXBs detected by LISA at a distance $d=1$ kpc are $\sim10^{33}-10^{35}~\rm erg\,s^{-1}$ ($\sim10^{34}-10^{35}~\rm erg\,s^{-1}$ for $d=10$ kpc), hence it is possible to detect their electromagnetic counterparts. It is worth emphasizing only some BH-MS systems with an initial orbital period very close to the bifurcation period can evolve toward low-frequency GW sources whose chirp masses can be measured. The maximum GW frequency of BH UCXBs forming by BH-MS pathway is about 3 mHz, which is smaller than the minimum GW frequency (6.4 mHz) of mass-transferring BH-WD originating from a dynamic process. Furthermore, we obtain an initial parameter space (donor-star masses and orbital periods) of progenitors of BH UCXB-GW sources, which can be applied to future population synthesis simulations. By a rough estimation, we predict that LISA could detect only a few BH UCXB-GW sources forming by the BH-MS channel.

astro-ph.HE↗

arXiv2022

VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges

Artificial Intelligence (AI) and its applications have sparked extraordinary interest in recent years. This achievement can be ascribed in part to advances in AI subfields including Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). Deep learning, a sub-field of machine learning that employs artificial neural network concepts, has enabled the most rapid growth in these domains. The integration of vision and language has sparked a lot of attention as a result of this. The tasks have been created in such a way that they properly exemplify the concepts of deep learning. In this review paper, we provide a thorough and an extensive review of the state of the arts approaches, key models design principles and discuss existing datasets, methods, their problem formulation and evaluation measures for VQA and Visual reasoning tasks to understand vision and language representation learning. We also present some potential future paths in this field of research, with the hope that our study may generate new ideas and novel approaches to handle existing difficulties and develop new applications.

cs.CV↗

arXiv2022

Huge out-of-plane piezoelectric response in ferromagnetic monolayer NiClI

The combination of piezoelectricity and ferromagnetic (FM) order in a two-dimensional (2D) material, namely 2D piezoelectric ferromagnetism (PFM), may open up unprecedented opportunities for novel device applications. Here, we predict an in-plane FM semiconductor Janus monolayer NiClI with considerably large magnetic anisotropy energy (MAE) of 1.439 meV, exhibiting dynamic, mechanical and thermal stabilities. The NiClI monolayer possesses larger in-plane piezoelectricity ($d_{11}$$=$5.21 pm/V) comparable to that of $\mathrm{MoS_2}$. Furthermore, NiClI has huge out-of-plane piezoelectricity ($d_{31}$$=$1.89 pm/V), which is highly desirable for ultrathin piezoelectric device application. It is proved that huge out-of-plane piezoelectricity is robust against electronic correlation, which confirms reliability of huge $d_{31}$. Finally, being analogous to NiClI, PFM with large out-of-plane piezoelectricity can also be achieved in the Janus monolayers of NiClBr and NiBrI, with the predicted $d_{31}$ of 0.73 pm/V and 1.15 pm/V, respectively. The predicted huge out-of-plane piezoelectric response makes Janus monolayer NiClI a good platform for multifunctional semiconductor spintronic applications, which is also compatible with the bottom/top gate technologies of conventional semiconductor nanoelectronic devices.

cond-mat.mtrl-sci↗

arXiv2021

QuatDE: Dynamic Quaternion Embedding for Knowledge Graph Completion

Knowledge graph embedding has been an active research topic for knowledge base completion (KGC), with progressive improvement from the initial TransE, TransH, RotatE et al to the current state-of-the-art QuatE. However, QuatE ignores the multi-faceted nature of the entity and the complexity of the relation, only using rigorous operation on quaternion space to capture the interaction between entitiy pair and relation, leaving opportunities for better knowledge representation which will finally help KGC. In this paper, we propose a novel model, QuatDE, with a dynamic mapping strategy to explicitly capture the variety of relational patterns and separate different semantic information of the entity, using transition vectors to adjust the point position of the entity embedding vectors in the quaternion space via Hamilton product, enhancing the feature interaction capability between elements of the triplet. Experiment results show QuatDE achieves state-of-the-art performance on three well-established knowledge graph completion benchmarks. In particular, the MR evaluation has relatively increased by 26% on WN18 and 15% on WN18RR, which proves the generalization of QuatDE.

cs.CL↗

arXiv2021

Deep Neural Network Based Relation Extraction: An Overview

Knowledge is a formal way of understanding the world, providing a human-level cognition and intelligence for the next-generation artificial intelligence (AI). One of the representations of knowledge is semantic relations between entities. An effective way to automatically acquire this important knowledge, called Relation Extraction (RE), a sub-task of information extraction, plays a vital role in Natural Language Processing (NLP). Its purpose is to identify semantic relations between entities from natural language text. To date, there are several studies for RE in previous works, which have documented these techniques based on Deep Neural Networks (DNNs) become a prevailing technique in this research. Especially, the supervised and distant supervision methods based on DNNs are the most popular and reliable solutions for RE. This article 1) introduces some general concepts, and further 2) gives a comprehensive overview of DNNs in RE from two points of view: supervised RE, which attempts to improve the standard RE systems, and distant supervision RE, which adopts DNNs to design sentence encoder and de-noise method. We further 3) cover some novel methods and recent trends as well as discuss possible future research directions for this task.

cs.CL↗

arXiv2019

Sparse Label Smoothing Regularization for Person Re-Identification

Person re-identification (re-id) is a cross-camera retrieval task which establishes a correspondence between images of a person from multiple cameras. Deep Learning methods have been successfully applied to this problem and have achieved impressive results. However, these methods require a large amount of labeled training data. Currently labeled datasets in person re-id are limited in their scale and manual acquisition of such large-scale datasets from surveillance cameras is a tedious and labor-intensive task. In this paper, we propose a framework that performs intelligent data augmentation and assigns partial smoothing label to generated data. Our approach first exploits the clustering property of existing person re-id datasets to create groups of similar objects that model cross-view variations. Each group is then used to generate realistic images through adversarial training. Our aim is to emphasize feature similarity between generated samples and the original samples. Finally, we assign a non-uniform label distribution to the generated samples and define a regularized loss function for training. The proposed approach tackles two problems (1) how to efficiently use the generated data and (2) how to address the over-smoothness problem found in current regularization methods. Extensive experiments on four larges cale datasets show that our regularization method significantly improves the Re-ID accuracy compared to existing methods.

cs.CV↗

arXiv2018

Self Attention Grid for Person Re-Identification

In this paper, we present an attention mechanism scheme to improve person re-identification task. Inspired by biology, we propose Self Attention Grid (SAG) to discover the most informative parts from a high-resolution image using its internal representation. In particular, given an input image, the proposed model is fed with two copies of the same image and consists of two branches. The upper branch processes the high-resolution image and learns high dimensional feature representation while the lower branch processes the low-resolution image and learn a filtering attention grid. We apply a max filter operation to non-overlapping sub-regions on the high feature representation before element-wise multiplied with the output of the second branch. The feature maps of the second branch are subsequently weighted to reflect the importance of each patch of the grid using a softmax operation. Our attention module helps the network learn the most discriminative visual features of multiple image regions and is specifically optimized to attend feature representation at different levels. Extensive experiments on three large-scale datasets show that our self-attention mechanism significantly improves the baseline model and outperforms various state-of-art models by a large margin.

cs.CV↗