Searcharxiv⌕ Search

arXiv subjects

Peng Wang

Publications and source records attributed to Peng Wang.

At least 469 records · Page 26Linked to original sources
arXiv2024

C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning

Vision-Language Instruction Tuning (VLIT) is a critical training phase for Large Vision-Language Models (LVLMs). With the improving capabilities of open-source LVLMs, researchers have increasingly turned to generate VLIT data by using open-source LVLMs and achieved significant progress. However, such data generation approaches are bottlenecked by the following challenges: 1) Since multi-modal models tend to be influenced by prior language knowledge, directly using LVLMs to generate VLIT data would inevitably lead to low content relevance between generated data and images. 2) To improve the ability of the models to generate VLIT data, previous methods have incorporated an additional training phase to boost the generative capacity. This process hurts the generalization of the models to unseen inputs (i.e., "exposure bias" problem). In this paper, we propose a new Content Correlated VLIT data generation via Contrastive Learning (C3L). Specifically, we design a new content relevance module which enhances the content relevance between VLIT data and images by computing Image Instruction Correspondence Scores S(I2C). Moreover, a contrastive learning module is introduced to further boost the VLIT data generation capability of the LVLMs. A large number of automatic measures on four benchmarks show the effectiveness of our method.

cs.CV↗
arXiv2024

GRAM: An Interpretable Approach for Graph Anomaly Detection using Gradient Attention Maps

Detecting unusual patterns in graph data is a crucial task in data mining. However, existing methods face challenges in consistently achieving satisfactory performance and often lack interpretability, which hinders our understanding of anomaly detection decisions. In this paper, we propose a novel approach to graph anomaly detection that leverages the power of interpretability to enhance performance. Specifically, our method extracts an attention map derived from gradients of graph neural networks, which serves as a basis for scoring anomalies. Notably, our approach is flexible and can be used in various anomaly detection settings. In addition, we conduct theoretical analysis using synthetic data to validate our method and gain insights into its decision-making process. To demonstrate the effectiveness of our method, we extensively evaluate our approach against state-of-the-art graph anomaly detection techniques on real-world graph classification and wireless network datasets. The results consistently demonstrate the superior performance of our method compared to the baselines.

cs.LG↗
arXiv2024

EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models

Large Language Models (LLMs) usually suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data. To this end, many knowledge editing approaches for LLMs have emerged -- aiming to subtly inject/edit updated knowledge or adjust undesired behavior while minimizing the impact on unrelated inputs. Nevertheless, due to significant differences among various knowledge editing methods and the variations in task setups, there is no standard implementation framework available for the community, which hinders practitioners from applying knowledge editing to applications. To address these issues, we propose EasyEdit, an easy-to-use knowledge editing framework for LLMs. It supports various cutting-edge knowledge editing approaches and can be readily applied to many well-known LLMs such as T5, GPT-J, LlaMA, etc. Empirically, we report the knowledge editing results on LlaMA-2 with EasyEdit, demonstrating that knowledge editing surpasses traditional fine-tuning in terms of reliability and generalization. We have released the source code on GitHub, along with Google Colab tutorials and comprehensive documentation for beginners to get started. Besides, we present an online system for real-time knowledge editing, and a demo video.

cs.CL↗
arXiv2024

LLM Granularity for On-the-Fly Robot Control

Assistive robots have attracted significant attention due to their potential to enhance the quality of life for vulnerable individuals like the elderly. The convergence of computer vision, large language models, and robotics has introduced the `visuolinguomotor' mode for assistive robots, where visuals and linguistics are incorporated into assistive robots to enable proactive and interactive assistance. This raises the question: \textit{In circumstances where visuals become unreliable or unavailable, can we rely solely on language to control robots, i.e., the viability of the `linguomotor` mode for assistive robots?} This work takes the initial steps to answer this question by: 1) evaluating the responses of assistive robots to language prompts of varying granularities; and 2) exploring the necessity and feasibility of controlling the robot on-the-fly. We have designed and conducted experiments on a Sawyer cobot to support our arguments. A Turtlebot robot case is designed to demonstrate the adaptation of the solution to scenarios where assistive robots need to maneuver to assist. Codes will be released on GitHub soon to benefit the community.

cs.RO↗
arXiv2024

Enabling Large-Scale and High-Precision Fluid Simulations on Near-Term Quantum Computers

Quantum computational fluid dynamics (QCFD) offers a promising alternative to classical computational fluid dynamics (CFD) by leveraging quantum algorithms for higher efficiency. This paper introduces a comprehensive QCFD method, including an iterative method "Iterative-QLS" that suppresses error in quantum linear solver, and a subspace method to scale the solution to a larger size. We implement our method on a superconducting quantum computer, demonstrating successful simulations of steady Poiseuille flow and unsteady acoustic wave propagation. The Poiseuille flow simulation achieved a relative error of less than $0.2\%$, and the unsteady acoustic wave simulation solved a 5043-dimensional matrix. We emphasize the utilization of the quantum-classical hybrid approach in applications of near-term quantum computers. By adapting to quantum hardware constraints and offering scalable solutions for large-scale CFD problems, our method paves the way for practical applications of near-term quantum computers in computational science.

physics.comp-ph↗
arXiv2024

Enhancing Visible-Infrared Person Re-identification with Modality- and Instance-aware Visual Prompt Learning

The Visible-Infrared Person Re-identification (VI ReID) aims to match visible and infrared images of the same pedestrians across non-overlapped camera views. These two input modalities contain both invariant information, such as shape, and modality-specific details, such as color. An ideal model should utilize valuable information from both modalities during training for enhanced representational capability. However, the gap caused by modality-specific information poses substantial challenges for the VI ReID model to handle distinct modality inputs simultaneously. To address this, we introduce the Modality-aware and Instance-aware Visual Prompts (MIP) network in our work, designed to effectively utilize both invariant and specific information for identification. Specifically, our MIP model is built on the transformer architecture. In this model, we have designed a series of modality-specific prompts, which could enable our model to adapt to and make use of the specific information inherent in different modality inputs, thereby reducing the interference caused by the modality gap and achieving better identification. Besides, we also employ each pedestrian feature to construct a group of instance-specific prompts. These customized prompts are responsible for guiding our model to adapt to each pedestrian instance dynamically, thereby capturing identity-level discriminative clues for identification. Through extensive experiments on SYSU-MM01 and RegDB datasets, the effectiveness of both our designed modules is evaluated. Additionally, our proposed MIP performs better than most state-of-the-art methods.

cs.CV↗
arXiv2024

Expressibility of linear combination of ansatz circuits

Variational Quantum Eigensolver is considered promising for medium-scale noisy quantum computers. Expressibility is an important metric for measuring the capability of a variational quantum Ansatz circuit. A commonly used method to increase expressibility is to increase the circuit depth. However, increasing the circuit depth also introduces more noise. We propose to use a linear combination of ansatzes to improve the expressibility of variational circuits, thus avoiding the increase of circuit depth. Concurrently, we introduce a novel measurement strategy that circumvents the necessity for the Hadamard test, thereby significantly diminishing the reliance on two-qubit gates, which are presently the predominant contributors to quantum noise. We also provide a corresponding gradient calculation method, which makes it convenient to update the parameters. Compared with the method of increasing the circuit depth, our method of improving expressibility is more practical. Numerical simulations demonstrate the effectiveness of our method.

quant-ph↗
arXiv2024

An alternative to purification in CFT

In conformal field theories, in contrast to \emph{adding} some auxiliary states into the bipartite mixed state $ρ_{AB}$ as the usual purifications do, we show a pure entangled state $ψ_{AB}$ can be constructed by \emph{subtracting} the undetectable regions. In this pure state $ψ_{AB}$, the von Neumann entropy $S_{\text{vN}}(A)$ naturally captures quantum entanglement between $A$ and $B$. We verify that $S_{\text{vN}}(A)$ is equal to the entanglement wedge cross-section $E_{W}$ in AdS spacetime, which is conjectured to be the holographic dual of the entanglement of purification. We show such constructed entanglement entropy has a phase transition. The ordinary entanglement entropies of critical and non-critical QFTs are simply limits of the two phases.

hep-th↗
arXiv2024

Autoregressive Pretraining with Mamba in Vision

The vision community has started to build with the recently developed state space model, Mamba, as the new backbone for a range of tasks. This paper shows that Mamba's visual capability can be significantly enhanced through autoregressive pretraining, a direction not previously explored. Efficiency-wise, the autoregressive nature can well capitalize on the Mamba's unidirectional recurrent structure, enabling faster overall training speed compared to other training strategies like mask modeling. Performance-wise, autoregressive pretraining equips the Mamba architecture with markedly higher accuracy over its supervised-trained counterparts and, more importantly, successfully unlocks its scaling potential to large and even huge model sizes. For example, with autoregressive pretraining, a base-size Mamba attains 83.2\% ImageNet accuracy, outperforming its supervised counterpart by 2.0\%; our huge-size Mamba, the largest Vision Mamba to date, attains 85.0\% ImageNet accuracy (85.5\% when finetuned with $384\times384$ inputs), notably surpassing all other Mamba variants in vision. The code is available at \url{https://github.com/OliverRensu/ARM}.

cs.CV↗
arXiv2024

Symmetric Matrix Completion with ReLU Sampling

We study the problem of symmetric positive semi-definite low-rank matrix completion (MC) with deterministic entry-dependent sampling. In particular, we consider rectified linear unit (ReLU) sampling, where only positive entries are observed, as well as a generalization to threshold-based sampling. We first empirically demonstrate that the landscape of this MC problem is not globally benign: Gradient descent (GD) with random initialization will generally converge to stationary points that are not globally optimal. Nevertheless, we prove that when the matrix factor with a small rank satisfies mild assumptions, the nonconvex objective function is geodesically strongly convex on the quotient manifold in a neighborhood of a planted low-rank matrix. Moreover, we show that our assumptions are satisfied by a matrix factor with i.i.d. Gaussian entries. Finally, we develop a tailor-designed initialization for GD to solve our studied formulation, which empirically always achieves convergence to the global minima. We also conduct extensive experiments and compare MC methods, investigating convergence and completion performance with respect to initialization, noise level, dimension, and rank.

cs.LG↗
arXiv2024

Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer

Despite advancements of end-to-end (E2E) models in speech recognition, named entity recognition (NER) is still challenging but critical for semantic understanding. Previous studies mainly focus on various rule-based or attention-based contextual biasing algorithms. However, their performance might be sensitive to the biasing weight or degraded by excessive attention to the named entity list, along with a risk of false triggering. Inspired by the success of the class-based language model (LM) in NER in conventional hybrid systems and the effective decoupling of acoustic and linguistic information in the factorized neural Transducer (FNT), we propose C-FNT, a novel E2E model that incorporates class-based LMs into FNT. In C-FNT, the LM score of named entities can be associated with the name class instead of its surface form. The experimental results show that our proposed C-FNT significantly reduces error in named entities without hurting performance in general word recognition.

eess.AS↗
arXiv2024

RaFe: Ranking Feedback Improves Query Rewriting for RAG

As Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved, query rewriting has been widely incorporated into the RAG system for downstream tasks like open-domain QA. Many works have attempted to utilize small models with reinforcement learning rather than costly LLMs to improve query rewriting. However, current methods require annotations (e.g., labeled relevant documents or downstream answers) or predesigned rewards for feedback, which lack generalization, and fail to utilize signals tailored for query rewriting. In this paper, we propose ours, a framework for training query rewriting models free of annotations. By leveraging a publicly available reranker, ours~provides feedback aligned well with the rewriting objectives. Experimental results demonstrate that ours~can obtain better performance than baselines.

cs.CL↗
arXiv2024

Learning Social Graph for Inactive User Recommendation

Social relations have been widely incorporated into recommender systems to alleviate data sparsity problem. However, raw social relations don't always benefit recommendation due to their inferior quality and insufficient quantity, especially for inactive users, whose interacted items are limited. In this paper, we propose a novel social recommendation method called LSIR (\textbf{L}earning \textbf{S}ocial Graph for \textbf{I}nactive User \textbf{R}ecommendation) that learns an optimal social graph structure for social recommendation, especially for inactive users. LSIR recursively aggregates user and item embeddings to collaboratively encode item and user features. Then, graph structure learning (GSL) is employed to refine the raw user-user social graph, by removing noisy edges and adding new edges based on the enhanced embeddings. Meanwhile, mimic learning is implemented to guide active users in mimicking inactive users during model training, which improves the construction of new edges for inactive users. Extensive experiments on real-world datasets demonstrate that LSIR achieves significant improvements of up to 129.58\% on NDCG in inactive user recommendation. Our code is available at~\url{https://github.com/liun-online/LSIR}.

cs.SI↗
arXiv2024

Willmore surfaces in spheres via loop groups: a survey

In the past decades, the authors made some systematic research on global and local properties of Willmore surfaces in terms of the DPW method. In this note we give a survey, mainly including the basic framework of the DPW method for the global geometry of Willmore surfaces via the conformal Gauss map, applications on constructions of Willmore $2$-spheres, characterizations of minimal surfaces, Willmore deformations of Willmore surfaces and Bjoerling problems for Willmore surfaces. Moreover, we also obtained some results on harmonic maps via DPW, including a duality theorem for harmonic maps into an inner non-compact symmetric space and its dual inner compact symmetric space, and harmonic maps of finite uniton type.

math.DG↗
arXiv2024

On the index of minimal hypersurfaces in $\mathbb{S}^{n+1}$ with $λ_1<n$

In this paper, we prove that a closed minimal hypersurface in $\SSS$ with $λ_1<n$ has Morse index at least $n+4$, providing a partial answer to a conjecture of Perdomo. As a corollary, we re-obtain a partial proof of the famous Urbano Theorem for minimal tori in $\mathbb{S}^3$: a minimal torus in $\mathbb{S}^3$ has Morse index at least $5$, with equality holding if and only if it is congruent to the Clifford torus. The proof is based on a comparison theorem between eigenvalues of two elliptic operators, which also provides us simpler new proofs of some known results on index estimates of both minimal and $r$-minimal hypersurfaces in a sphere.

math.DG↗
arXiv2024

Morse index of minimal products of minimal submanifolds in spheres

Tang-Zhang, Choe-Hoppe, showed independently that one can produce minimal submanifolds in spheres via Clifford type minimal product of minimal submanifolds. In this note, we show that the minimal product is immersed by its first eigenfunctions (of its Laplacian) if and only if the two beginning minimal submanifolds are immersed by their first eigenfunctions. Moreover, we give estimates of Morse index and nullity of the minimal product. In particular, we show that the Clifford minimal submanifold $\left(\sqrt{\frac{n_1}{n}}S^{n_1},\cdots,\sqrt{\frac{n_k}{n}}S^{n_k}\right)\subset S^{n+k-1}$ has index $(k-1)(n+k+1)$ and nullity $(k-1)\sum_{1\leq i<j\leq k}(n_i+1)(n_j+1)$ (with $n=\sum n_j$).

math.DG↗
arXiv2024

On the index of minimal 2-tori in the 4-sphere

In this note we prove that any minimal $2$-torus in $S^4$ has Morse index at least $6$, with equality if and only if it is congruent to the Clifford torus in some great $S^3\subset S^4$.For a minimal $2$-torus in $S^n$ with vanishing Hopf differential, we show that its index is at least $n+3$, and that this estimate is sharp: the equilateral $2$-torus fully embedded in $S^5\subset S^n$ as a homogeneous minimal surface in $S^n$ has index exactly $n+3$.

math.DG↗