Searcharxiv⌕ Search

arXiv subjects

Ming Zhang

Publications and source records attributed to Ming Zhang.

At least 199 records · Page 11Linked to original sources
arXiv2024

Measuring Vision-Language STEM Skills of Neural Models

We introduce a new challenge to test the STEM skills of neural models. The problems in the real world often require solutions, combining knowledge from STEM (science, technology, engineering, and math). Unlike existing datasets, our dataset requires the understanding of multimodal vision-language information of STEM. Our dataset features one of the largest and most comprehensive datasets for the challenge. It includes 448 skills and 1,073,146 questions spanning all STEM subjects. Compared to existing datasets that often focus on examining expert-level ability, our dataset includes fundamental skills and questions designed based on the K-12 curriculum. We also add state-of-the-art foundation models such as CLIP and GPT-3.5-Turbo to our benchmark. Results show that the recent model advances only help master a very limited number of lower grade-level skills (2.5% in the third grade) in our dataset. In fact, these models are still well below (averaging 54.7%) the performance of elementary students, not to mention near expert-level performance. To understand and increase the performance on our dataset, we teach the models on a training split of our dataset. Even though we observe improved performance, the model performance remains relatively low compared to average elementary students. To solve STEM problems, we will need novel algorithmic innovations from the community.

cs.CL↗
arXiv2024

Measuring Social Norms of Large Language Models

We present a new challenge to examine whether large language models understand social norms. In contrast to existing datasets, our dataset requires a fundamental understanding of social norms to solve. Our dataset features the largest set of social norm skills, consisting of 402 skills and 12,383 questions covering a wide set of social norms ranging from opinions and arguments to culture and laws. We design our dataset according to the K-12 curriculum. This enables the direct comparison of the social understanding of large language models to humans, more specifically, elementary students. While prior work generates nearly random accuracy on our benchmark, recent large language models such as GPT3.5-Turbo and LLaMA2-Chat are able to improve the performance significantly, only slightly below human performance. We then propose a multi-agent framework based on large language models to improve the models' ability to understand social norms. This method further improves large language models to be on par with humans. Given the increasing adoption of large language models in real-world applications, our finding is particularly important and presents a unique direction for future improvements.

cs.CL↗
arXiv2024

Towards Graph Contrastive Learning: A Survey and Beyond

In recent years, deep learning on graphs has achieved remarkable success in various domains. However, the reliance on annotated graph data remains a significant bottleneck due to its prohibitive cost and time-intensive nature. To address this challenge, self-supervised learning (SSL) on graphs has gained increasing attention and has made significant progress. SSL enables machine learning models to produce informative representations from unlabeled graph data, reducing the reliance on expensive labeled data. While SSL on graphs has witnessed widespread adoption, one critical component, Graph Contrastive Learning (GCL), has not been thoroughly investigated in the existing literature. Thus, this survey aims to fill this gap by offering a dedicated survey on GCL. We provide a comprehensive overview of the fundamental principles of GCL, including data augmentation strategies, contrastive modes, and contrastive optimization objectives. Furthermore, we explore the extensions of GCL to other aspects of data-efficient graph learning, such as weakly supervised learning, transfer learning, and related scenarios. We also discuss practical applications spanning domains such as drug discovery, genomics analysis, recommender systems, and finally outline the challenges and potential future directions in this field.

cs.LG↗
arXiv2024

Necessity of orthogonal basis vectors for the two-anyon problem in one-dimensional lattice

Few-body physics for anyons has been intensively studied within the anyon-Hubbard model, including the quantum walk and Bloch oscillations of two-anyon states. However, the known theoretical proposal and experimental simulations of two-anyon states in one-dimensional lattice have been carried out by expanding the wavefunction in terms of non-orthogonal basis vectors, which introduces extra non-physical degrees of freedom. In the present work, we deduce the finite difference equations for the two-anyon state in the one-dimensional lattice by solving the Schrödinger equation with orthogonal basis vectors. Such an orthogonal scheme gives all the orthogonal physical eigenstates for the time-independent two-anyon Schrödinger equation, while the conventional (non-orthogonal) method produces a lot of non-physical redundant eigen-solutions whose components violate the anyonic relations. The dynamical property of the two-anyon states in a sufficiently large lattice has been investigated and compared in both the orthogonal and conventional schemes, which proves to depend crucially on the initial states. When the initial states with two anyons on the same site or (next-)neighboring sites are suitably chosen to be in accordance with the anyonic coefficient relation, we observe exactly the same dynamical behavior in the two schemes, including the revival probability, the probability density function, and the two-body correlation, otherwise, the conventional scheme will produce erroneous results which not any more describe anyons. The period of the Bloch oscillation in the pseudo-fermionic limit is found to be twice that in the bosonic limit, while the oscillations disappear for statistical parameters in between. Our findings are vital for quantum simulations of few-body physics with anyons in the lattice.

cond-mat.quant-gas↗
arXiv2024

All in One Framework for Multimodal Re-identification in the Wild

In Re-identification (ReID), recent advancements yield noteworthy progress in both unimodal and cross-modal retrieval tasks. However, the challenge persists in developing a unified framework that could effectively handle varying multimodal data, including RGB, infrared, sketches, and textual information. Additionally, the emergence of large-scale models shows promising performance in various vision tasks but the foundation model in ReID is still blank. In response to these challenges, a novel multimodal learning paradigm for ReID is introduced, referred to as All-in-One (AIO), which harnesses a frozen pre-trained big model as an encoder, enabling effective multimodal retrieval without additional fine-tuning. The diverse multimodal data in AIO are seamlessly tokenized into a unified space, allowing the modality-shared frozen encoder to extract identity-consistent features comprehensively across all modalities. Furthermore, a meticulously crafted ensemble of cross-modality heads is designed to guide the learning trajectory. AIO is the \textbf{first} framework to perform all-in-one ReID, encompassing four commonly used modalities. Experiments on cross-modal and multimodal ReID reveal that AIO not only adeptly handles various modal data but also excels in challenging contexts, showcasing exceptional performance in zero-shot and domain generalization scenarios.

cs.CV↗
arXiv2024

Very Long Baseline Array Observations of Parsec-scale Radio Emission in Dual Active Galactic Nuclei

It is believed that dual active galactic nuclei (dual AGN) will form during galaxies merge. Studying dual-AGN emission can provide valuable insights into galaxy merging and evolution. To investigate parsec-scale radio emission properties, we observed eight radio components of four selected dual-AGN systems using the Very Long Baseline Array (VLBA) at 5 GHz in multiple-phase-center mode. Among them, two compact radio components, labeled J0051+0020B and J2300-0005A, were detected clearly on parsec scales for the first time. However, the radio emission of the other six components was resolved out in the high-resolution images. We provided the values or upper limits of the brightness temperature and radio emission power, and analyzed the emission origins in detail for each target. Based on their physical properties reported in this work and in the literature, we suggest the radio emission in J0051+0020B and J2300-0005A originates primarily from compact jets, while the other six sources show more complex emission mechanisms. In addition, our VLBA observations suggest the systematic X-ray deficit in our dual-AGN sample is likely attributed to the tidally induced effect and possible viewing angle effect.

astro-ph.GA↗
arXiv2024

Thermodynamics of charged Lifshitz black holes with scalar hair

In this work, we discuss the generalized Einstein-Maxwell-Dilaton gravity theory with a nonminimal coupling between the Maxwell field and scalar field. Considering different geometric properties of black hole horizon structure, the charged dilaton Lifshitz black hole solutions are presented in 4-dimensional spacetimes. Later, utilizing the Wald Formalism, we derive the thermodynamic first law of black hole and conserved quantities. According to the relationship between the heat capacity and the local stability of black hole, we study the stability of charged Lifshitz black holes and identify the thermodynamic stable region of black holes that meet the criteria.

gr-qc↗
arXiv2024

CURSOR: Scalable Mixed-Order Hypergraph Matching with CUR Decomposition

To achieve greater accuracy, hypergraph matching algorithms require exponential increases in computational resources. Recent kd-tree-based approximate nearest neighbor (ANN) methods, despite the sparsity of their compatibility tensor, still require exhaustive calculations for large-scale graph matching. This work utilizes CUR tensor decomposition and introduces a novel cascaded second and third-order hypergraph matching framework (CURSOR) for efficient hypergraph matching. A CUR-based second-order graph matching algorithm is used to provide a rough match, and then the core of CURSOR, a fiber-CUR-based tensor generation method, directly calculates entries of the compatibility tensor by leveraging the initial second-order match result. This significantly decreases the time complexity and tensor density. A probability relaxation labeling (PRL)-based matching algorithm, especially suitable for sparse tensors, is developed. Experiment results on large-scale synthetic datasets and widely-adopted benchmark sets demonstrate the superiority of CURSOR over existing methods. The tensor generation method in CURSOR can be integrated seamlessly into existing hypergraph matching methods to improve their performance and lower their computational costs.

cs.CV↗
arXiv2024

When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications

Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method, is successfully employed in NLP applications including few-shot text classification and multi-domain low-resource language generation. Many impacting factors, including data quantity, similarity among tasks, and the balance between general language model and task-specific adaptation, can affect the performance of MAML in NLP, but few works have thoroughly studied them. In this paper, we conduct an empirical study to investigate these impacting factors and conclude when MAML works the best based on the experimental results.

cs.CL↗
arXiv2024

A quantum algorithm for the Kalman filter using block encoding

Quantum algorithms offer significant speed-ups over their classical counterparts in various applications. In this paper, we develop quantum algorithms for the Kalman filter widely used in classical control engineering using the block encoding method. The entire calculation process is achieved by performing matrix operations on Hamiltonians based on the block encoding framework, including addition, multiplication, and inversion, which can be completed in a unified framework compared to previous quantum algorithms for solving control problems. We demonstrate that the quantum algorithm exponentially accelerates the computation of the Kalman filter compared to traditional methods. The time complexity can be reduced from $O(n^3)$ to $O(κpoly\log(n/ε)\log(1/ε'))$, where $n$ represents the matrix dimension, $κ$ denotes the condition number for the matrix to be inverted, $ε$ indicates desired precision in block encoding, $ε'$ signifies desired precision in matrix inversion. This paper provides a comprehensive quantum solution for implementing the Kalman filter and serves as an attempt to broaden the scope of quantum computation applications. Finally, we present an illustrative example implemented in Qiskit (a Python-based open-source toolkit) as a proof-of-concept.

math.QA↗
arXiv2024

Notes on emergent conformal symmetry for black holes

We examine the motion of the massless scalar field and nearly bound null geodesics in the near-ring region of a black hole, which may possess either acceleration or a gravitomagnetic mass. Around such black holes, the photon ring deviates from the equatorial plane. In the large angular momentum limit, we demonstrate that the massless scalar field exhibits an emergent conformal symmetry in this near-ring region. Additionally, in the nearly bound limit, we observe the emergence of a conformal symmetry for the null geodesics that constitute the photon ring in the black hole image. These findings suggest that the hidden conformal symmetry, associated with the Lie algebra $\mathfrak{s l}(2, \mathbb{R})$, persists even for black holes lacking north-south reflection symmetry, thereby broadening the foundation of photon ring holography. Finally, we show that the conformal symmetry also emerges for nearly bound timelike geodesics and scalar fields in proximity to the particle ring, and with specific mass around a Schwarzschild black hole.

hep-th↗
arXiv2024

Multi-modal Semantic Understanding with Contrastive Cross-modal Feature Alignment

Multi-modal semantic understanding requires integrating information from different modalities to extract users' real intention behind words. Most previous work applies a dual-encoder structure to separately encode image and text, but fails to learn cross-modal feature alignment, making it hard to achieve cross-modal deep information interaction. This paper proposes a novel CLIP-guided contrastive-learning-based architecture to perform multi-modal feature alignment, which projects the features derived from different modalities into a unified deep space. On multi-modal sarcasm detection (MMSD) and multi-modal sentiment analysis (MMSA) tasks, the experimental results show that our proposed model significantly outperforms several baselines, and our feature alignment strategy brings obvious performance gain over models with different aggregating methods and models even enriched with knowledge. More importantly, our model is simple to implement without using task-specific external knowledge, and thus can easily migrate to other multi-modal tasks. Our source codes are available at https://github.com/ChangKe123/CLFA.

cs.CL↗
arXiv2024

Quantum tomography of molecules using ultrafast electron diffraction

We propose a quantum tomography (QT) approach to retrieve the temporally evolving reduced density matrix in elecotronic state basis, where the populations and coherence between ground state and excited state are reconstructed from the ultrafast electron diffraction signal. In order to showcase the capability of the proposed QT approach, we simulate the nuclear wavepacket dynamics and ultrafast electron diffraction of photoexcited pyrrole molecules using ab initio quantum chemical CASSCF method. From simulated time-resolved diffraction data, we retrieve the evolving density matrix in a crude diabatic representation basis and reveal the symmetry of the excited pyrrole wavepacket. Our QT approach opens the route to make quantum version of "molecular movie" that covers the electronic degree of freedom, and equips ultrafast electron diffraction with the power to reveal the coherence between electronic states, relaxation and dynamics of population transfer.

physics.chem-ph↗
arXiv2024

COOL: A Conjoint Perspective on Spatio-Temporal Graph Neural Network for Traffic Forecasting

This paper investigates traffic forecasting, which attempts to forecast the future state of traffic based on historical situations. This problem has received ever-increasing attention in various scenarios and facilitated the development of numerous downstream applications such as urban planning and transportation management. However, the efficacy of existing methods remains sub-optimal due to their tendency to model temporal and spatial relationships independently, thereby inadequately accounting for complex high-order interactions of both worlds. Moreover, the diversity of transitional patterns in traffic forecasting makes them challenging to capture for existing approaches, warranting a deeper exploration of their diversity. Toward this end, this paper proposes Conjoint Spatio-Temporal graph neural network (abbreviated as COOL), which models heterogeneous graphs from prior and posterior information to conjointly capture high-order spatio-temporal relationships. On the one hand, heterogeneous graphs connecting sequential observation are constructed to extract composite spatio-temporal relationships via prior message passing. On the other hand, we model dynamic relationships using constructed affinity and penalty graphs, which guide posterior message passing to incorporate complementary semantic information into node representations. Moreover, to capture diverse transitional properties to enhance traffic forecasting, we propose a conjoint self-attention decoder that models diverse temporal patterns from both multi-rank and multi-scale views. Experimental results on four popular benchmark datasets demonstrate that our proposed COOL provides state-of-the-art performance compared with the competitive baselines.

cs.LG↗
arXiv2024

A Comprehensive Survey on Deep Graph Representation Learning

Graph representation learning aims to effectively encode high-dimensional sparse graph-structured data into low-dimensional dense vectors, which is a fundamental task that has been widely studied in a range of fields, including machine learning and data mining. Classic graph embedding methods follow the basic idea that the embedding vectors of interconnected nodes in the graph can still maintain a relatively close distance, thereby preserving the structural information between the nodes in the graph. However, this is sub-optimal due to: (i) traditional methods have limited model capacity which limits the learning performance; (ii) existing techniques typically rely on unsupervised learning strategies and fail to couple with the latest learning paradigms; (iii) representation learning and downstream tasks are dependent on each other which should be jointly enhanced. With the remarkable success of deep learning, deep graph representation learning has shown great potential and advantages over shallow (traditional) methods, there exist a large number of deep graph representation learning techniques have been proposed in the past decade, especially graph neural networks. In this survey, we conduct a comprehensive survey on current deep graph representation learning algorithms by proposing a new taxonomy of existing state-of-the-art literature. Specifically, we systematically summarize the essential components of graph representation learning and categorize existing approaches by the ways of graph neural network architectures and the most recent advanced learning paradigms. Moreover, this survey also provides the practical and promising applications of deep graph representation learning. Last but not least, we state new perspectives and suggest challenging directions which deserve further investigations in the future.

cs.LG↗
arXiv2024

RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning

Recent advancements in Large Language Models (LLMs) and Large Multi-modal Models (LMMs) have shown potential in various medical applications, such as Intelligent Medical Diagnosis. Although impressive results have been achieved, we find that existing benchmarks do not reflect the complexity of real medical reports and specialized in-depth reasoning capabilities. In this work, we introduced RJUA-MedDQA, a comprehensive benchmark in the field of medical specialization, which poses several challenges: comprehensively interpreting imgage content across diverse challenging layouts, possessing numerical reasoning ability to identify abnormal indicators and demonstrating clinical reasoning ability to provide statements of disease diagnosis, status and advice based on medical contexts. We carefully design the data generation pipeline and proposed the Efficient Structural Restoration Annotation (ESRA) Method, aimed at restoring textual and tabular content in medical report images. This method substantially enhances annotation efficiency, doubling the productivity of each annotator, and yields a 26.8% improvement in accuracy. We conduct extensive evaluations, including few-shot assessments of 5 LMMs which are capable of solving Chinese medical QA tasks. To further investigate the limitations and potential of current LMMs, we conduct comparative experiments on a set of strong LLMs by using image-text generated by ESRA method. We report the performance of baselines and offer several observations: (1) The overall performance of existing LMMs is still limited; however LMMs more robust to low-quality and diverse-structured images compared to LLMs. (3) Reasoning across context and image content present significant challenges. We hope this benchmark helps the community make progress on these challenging tasks in multi-modal medical document understanding and facilitate its application in healthcare.

cs.CL↗
arXiv2024

Preparing Lessons for Progressive Training on Language Models

The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes. Prior work suggests using pretrained small models to improve training efficiency, but this approach may not be suitable for new model structures. On the other hand, training from scratch can be slow, and progressively stacking layers often fails to achieve significant acceleration. To address these challenges, we propose a novel method called Apollo, which prep\textbf{a}res lessons for ex\textbf{p}anding \textbf{o}perations by \textbf{l}earning high-\textbf{l}ayer functi\textbf{o}nality during training of low layers. Our approach involves low-value-prioritized sampling (LVPS) to train different depths and weight sharing to facilitate efficient expansion. We also introduce an interpolation method for stable model depth extension. Experiments demonstrate that Apollo achieves state-of-the-art acceleration ratios, even rivaling methods using pretrained models, making it a universal and efficient solution for training deep models while reducing time, financial, and environmental costs.

cs.LG↗