arXiv subjects

Yuanbin Wu

Publications and source records attributed to Yuanbin Wu.

At least 37 records · Page 2Linked to original sources

arXiv2025

The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights

Recent research has increasingly focused on multimodal mathematical reasoning, particularly emphasizing the creation of relevant datasets and benchmarks. Despite this, the role of visual information in reasoning has been underexplored. Our findings show that existing multimodal mathematical models minimally leverage visual information, and model performance remains largely unaffected by changes to or removal of images in the dataset. We attribute this to the dominance of textual information and answer options that inadvertently guide the model to correct answers. To improve evaluation methods, we introduce the HC-M3D dataset, specifically designed to require image reliance for problem-solving and to challenge models with similar, yet distinct, images that change the correct answer. In testing leading models, their failure to detect these subtle visual differences suggests limitations in current visual perception capabilities. Additionally, we observe that the common approach of improving general VQA capabilities by combining various types of image encoders does not contribute to math reasoning performance. This finding also presents a challenge to enhancing visual reliance during math reasoning. Our benchmark and code would be available at \href{https://github.com/Yufang-Liu/visual_modality_role}{https://github.com/Yufang-Liu/visual\_modality\_role}.

cs.CV↗

arXiv2025

Isomer production by multi-photon excitation

The multi-photon excitation to the $8$-eV nuclear isomeric state $^{229\text{m}}$Th in the direct laser-nucleus interaction is investigated theoretically. We solve the time-dependent Schrödinger equation with the method which allows us to study the $n$-photon absorption in the nuclear excitation in the direct laser-nucleus interaction. Based on the laser facilities available currently or in the near future, we analyze the impact of the laser parameters on the excitation probability of the multi-photon excitation. The possibilities of the $2$-, $3$- and $4$-photon excitations to the isomeric state $^{229\text{m}}$Th from the ground state are discussed in details. Our results show the strong impact of the laser intensity and pulse duration on the multi-photon excitation probability. The onset of high-order effects in the multi-photon excitation in the direct laser-nucleus interaction is also revealed. Our findings open new possibilities to study the multi-photon laser-nucleus interaction in high-power laser facilities.

nucl-th↗

arXiv2025

All-vortex nonlinear Compton scattering in a polarized laser field

The process of all-vortex nonlinear Compton scattering in an intense and polarized laser field, in which the initial and final electrons and the emitted $γ$ photon are all in vortex states, is studied theoretically. We develop a formalism for the process, which allows us to study the exchanges of the orbital angular momentum (OAM) and spin angular momentum among the electron, $γ$ photon, and laser. A wave packet of the Bessel-Gaussian type is adopted to describe the initial vortex electron for the purpose of normalization. Both circularly and linearly polarized lasers are examined. Substantial numerical calculations are performed to reveal the physics of the exchanges of the OAM and spin angular momentum. The strong impact of the laser intensity and the opening angle of the initial vortex electron is demonstrated. Our results also suggest possible scenarios for the separation of the emitted $γ$ photons with different OAMs, as well as a possible way which could help to distinguish the OAM and spin of the vortex particle by the multi-peak structure in the spectrum of the emitted photon.

hep-ph↗

arXiv2025

Probing Nuclear Excitation by Electron Capture in an Electron Beam Ion Trap with Non-destructive Isomer Detection via Precision Mass Spectrometry

Nuclear excitation by electron capture (NEEC) is an important nuclear excitation mechanism which still lacks conclusive experimental verification. This is primarily attributed to strong background x-/$γ$-ray noise and competing nuclear excitation processes which would overshadow the signals in various environments that NEEC takes place. Here, we propose an experimental approach to observe the NEEC process within a background-free environment. Through collisions with a highly-compressed mono-energetic electron beam in an electron beam ion trap, nuclei may get excited to a long-lived isomeric state via the NEEC process. Subsequently, ions can be extracted and Penning-trap mass spectrometry employed to unambiguously detect the isomer. Our study focuses on the promising candidate $^{189}\mathrm{Os}$, demonstrating measurable detection rates of the NEEC process and discussing the feasibility of the proposed approach. This new approach for observing the NEEC process may be realized in the near future.

nucl-ex↗

arXiv2024

EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations

Current Large Language Models (LLMs) for understanding proteins primarily treats amino acid sequences as a text modality. Meanwhile, Protein Language Models (PLMs), such as ESM-2, have learned massive sequential evolutionary knowledge from the universe of natural protein sequences. Furthermore, structure-based encoders like ProteinMPNN learn the structural information of proteins through Graph Neural Networks. However, whether the incorporation of protein encoders can enhance the protein understanding of LLMs has not been explored. To bridge this gap, we propose EvoLlama, a multimodal framework that connects a structure-based encoder, a sequence-based protein encoder and an LLM for protein understanding. EvoLlama consists of a ProteinMPNN structure encoder, an ESM-2 protein sequence encoder, a multimodal projector to align protein and text representations and a Llama-3 text decoder. To train EvoLlama, we fine-tune it on protein-oriented instructions and protein property prediction datasets verbalized via natural language instruction templates. Our experiments show that EvoLlama's protein understanding capabilities have been significantly enhanced, outperforming other fine-tuned protein-oriented LLMs in zero-shot settings by an average of 1%-8% and surpassing the state-of-the-art baseline with supervised fine-tuning by an average of 6%. On protein property prediction datasets, our approach achieves promising results that are competitive with state-of-the-art task-specific baselines. We will release our code in a future version.

cs.LG↗

arXiv2024

AntLM: Bridging Causal and Masked Language Models

Causal Language Modeling (CLM) and Masked Language Modeling (MLM) are two mainstream learning paradigms based on Transformer networks, specifically the Decoder-only and Encoder-only architectures. The strengths of each paradigm in downstream tasks have shown a mix of advantages and disadvantages. In the past BabyLM Challenge 2023, although the MLM paradigm achieved the best average performance, the CLM paradigm demonstrated significantly faster convergence rates. For the BabyLM Challenge 2024, we propose a novel language modeling paradigm named $\textbf{AntLM}$, which integrates both CLM and MLM to leverage the advantages of these two classic paradigms. We chose the strict-small track and conducted experiments on two foundation models: BabyLlama, representing CLM, and LTG-BERT, representing MLM. During the training process for specific foundation models, we alternate between applying CLM or MLM training objectives and causal or bidirectional attention masks. Experimental results show that combining the two pretraining objectives leverages their strengths, enhancing overall training performance. Under the same epochs, $AntLM_{BabyLlama}$ improves Macro-average by 1%, and $AntLM_{LTG-BERT}$ achieves a 2.2% increase over the baselines.

cs.CL↗

arXiv2024

Generation with Dynamic Vocabulary

We introduce a new dynamic vocabulary for language models. It can involve arbitrary text spans during generation. These text spans act as basic generation bricks, akin to tokens in the traditional static vocabularies. We show that, the ability to generate multi-tokens atomically improve both generation quality and efficiency (compared to the standard language model, the MAUVE metric is increased by 25%, the latency is decreased by 20%). The dynamic vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. For example, we demonstrate that dynamic vocabulary can be applied to different domains in a training-free manner. It also helps to generate reliable citations in question answering tasks (substantially enhancing citation results without compromising answer accuracy).

cs.CL↗

arXiv2024

Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models

Large Vision-Language Models (LVLMs) have achieved impressive performance, yet research has pointed out a serious issue with object hallucinations within these models. However, there is no clear conclusion as to which part of the model these hallucinations originate from. In this paper, we present an in-depth investigation into the object hallucination problem specifically within the CLIP model, which serves as the backbone for many state-of-the-art vision-language systems. We unveil that even in isolation, the CLIP model is prone to object hallucinations, suggesting that the hallucination problem is not solely due to the interaction between vision and language modalities. To address this, we propose a counterfactual data augmentation method by creating negative samples with a variety of hallucination issues. We demonstrate that our method can effectively mitigate object hallucinations for CLIP model, and we show the the enhanced model can be employed as a visual encoder, effectively alleviating the object hallucination issue in LVLMs.

cs.CV↗

arXiv2024

CERD: A Comprehensive Chinese Rhetoric Dataset for Rhetorical Understanding and Generation in Essays

Existing rhetorical understanding and generation datasets or corpora primarily focus on single coarse-grained categories or fine-grained categories, neglecting the common interrelations between different rhetorical devices by treating them as independent sub-tasks. In this paper, we propose the Chinese Essay Rhetoric Dataset (CERD), consisting of 4 commonly used coarse-grained categories including metaphor, personification, hyperbole and parallelism and 23 fine-grained categories across both form and content levels. CERD is a manually annotated and comprehensive Chinese rhetoric dataset with five interrelated sub-tasks. Unlike previous work, our dataset aids in understanding various rhetorical devices, recognizing corresponding rhetorical components, and generating rhetorical sentences under given conditions, thereby improving the author's writing proficiency and language usage skills. Extensive experiments are conducted to demonstrate the interrelations between multiple tasks in CERD, as well as to establish a benchmark for future research on rhetoric. The experimental results indicate that Large Language Models achieve the best performance across most tasks, and jointly fine-tuning with multiple tasks further enhances performance.

cs.CL↗

arXiv2024

A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications

In recent years, the field of electroencephalography (EEG) analysis has witnessed remarkable advancements, driven by the integration of machine learning and artificial intelligence. This survey aims to encapsulate the latest developments, focusing on emerging methods and technologies that are poised to transform our comprehension and interpretation of brain activity. We delve into self-supervised learning methods that enable the robust representation of brain signals, which are fundamental for a variety of downstream applications. We also explore emerging discriminative methods, including graph neural networks (GNN), foundation models, and large language models (LLMs)-based approaches. Furthermore, we examine generative technologies that harness EEG data to produce images or text, offering novel perspectives on brain activity visualization and interpretation. The survey provides an extensive overview of these cutting-edge techniques, their current applications, and the profound implications they hold for future research and clinical practice. The relevant literature and open-source materials have been compiled and are consistently being refreshed at \url{https://github.com/wpf535236337/LLMs4TS}

eess.SP↗

arXiv2024

Efficient production of $^{229m}$Th via nuclear excitation by electron capture

The nuclear isomeric state $^{229m}$Th with an exceptionally low excitation energy makes the $^{229}$Th isotope a crucial candidate for nuclear clocks and many other applications. Efficient and controllable production of $^{229m}$Th is essential and still remains a challenge. Here we report a novel approach for efficient production of $^{229m}$Th by the excitation of $^{229}$Th to the above-lying excited state at $29.19$ keV energy via the process of nuclear excitation by electron capture (NEEC). We show theoretically that the production rate of $^{229m}$Th per nucleus with accessible conditions can be six orders of magnitude larger than the value experimentally demonstrated using $29$-keV synchrotron radiation for this indirect excitation. With the efficient production of $^{229m}$Th, our results identify scenarios, as well as the characteristic NEEC signature with which NEEC events could be unambiguously identified, for a clear experimental identification of the long-sought NEEC phenomenon.

nucl-th↗

arXiv2024

Angular Momentum-Resolved Inelastic Electron Scattering for Nuclear Giant Resonances

Giant resonances (GRs) provide crucial insights into nuclear physics and astrophysics. Exciting GRs using particles like electrons is effective, yet the angular momentum (AM) transfer of electrons, including both intrinsic spin and orbital degrees of freedom in inelastic scattering, has never been studied. Here, we investigate AM transfer in GRs excited by plane-wave and vortex electrons, developing a comprehensive AM-resolved inelastic electron scattering theory. We find that even plane-wave electrons can model-independently extract transition strengths of higher multipolarity by selecting specific AM states of scattered electrons. Additionally, relativistic vortex electrons with orbital angular momentum (OAM) $\pm1$ can be efficiently generated. Vortex electrons can also be used to extract GR transition strength as in the plane-wave case, regardless of the position of nucleus relative to the beam axis. Furthermore, relativistic vortex electrons with larger OAM can be generated for on-axis nuclei due to AM conservation. Our method offers new perspectives for nuclear structure research and paves the way for generating vortex particles.

nucl-th↗

arXiv2024

Length Generalization of Causal Transformers without Position Encoding

Generalizing to longer sentences is important for recent Transformer-based language models. Besides algorithms manipulating explicit position features, the success of Transformers without position encodings (NoPE) provides a new way to overcome the challenge. In this paper, we study the length generalization property of NoPE. We find that although NoPE can extend to longer sequences than the commonly used explicit position encodings, it still has a limited context length. We identify a connection between the failure of NoPE's generalization and the distraction of attention distributions. We propose a parameter-efficient tuning for searching attention heads' best temperature hyper-parameters, which substantially expands NoPE's context size. Experiments on long sequence language modeling, the synthetic passkey retrieval task and real-world long context tasks show that NoPE can achieve competitive performances with state-of-the-art length generalization algorithms. The source code is publicly accessible

cs.CL↗

arXiv2024

Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is an important subtask of sentiment analysis, which aims to extract the aspects and predict their sentiments. Most existing studies focus on improving the performance of the target domain by fine-tuning domain-specific models (trained on source domains) based on the target domain dataset. Few works propose continual learning tasks for ABSA, which aim to learn the target domain's ability while maintaining the history domains' abilities. In this paper, we propose a Large Language Model-based Continual Learning (\texttt{LLM-CL}) model for ABSA. First, we design a domain knowledge decoupling module to learn a domain-invariant adapter and separate domain-variant adapters dependently with an orthogonal constraint. Then, we introduce a domain knowledge warmup strategy to align the representation between domain-invariant and domain-variant knowledge. In the test phase, we index the corresponding domain-variant knowledge via domain positioning to not require each sample's domain ID. Extensive experiments over 19 datasets indicate that our \texttt{LLM-CL} model obtains new state-of-the-art performance.

cs.CL↗

arXiv2024

Text2MDT: Extracting Medical Decision Trees from Medical Texts

Knowledge of the medical decision process, which can be modeled as medical decision trees (MDTs), is critical to build clinical decision support systems. However, the current MDT construction methods rely heavily on time-consuming and laborious manual annotation. In this work, we propose a novel task, Text2MDT, to explore the automatic extraction of MDTs from medical texts such as medical guidelines and textbooks. We normalize the form of the MDT and create an annotated Text-to-MDT dataset in Chinese with the participation of medical experts. We investigate two different methods for the Text2MDT tasks: (a) an end-to-end framework which only relies on a GPT style large language models (LLM) instruction tuning to generate all the node information and tree structures. (b) The pipeline framework which decomposes the Text2MDT task to three subtasks. Experiments on our Text2MDT dataset demonstrate that: (a) the end-to-end method basd on LLMs (7B parameters or larger) show promising results, and successfully outperform the pipeline methods. (b) The chain-of-thought (COT) prompting method \cite{Wei2022ChainOT} can improve the performance of the fine-tuned LLMs on the Text2MDT test set. (c) the lightweight pipelined method based on encoder-based pretrained models can perform comparably with LLMs with model complexity two magnititudes smaller. Our Text2MDT dataset is open-sourced at \url{https://tianchi.aliyun.com/dataset/95414}, and the source codes are open-sourced at \url{https://github.com/michael-wzhu/text2dt}.

cs.CL↗

arXiv2023

Unlearning with Fisher Masking

Machine unlearning aims to revoke some training data after learning in response to requests from users, model developers, and administrators. Most previous methods are based on direct fine-tuning, which may neither remove data completely nor retain full performances on the remain data. In this work, we find that, by first masking some important parameters before fine-tuning, the performances of unlearning could be significantly improved. We propose a new masking strategy tailored to unlearning based on Fisher information. Experiments on various datasets and network structures show the effectiveness of the method: without any fine-tuning, the proposed Fisher masking could unlearn almost completely while maintaining most of the performance on the remain data. It also exhibits stronger stability compared to other unlearning baselines

cs.LG↗

arXiv2023

A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition

Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets, which always obtain using crowdsourcing. However, it is hard to obtain a unified and correct label via majority voting from multiple annotators for NER due to the large labeling space and complexity of this task. To address this problem, we aim to utilize the original multi-annotator labels directly. Particularly, we propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER. This model learns a token- and content-dependent confidence via an Expectation-Maximization (EM) algorithm by minimizing empirical risk. The true posterior estimator and confidence estimator perform iteratively to update the true posterior and confidence respectively. We conduct extensive experimental results on both real-world and synthetic datasets, which show that our model can improve performance effectively compared with strong baselines.

cs.CL↗

arXiv2023

CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors

Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it. However, it is nontrivial to perform information extraction (IE) tasks with NL-LLMs since the output of the IE task is usually structured and therefore is hard to be converted into plain text. In this paper, we propose to recast the structured output in the form of code instead of natural language and utilize generative LLMs of code (Code-LLMs) such as Codex to perform IE tasks, in particular, named entity recognition and relation extraction. In contrast to NL-LLMs, we show that Code-LLMs can be well-aligned with these IE tasks by designing code-style prompts and formulating these IE tasks as code generation tasks. Experiment results on seven benchmarks show that our method consistently outperforms fine-tuning moderate-size pre-trained models specially designed for IE tasks (e.g., UIE) and prompting NL-LLMs under few-shot settings. We further conduct a series of in-depth analyses to demonstrate the merits of leveraging Code-LLMs for IE tasks.

cs.CL↗