Searcharxiv⌕ Search

arXiv subjects

Lin Qiu

Publications and source records attributed to Lin Qiu.

At least 55 records · Page 3Linked to original sources

ECon: On the Detection and Resolution of Evidence Conflicts

The rise of large language models (LLMs) has significantly influenced the quality of information in decision-making systems, leading to the prevalence of AI-generated content and challenges in detecting misinformation and managing conflicting information, or "inter-evidence conflicts." This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios. We evaluate conflict detection methods, including Natural Language Inference (NLI) models, factual consistency (FC) models, and LLMs, on these conflicts (RQ1) and analyze LLMs' conflict resolution behaviors (RQ2). Our key findings include: (1) NLI and LLM models exhibit high precision in detecting answer conflicts, though weaker models suffer from low recall; (2) FC models struggle with lexically similar answer conflicts, while NLI and LLM models handle these better; and (3) stronger models like GPT-4 show robust performance, especially with nuanced conflicts. For conflict resolution, LLMs often favor one piece of conflicting evidence without justification and rely on internal knowledge if they have prior beliefs.

cs.CL↗

Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?

Analogical reasoning plays a critical role in human cognition, enabling us to understand new concepts by associating them with familiar ones. Previous research in the AI community has mainly focused on identifying and generating analogies and then examining their quality under human evaluation, which overlooks the practical application of these analogies in real-world settings. Inspired by the human education process, in this paper, we propose to investigate how analogies created by teacher language models (LMs) can assist student LMs in understanding scientific concepts, thereby aligning more closely with practical scenarios. Our results suggest that free-form analogies can indeed aid LMs in understanding concepts. Additionally, analogies generated by student LMs can improve their own performance on scientific question answering, demonstrating their capability to use analogies for self-learning new knowledge. Resources are available at https://github.com/siyuyuan/SCUA.

cs.CL↗

$J/ψ$ photoproduction near threshold and signals for the hidden charm pentaquarks

We study the $J/ψ$ photoproduction $γp\to J/ψp$ near threshold to investigate the role played by the open charm channels and possible signals from the hidden charm $P_c$ pentaquark states. With the diffractive mechanism described by a Pomeron exchange model extrapolated from high energies to the lower energy region, it shows that the differential cross sections compared with the recent results from the GlueX Collaboration have apparent deficits in the large scattering angles. The inclusion of the open charm channels and intermediate $P_c$ states can significantly improve the descriptions of the differential cross section data, in particular, at the energy regions of the $Λ_c\bar{D}^{(*)}$ thresholds. This can explain the structures observed by the GlueX Collaboration at the $Λ_c\bar{D}^{(*)}$ thresholds as the open charm CUSP effects. Given that these $P_c$ pentaquark states are hadronic molecules dynamically generated by the $Σ_c\bar{D}^{(*)}$, we find that the production of these $P_c$ states should be suppressed at leading order since their couplings to both $γp$ and $J/ψp$ are through loop transitions. This can explain why no obvious signals for the $P_c$ states are observed by the present datasets. In order to further disentangling the role played by $s$-channel mechanisms, we also investigate the polarized beam asymmetry which shows sensitivities to the open charm threshold effects and interferences from the $P_c$ productions. Experimental measurement of this observable at the Jefferson Laboratory is strongly encouraged.

hep-ph↗

Toward a More Complete OMR Solution

Optical music recognition (OMR) aims to convert music notation into digital formats. One approach to tackle OMR is through a multi-stage pipeline, where the system first detects visual music notation elements in the image (object detection) and then assembles them into a music notation (notation assembly). Most previous work on notation assembly unrealistically assumes perfect object detection. In this study, we focus on the MUSCIMA++ v2.0 dataset, which represents musical notation as a graph with pairwise relationships among detected music objects, and we consider both stages together. First, we introduce a music object detector based on YOLOv8, which improves detection performance. Second, we introduce a supervised training pipeline that completes the notation assembly stage based on detection output. We find that this model is able to outperform existing models trained on perfect detection output, showing the benefit of considering the detection and assembly stages in a more holistic way. These findings, together with our novel evaluation metric, are important steps toward a more complete OMR solution.

cs.CV↗

The Tug-of-War Between Deepfake Generation and Detection

Multimodal generative models are rapidly evolving, leading to a surge in the generation of realistic video and audio that offers exciting possibilities but also serious risks. Deepfake videos, which can convincingly impersonate individuals, have particularly garnered attention due to their potential misuse in spreading misinformation and creating fraudulent content. This survey paper examines the dual landscape of deepfake video generation and detection, emphasizing the need for effective countermeasures against potential abuses. We provide a comprehensive overview of current deepfake generation techniques, including face swapping, reenactment, and audio-driven animation, which leverage cutting-edge technologies like GANs and diffusion models to produce highly realistic fake videos. Additionally, we analyze various detection approaches designed to differentiate authentic from altered videos, from detecting visual artifacts to deploying advanced algorithms that pinpoint inconsistencies across video and audio signals. The effectiveness of these detection methods heavily relies on the diversity and quality of datasets used for training and evaluation. We discuss the evolution of deepfake datasets, highlighting the importance of robust, diverse, and frequently updated collections to enhance the detection accuracy and generalizability. As deepfakes become increasingly indistinguishable from authentic content, developing advanced detection techniques that can keep pace with generation technologies is crucial. We advocate for a proactive approach in the "tug-of-war" between deepfake creators and detectors, emphasizing the need for continuous research collaboration, standardization of evaluation metrics, and the creation of comprehensive benchmarks.

cs.CV↗

RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation

Despite Retrieval-Augmented Generation (RAG) showing promising capability in leveraging external knowledge, a comprehensive evaluation of RAG systems is still challenging due to the modular nature of RAG, evaluation of long-form responses and reliability of measurements. In this paper, we propose a fine-grained evaluation framework, RAGChecker, that incorporates a suite of diagnostic metrics for both the retrieval and generation modules. Meta evaluation verifies that RAGChecker has significantly better correlations with human judgments than other evaluation metrics. Using RAGChecker, we evaluate 8 RAG systems and conduct an in-depth analysis of their performance, revealing insightful patterns and trade-offs in the design choices of RAG architectures. The metrics of RAGChecker can guide researchers and practitioners in developing more effective RAG systems. This work has been open sourced at https://github.com/amazon-science/RAGChecker.

cs.CL↗

EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs

Narrative reasoning relies on the understanding of eventualities in story contexts, which requires a wealth of background world knowledge. To help machines leverage such knowledge, existing solutions can be categorized into two groups. Some focus on implicitly modeling eventuality knowledge by pretraining language models (LMs) with eventuality-aware objectives. However, this approach breaks down knowledge structures and lacks interpretability. Others explicitly collect world knowledge of eventualities into structured eventuality-centric knowledge graphs (KGs). However, existing research on leveraging these knowledge sources for free-texts is limited. In this work, we propose an initial comprehensive framework called EventGround, which aims to tackle the problem of grounding free-texts to eventuality-centric KGs for contextualized narrative reasoning. We identify two critical problems in this direction: the event representation and sparsity problems. We provide simple yet effective parsing and partial information extraction methods to tackle these problems. Experimental results demonstrate that our approach consistently outperforms baseline models when combined with graph neural network (GNN) or large language model (LLM) based graph reasoning models. Our framework, incorporating grounded knowledge, achieves state-of-the-art performance while providing interpretable evidence.

cs.CL↗

RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models

Large Language Models (LLMs) have shown impressive capabilities but also a concerning tendency to hallucinate. This paper presents RefChecker, a framework that introduces claim-triplets to represent claims in LLM responses, aiming to detect fine-grained hallucinations. In RefChecker, an extractor generates claim-triplets from a response, which are then evaluated by a checker against a reference. We delineate three task settings: Zero, Noisy and Accurate Context, to reflect various real-world use cases. We curated a benchmark spanning various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven LLMs. RefChecker supports both proprietary and open-source models as the extractor and checker. Experiments demonstrate that claim-triplets enable superior hallucination detection, compared to other granularities such as response, sentence and sub-sentence level claims. RefChecker outperforms prior methods by 6.8 to 26.1 points on our benchmark and the checking results of RefChecker are strongly aligned with human judgments. This work is open sourced at https://github.com/amazon-science/RefChecker

cs.CL↗

Coupled-channel description of charmed heavy hadronic molecules within the meson-exchange model and its implication

Motivated by the first observation of the double-charm tetraquark $T_{cc}^+(3875)$ by the LHCb Collaboration, we investigate the nature of $T_{cc}^+$ as an isoscalar $DD^*$ hadronic molecule in a meson-exchange potential model incorporated by the coupled-channel effects and three-body unitarity. The $D^0D^0π^+$ invariant mass spectrum can be well-described and the $T_{cc}^+$ pole structure can be precisely extracted. Under the hypothesis that the interactions between the heavy flavor hadrons can be saturated by the light meson-exchange potentials, the near-threshold dynamics of $T_{cc}^+$ can shed light on the binding of its heavy-quark spin symmetry (HQSS) partner $D^*D^*$ ($I=0$) and on the nature of other heavy hadronic molecule candidates such as $X(3872)$ and $Z_c(3900)$ in the charmed-anticharmed systems. The latter states can be related to $T_{cc}^+$ in the meson-exchange potential model with limited assumptions based on the SU(3) flavor symmetry relations. The combined analysis, on the one hand, indicates the HQSS breaking effects among those HQSS partners, and on the other hand, highlights the role played by the short and long-distance dynamics for the near threshold $D^{(*)}D^{(*)}$ and $D^{(*)}\bar{D}^{(*)}+c.c.$ systems.

hep-ph↗

Evaluating Psychological Safety of Large Language Models

In this work, we designed unbiased prompts to systematically evaluate the psychological safety of large language models (LLMs). First, we tested five different LLMs by using two personality tests: Short Dark Triad (SD-3) and Big Five Inventory (BFI). All models scored higher than the human average on SD-3, suggesting a relatively darker personality pattern. Despite being instruction fine-tuned with safety metrics to reduce toxicity, InstructGPT, GPT-3.5, and GPT-4 still showed dark personality patterns; these models scored higher than self-supervised GPT-3 on the Machiavellianism and narcissism traits on SD-3. Then, we evaluated the LLMs in the GPT series by using well-being tests to study the impact of fine-tuning with more training data. We observed a continuous increase in the well-being scores of GPT models. Following these observations, we showed that fine-tuning Llama-2-chat-7B with responses from BFI using direct preference optimization could effectively reduce the psychological toxicity of the model. Based on the findings, we recommended the application of systematic and comprehensive psychological metrics to further evaluate and improve the safety of LLMs.

cs.CL↗

Meeting Bridges: Designing Information Artifacts that Bridge from Synchronous Meetings to Asynchronous Collaboration

A recent surge in remote meetings has led to complaints of ``Zoom fatigue'' and ``collaboration overload,'' negatively impacting worker productivity and well-being. One way to alleviate the burden of meetings is to de-emphasize their synchronous participation by shifting work to and enabling sensemaking during post-meeting asynchronous activities. Towards this goal, we propose the design concept of meeting bridges, or information artifacts that can encapsulate meeting information towards bridging to and facilitating post-meeting activities. Through 13 interviews and a survey of 198 information workers, we learn how people use online meeting information after meetings are over, finding five main uses: as an archive, as task reminders, to onboard or support inclusion, for group sensemaking, and as a launching point for follow-on collaboration. However, we also find that current common meeting artifacts, such as notes and recordings, present challenges in serving as meeting bridges. After conducting co-design sessions with 16 participants, we distill key principles for the design of meeting bridges to optimally support asynchronous collaboration goals. Overall, our findings point to the opportunity of designing information artifacts that not only support users to access but also continue to transform and engage in meeting information post-meeting.

cs.HC↗

Spectral integrated neural networks (SINNs) for solving forward and inverse dynamic problems

This paper proposes a novel neural network framework, denoted as spectral integrated neural networks (SINNs), for resolving three-dimensional forward and inverse dynamic problems. In the SINNs, the spectral integration method is applied to perform temporal discretization, and then a fully connected neural network is adopted to solve resulting partial differential equations (PDEs) in the spatial domain. Specifically, spatial coordinates are employed as inputs in the network architecture, and the output layer is configured with multiple outputs, each dedicated to approximating solutions at different time instances characterized by Gaussian points used in the spectral method. By leveraging the automatic differentiation technique and spectral integration scheme, the SINNs minimize the loss function, constructed based on the governing PDEs and boundary conditions, to obtain solutions for dynamic problems. Additionally, we utilize polynomial basis functions to expand the unknown function, aiming to enhance the performance of SINNs in addressing inverse problems. The conceived framework is tested on six forward and inverse dynamic problems, involving nonlinear PDEs. Numerical results demonstrate the superior performance of SINNs over the popularly used physics-informed neural networks in terms of convergence speed, computational accuracy and efficiency. It is also noteworthy that the SINNs exhibit the capability to deliver accurate and stable solutions for long-time dynamic problems.

math.NA↗

Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus

Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields. However, LLMs are prone to hallucinate untruthful or nonsensical outputs that fail to meet user expectations in many real-world applications. Existing works for detecting hallucinations in LLMs either rely on external knowledge for reference retrieval or require sampling multiple responses from the LLM for consistency verification, making these methods costly and inefficient. In this paper, we propose a novel reference-free, uncertainty-based method for detecting hallucinations in LLMs. Our approach imitates human focus in factuality checking from three aspects: 1) focus on the most informative and important keywords in the given text; 2) focus on the unreliable tokens in historical context which may lead to a cascade of hallucinations; and 3) focus on the token properties such as token type and token frequency. Experimental results on relevant datasets demonstrate the effectiveness of our proposed method, which achieves state-of-the-art performance across all the evaluation metrics and eliminates the need for additional information.

cs.CL↗

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding

Analogy-making between narratives is crucial for human reasoning. In this paper, we evaluate the ability to identify and generate analogies by constructing a first-of-its-kind large-scale story-level analogy corpus, \textsc{StoryAnalogy}, which contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We design a set of tests on \textsc{StoryAnalogy}, presenting the first evaluation of story-level analogy identification and generation. Interestingly, we find that the analogy identification tasks are incredibly difficult not only for sentence embedding models but also for the recent large language models (LLMs) such as ChatGPT and LLaMa. ChatGPT, for example, only achieved around 30% accuracy in multiple-choice questions (compared to over 85% accuracy for humans). Furthermore, we observe that the data in \textsc{StoryAnalogy} can improve the quality of analogy generation in LLMs, where a fine-tuned FlanT5-xxl model achieves comparable performance to zero-shot ChatGPT.

cs.CL↗

An Anchor-Point Based Image-Model for Room Impulse Response Simulation with Directional Source Radiation and Sensor Directivity Patterns

The image model method has been widely used to simulate room impulse responses and the endeavor to adapt this method to different applications has also piqued great interest over the last few decades. This paper attempts to extend the image model method and develops an anchor-point-image-model (APIM) approach as a solution for simulating impulse responses by including both the source radiation and sensor directivity patterns. To determine the orientations of all the virtual sources, anchor points are introduced to real sources, which subsequently lead to the determination of the orientations of the virtual sources. An algorithm is developed to generate room impulse responses with APIM by taking into account the directional pattern functions, factional time delays, as well as the computational complexity. The developed model and algorithms can be used in various acoustic problems to simulate room acoustics and improve and evaluate processing algorithms.

cs.SD↗

Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations

Discourse analysis is an important task because it models intrinsic semantic structures between sentences in a document. Discourse markers are natural representations of discourse in our daily language. One challenge is that the markers as well as pre-defined and human-labeled discourse relations can be ambiguous when describing the semantics between sentences. We believe that a better approach is to use a contextual-dependent distribution over the markers to express discourse information. In this work, we propose to learn a Distributed Marker Representation (DMR) by utilizing the (potentially) unlimited discourse marker data with a latent discourse sense, thereby bridging markers with sentence pairs. Such representations can be learned automatically from data without supervision, and in turn provide insights into the data itself. Experiments show the SOTA performance of our DMR on the implicit discourse relation recognition task and strong interpretability. Our method also offers a valuable tool to understand complex ambiguity and entanglement among discourse markers and manually defined discourse relations.

cs.CL↗

Adaptive physics-informed neural networks for dynamic thermo-mechanical coupling problems in large-size-ratio functionally graded materials

In this paper, we present the adaptive physics-informed neural networks (PINNs) for resolving three dimensional (3D) dynamic thermo-mechanical coupling problems in large-size-ratio functionally graded materials (FGMs). The physical laws described by coupled governing equations and the constraints imposed by the initial and boundary conditions are leveraged to form the loss function of PINNs by means of the automatic differentiation algorithm, and an adaptive loss balancing scheme is introduced to improve the performance of PINNs. The adaptive PINNs are meshfree and trained on batches of randomly sampled collocation points, which is the key feature and superiority of the approach, since mesh-based methods will encounter difficulties in solving problems with large size ratios. The developed methodology is tested for several 3D thermo-mechanical coupling problems in large-size-ratio FGMs, and the numerical results demonstrate that the adaptive PINNs are effective and reliable for dealing with coupled problems in coating structures with large size ratios up to 109, as well as complex large-size-ratio geometries such as the electrostatic comb, the airplane and the submarine.

cs.CE↗

Predictions for feed-down enhancements at the $Λ_c \bar{D}$ and $Λ_c \bar{D}^*$ thresholds via the triangle and box singularities

We demonstrate that triangle singularity (TS) and box singularity (BS) mechanisms can produce unique narrow enhancements at the $Λ_c\bar{D}$ and $Λ_c\bar{D}^*$ thresholds in the invariant mass spectra of $J/ψp$ and $J/ψpπ$, respectively. Taking into account that such mechanisms only depend on the initial $Σ_c^{(*)}\bar{D}^{(*)}$ interactions near threshold within the TS or BS kinematic regimes, the $Λ_c\bar{D}$ and $Λ_c\bar{D}^*$ threshold enhancements can be regarded as a feed-down phenomenon originated from both the heavier pentaquark decays and the $Σ_c^{(*)}\bar{D}^{(*)}$ scatterings from the continuum. A search for these structures in the $J/ψp$ and $J/ψpπ$ spectra in both exclusive and semi-inclusive processes will provide a smoking-gun evidence for the hadronic molecule nature of those observed pentaquarks and clarify the role played by the TS and BS in the near-threshold dynamics.

hep-ph↗