arXiv subjects

Junda Wang

Publications and source records attributed to Junda Wang.

28 records · Page 2Linked to original sources

arXiv2024

LEAF: Learning and Evaluation Augmented by Fact-Checking to Improve Factualness in Large Language Models

Large language models (LLMs) have shown remarkable capabilities in various natural language processing tasks, yet they often struggle with maintaining factual accuracy, particularly in knowledge-intensive domains like healthcare. This study introduces LEAF: Learning and Evaluation Augmented by Fact-Checking, a novel approach designed to enhance the factual reliability of LLMs, with a focus on medical question answering (QA). LEAF utilizes a dual strategy to enhance the factual accuracy of responses from models such as Llama 3 70B Instruct and Llama 3 8B Instruct. The first strategy, Fact-Check-Then-RAG, improves Retrieval-Augmented Generation (RAG) by incorporating fact-checking results to guide the retrieval process without updating model parameters. The second strategy, Learning from Fact-Checks via Self-Training, involves supervised fine-tuning (SFT) on fact-checked responses or applying Simple Preference Optimization (SimPO) with fact-checking as a ranking mechanism, both updating LLM parameters from supervision. These findings suggest that integrating fact-checked responses whether through RAG enhancement or self-training enhances the reliability and factual correctness of LLM outputs, offering a promising solution for applications where information accuracy is crucial.

cs.CL↗

arXiv2024

JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability

Large Language Models (LLMs) have demonstrated a remarkable potential in medical knowledge acquisition and question-answering. However, LLMs can potentially hallucinate and yield factually incorrect outcomes, even with domain-specific pretraining. Previously, retrieval augmented generation (RAG) has limited success in addressing hallucinations. Unlike previous methods in RAG where the retrieval model was trained separately from the LLM, we introduce JMLR (for Jointly trains LLM and information Retrieval) during the fine-tuning phase. The synchronized training mechanism enhances JMLR's ability to retrieve clinical guidelines and leverage medical knowledge to reason and answer questions and reduces the demand for computational resources. We evaluated JMLR on the important medical question-answering application. Our experimental results demonstrate that JMLR-13B (70.5%) outperforms a previous state-of-the-art open-source model using conventional pre-training and fine-tuning Meditron-70B (68.9%) and Llama2-13B with RAG (67.7%) on a medical question-answering dataset. Comprehensive evaluations reveal JMLR-13B enhances reasoning quality and reduces hallucinations better than Claude3-Opus. Additionally, JMLR-13B (148 GPU hours) also trains much faster than Meditron-70B (42630 GPU hours). Through this work, we provide a new and efficient knowledge enhancement method for healthcare, demonstrating the potential of integrating retrieval and LLM training for medical question-answering systems.

cs.CL↗

arXiv2024

NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notes

We introduce NoteChat, a novel cooperative multi-agent framework leveraging Large Language Models (LLMs) to generate patient-physician dialogues. NoteChat embodies the principle that an ensemble of role-specific LLMs, through structured role-play and strategic prompting, can perform their assigned roles more effectively. The synergy among these role-playing LLMs results in a cohesive and efficient dialogue generation. Evaluation on MTS-dialogue, a benchmark dataset for patient-physician dialogues-note pairs, shows that models trained with the augmented synthetic patient-physician dialogues by NoteChat outperforms other state-of-the-art models for generating clinical notes. Our comprehensive automatic and human evaluation demonstrates that NoteChat substantially surpasses state-of-the-art models like ChatGPT and GPT-4 up to 22.78% by domain experts in generating superior synthetic patient-physician dialogues based on clinical notes. NoteChat has the potential to engage patients directly and help clinical documentation, a leading cause of physician burnout.

cs.CL↗

arXiv2024

UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration of Prompt Engineering with GPT-4V for Dermatological Diagnosis

This paper presents our team's participation in the MEDIQA-ClinicalNLP2024 shared task B. We present a novel approach to diagnosing clinical dermatology cases by integrating large multimodal models, specifically leveraging the capabilities of GPT-4V under a retriever and a re-ranker framework. Our investigation reveals that GPT-4V, when used as a retrieval agent, can accurately retrieve the correct skin condition 85% of the time using dermatological images and brief patient histories. Additionally, we empirically show that Naive Chain-of-Thought (CoT) works well for retrieval while Medical Guidelines Grounded CoT is required for accurate dermatological diagnosis. Further, we introduce a Multi-Agent Conversation (MAC) framework and show its superior performance and potential over the best CoT strategy. The experiments suggest that using naive CoT for retrieval and multi-agent conversation for critique-based diagnosis, GPT-4V can lead to an early and accurate diagnosis of dermatological conditions. The implications of this work extend to improving diagnostic workflows, supporting dermatological education, and enhancing patient care by providing a scalable, accessible, and accurate diagnostic tool.

cs.AI↗

arXiv2024

Renormalization group of topological scattering networks

Exploring and understanding topological phases in systems with strong distributed disorder requires developing fundamentally new approaches to replace traditional tools such as topological band theory. Here, we present a general real-space renormalization group (RG) approach for scattering models, which is capable of dealing with strong distributed disorder without relying on the renormalization of Hamiltonians or wave functions. Such scheme, based on a block-scattering transformation combined with a replica strategy, is applied for a comprehensive study of strongly disordered unitary scattering networks with localized bulk states, uncovering a connection between topological physics and critical behavior. Our RG scheme leads to topological flow diagrams that unveil how the microscopic competition between reflection and non-reciprocity leads to the large-scale emergence of macroscopic scattering attractors, corresponding to trivial and topological insulators. Our findings are confirmed by a scaling analysis of the localization length (LL) and critical exponents, and experimentally validated. The results not only shed light on the fundamental understanding of topological phase transitions and scaling properties in strongly disordered regimes, but also pave the way for practical applications in modern topological condensed-matter and photonics, where disorder may be seen as a useful design degree of freedom, and no longer as a hindrance.

cond-mat.dis-nn↗

arXiv2023

Can the Black Lives Matter Movement Reduce Racial Disparities? Evidence from Medical Crowdfunding

Using high-frequency donation records from a major medical crowdfunding site and careful difference-in-difference analysis, we demonstrate that the 2020 BLM surge decreased the fundraising gap between Black and non-Black beneficiaries by around 50\%. The reduction is largely attributed to non-Black donors. Those beneficiaries in counties with moderate BLM activities were most impacted. We construct innovative instrumental variable approaches that utilize weekends and rainfall to identify the global and local effects of BLM protests. Results suggest a broad social movement has a greater influence on charitable-giving behavior than a local event. Social media significantly magnifies the impact of protests.

econ.GN↗

arXiv2023

UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?

This paper presents UMASS_BioNLP team participation in the MEDIQA-Chat 2023 shared task for Task-A and Task-C. We focus especially on Task-C and propose a novel LLMs cooperation system named a doctor-patient loop to generate high-quality conversation data sets. The experiment results demonstrate that our approaches yield reasonable performance as evaluated by automatic metrics such as ROUGE, medical concept recall, BLEU, and Self-BLEU. Furthermore, we conducted a comparative analysis between our proposed method and ChatGPT and GPT-4. This analysis also investigates the potential of utilizing cooperation LLMs to generate high-quality datasets.

cs.CL↗

arXiv2022

Causal Inference via Nonlinear Variable Decorrelation for Healthcare Applications

Causal inference and model interpretability research are gaining increasing attention, especially in the domains of healthcare and bioinformatics. Despite recent successes in this field, decorrelating features under nonlinear environments with human interpretable representations has not been adequately investigated. To address this issue, we introduce a novel method with a variable decorrelation regularizer to handle both linear and nonlinear confounding. Moreover, we employ association rules as new representations using association rule mining based on the original features to further proximate human decision patterns to increase model interpretability. Extensive experiments are conducted on four healthcare datasets (one synthetically generated and three real-world collections on different diseases). Quantitative results in comparison to baseline approaches on parameter estimation and causality computation indicate the model's superior performance. Furthermore, expert evaluation given by healthcare professionals validates the effectiveness and interpretability of the proposed model.

cs.LG↗

arXiv2021

Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination

The current development of vaccines for SARS-CoV-2 is unprecedented. Little is known, however, about the nuanced public opinions on the vaccines on social media. We adopt a human-guided machine learning framework using more than six million tweets from almost two million unique Twitter users to capture public opinions on the potential vaccines for SARS-CoV-2, classifying them into three groups: pro-vaccine, vaccine-hesitant, and anti-vaccine. We aggregate opinions at the state and country levels, and find that the major changes in the percentages of different opinion groups roughly correspond to the major pandemic-related events. Interestingly, the percentage of the pro-vaccine group is lower in the Southeast part of the United States. Using multinomial logistic regression, we compare demographics, social capital, income, religious status, political affiliations, geo-locations, sentiment of personal pandemic experience and non-pandemic experience, and county-level pandemic severity perception of these three groups to investigate the scope and causes of public opinions on vaccines. We find that socioeconomically disadvantaged groups are more likely to hold polarized opinions on potential COVID-19 vaccines. People who have the worst personal pandemic experience are more likely to hold the anti-vaccine opinion. Next, by conducting counterfactual analyses, we find that the U.S. public is most concerned about the safety, effectiveness, and political issues regarding potential vaccines for COVID-19, and improving personal pandemic experience increases the vaccine acceptance level. We believe this is the first large-scale social media-based study to analyze public opinions on potential COVID-19 vaccines that can inform more effective vaccine distribution policies and strategies.

cs.SI↗

arXiv2021

How COVID-19 Has Changed Crowdfunding: Evidence From GoFundMe

While the long-term effects of COVID-19 are yet to be determined, its immediate impact on crowdfunding is nonetheless significant. This study takes a computational approach to more deeply comprehend this change. Using a unique data set of all the campaigns published over the past two years on GoFundMe, we explore the factors that have led to the successful funding of a crowdfunding project. In particular, we study a corpus of crowdfunded projects, analyzing cover images and other variables commonly present on crowdfunding sites. Furthermore, we construct a classifier and a regression model to assess the significance of features based on XGBoost. In addition, we employ counterfactual analysis to investigate the causality between features and the success of crowdfunding. More importantly, sentiment analysis and the paired sample t-test are performed to examine the differences in crowdfunding campaigns before and after the COVID-19 outbreak that started in March 2020. First, we note that there is significant racial disparity in crowdfunding success. Second, we find that sad emotion expressed through the campaign's description became significant after the COVID-19 outbreak. Considering all these factors, our findings shed light on the impact of COVID-19 on crowdfunding campaigns.

cs.CY↗