arXiv subjects

Da Wu

Publications and source records attributed to Da Wu.

14 recordsLinked to original sources

arXiv2025

Multimodal Integrated Knowledge Transfer to Large Language Models through Preference Optimization with Biomedical Applications

The scarcity of high-quality multimodal biomedical data limits the ability to effectively fine-tune pretrained Large Language Models (LLMs) for specialized biomedical tasks. To address this challenge, we introduce MINT (Multimodal Integrated kNowledge Transfer), a framework that aligns unimodal large decoder models with domain-specific decision patterns from multimodal biomedical data through preference optimization. While MINT supports different optimization techniques, we primarily implement it with the Odds Ratio Preference Optimization (ORPO) framework as its backbone. This strategy enables the aligned LLMs to perform predictive tasks using text-only or image-only inputs while retaining knowledge learnt from multimodal data. MINT leverages an upstream multimodal machine learning (MML) model trained on high-quality multimodal data to transfer domain-specific insights to downstream text-only or image-only LLMs. We demonstrate its effectiveness through two key applications: (1) Rare genetic disease prediction from texts, where MINT uses a multimodal encoder model, trained on facial photos and clinical notes, to generate a preference dataset for aligning a lightweight Llama 3.2-3B-Instruct. Despite relying on text input only, the MINT-derived model outperforms models trained with SFT, RAG, or DPO, and even outperforms Llama 3.1-405B-Instruct. (2) Tissue type classification using cell nucleus images, where MINT uses a vision-language foundation model as the preference generator, containing knowledge learnt from both text and histopathological images to align downstream image-only models. The resulting MINT-derived model significantly improves the performance of Llama 3.2-Vision-11B-Instruct on tissue type classification. In summary, MINT provides an effective strategy to align unimodal LLMs with high-quality multimodal expertise through preference optimization.

q-bio.QM↗

arXiv2025

Spontaneous Surface Charging and Janus Nature of the Hexagonal Boron Nitride-Water Interface

Boron, nitrogen and carbon are neighbors in the periodic table and can form strikingly similar twin structures-hexagonal boron nitride (hBN) and graphene-yet nanofluidic experiments demonstrate drastically different water friction on them. We investigate this discrepancy by probing the interfacial water and atomic-scale properties of hBN using surface-specific vibrational spectroscopy, atomic-resolution atomic force microscopy (AFM), and machine learning-based molecular dynamics. Spectroscopy reveals that pristine hBN acquires significant negative charges upon contacting water at neutral pH, unlike hydrophobic graphene, leading to interfacial water alignment and stronger hydrogen bonding. AFM supports that this charging is not defect-induced. pH-dependent measurements suggest OH- chemisorption and physisorption, which simulations validate as two nearly equally stable states undergoing dynamic exchange. These findings challenge the notion of hBN as chemically inert and hydrophobic, revealing its spontaneous surface charging and Janus nature, and providing molecular insights into its higher water friction compared to carbon surfaces.

physics.chem-ph↗

arXiv2025

Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes

Background: Several studies show that large language models (LLMs) struggle with phenotype-driven gene prioritization for rare diseases. These studies typically use Human Phenotype Ontology (HPO) terms to prompt foundation models like GPT and LLaMA to predict candidate genes. However, in real-world settings, foundation models are not optimized for domain-specific tasks like clinical diagnosis, yet inputs are unstructured clinical notes rather than standardized terms. How LLMs can be instructed to predict candidate genes or disease diagnosis from unstructured clinical notes remains a major challenge. Methods: We introduce RAG-driven CoT and CoT-driven RAG, two methods that combine Chain-of-Thought (CoT) and Retrieval Augmented Generation (RAG) to analyze clinical notes. A five-question CoT protocol mimics expert reasoning, while RAG retrieves data from sources like HPO and OMIM (Online Mendelian Inheritance in Man). We evaluated these approaches on rare disease datasets, including 5,980 Phenopacket-derived notes, 255 literature-based narratives, and 220 in-house clinical notes from Childrens Hospital of Philadelphia. Results: We found that recent foundations models, including Llama 3.3-70B-Instruct and DeepSeek-R1-Distill-Llama-70B, outperformed earlier versions such as Llama 2 and GPT-3.5. We also showed that RAG-driven CoT and CoT-driven RAG both outperform foundation models in candidate gene prioritization from clinical notes; in particular, both methods with DeepSeek backbone resulted in a top-10 gene accuracy of over 40% on Phenopacket-derived clinical notes. RAG-driven CoT works better for high-quality notes, where early retrieval can anchor the subsequent reasoning steps in domain-specific evidence, while CoT-driven RAG has advantage when processing lengthy and noisy notes.

cs.CL↗

arXiv2023

GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Text

Individuals with suspected rare genetic disorders often undergo multiple clinical evaluations, imaging studies, laboratory tests, and genetic tests over a prolonged period of time, a process commonly described as the diagnostic odyssey. Addressing this odyssey has substantial clinical, psychosocial, and economic benefits. Many rare genetic diseases have distinctive facial features that artificial intelligence algorithms can use to facilitate clinical diagnosis, to prioritize candidate diseases for further laboratory or genetic testing, and to support the phenotype-driven reinterpretation of genome or exome sequencing data. Existing methods that use frontal facial photographs were built on conventional convolutional neural networks, rely exclusively on facial images, and cannot capture non-facial phenotypic traits or demographic information that are essential for accurate diagnosis. Here we introduce GestaltMML, a multimodal machine learning approach based solely on the Transformer architecture. It integrates facial images, demographic information (age, sex, ethnicity), and clinical notes (optionally a list of Human Phenotype Ontology terms) to improve prediction accuracy. We evaluate GestaltMML on 528 diseases from the GestaltMatcher Database and on several in-house and published cohorts, including Beckwith-Wiedemann syndrome, Sotos syndrome, NAA10-related neurodevelopmental syndrome, Cornelia de Lange syndrome, and KBG syndrome. GestaltMML improves on the state-of-the-art image-only ensembled model, narrows the diagnostic accuracy gap for patients from under-represented ancestries, and clarifies when multimodal fusion is beneficial and when image-only inference is preferable. The results suggest that GestaltMML can greatly narrow the candidate diagnoses of rare diseases and may facilitate the reinterpretation of sequencing data.

q-bio.QM↗

arXiv2023

Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models

The term "Reversal Curse" refers to the scenario where auto-regressive decoder large language models (LLMs), such as ChatGPT, trained on "A is B" fail to learn "B is A," assuming that B and A are distinct and can be uniquely identified from each other, demonstrating a basic failure of logical deduction. This raises a red flag in the use of GPT models for certain general tasks such as constructing knowledge graphs, considering their adherence to this symmetric principle. In our study, we examined a bidirectional LLM, BERT, and found that it is immune to the reversal curse. Driven by ongoing efforts to construct biomedical knowledge graphs with LLMs, we also embarked on evaluating more complex but essential deductive reasoning capabilities. This process included first training encoder and decoder language models to master the intersection and union operations on two sets and then moving on to assess their capability to infer different combinations of union and intersection operations on three newly created sets. The findings showed that while both encoder and decoder language models, trained for tasks involving two sets (union/intersection), were proficient in such scenarios, they encountered difficulties when dealing with operations that included three sets (various combinations of union and intersection). Our research highlights the distinct characteristics of encoder and decoder models in simple and complex logical reasoning. In practice, the choice between BERT and GPT should be guided by the specific requirements and nature of the task at hand, leveraging their respective strengths in bidirectional context comprehension and sequence prediction.

cs.CL↗

arXiv2023

Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT

We hypothesize that large language models (LLMs) based on the transformer architecture can enable automated detection of clinical phenotype terms, including terms not documented in the HPO. In this study, we developed two types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT as its pre-trained model, and PhenoGPT, a GPT-based model that can be initialized from diverse GPT models, including open-source versions such as GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO recognition tool that combines rule-based and deep learning methods. We found that our methods can extract more phenotype concepts, including novel ones not characterized by HPO. We also performed case studies on biomedical literature to illustrate how new phenotype information can be recognized and extracted. We compared current BERT-based versus GPT-based models for phenotype tagging, in multiple aspects including model architecture, memory usage, speed, accuracy, and privacy protection. We also discussed the addition of a negation step and an HPO normalization layer to the transformer models for improved HPO term tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery of phenotype terms from clinical notes and biomedical literature, facilitating automated downstream tasks to derive new biological insights on human diseases.

q-bio.QM↗

arXiv2023

Deep Reinforcement Learning Empowered Rate Selection of XP-HARQ

The complex transmission mechanism of cross-packet hybrid automatic repeat request (XP-HARQ) hinders its optimal system design. To overcome this difficulty, this letter attempts to use the deep reinforcement learning (DRL) to solve the rate selection problem of XP-HARQ over correlated fading channels. In particular, the long term average throughput (LTAT) is maximized by properly choosing the incremental information rate for each HARQ round on the basis of the outdated channel state information (CSI) available at the transmitter. The rate selection problem is first converted into a Markov decision process (MDP), which is then solved by capitalizing on the algorithm of deep deterministic policy gradient (DDPG) with prioritized experience replay. The simulation results finally corroborate the superiority of the proposed XP-HARQ scheme over the conventional HARQ with incremental redundancy (HARQ-IR) and the XP-HARQ with only statistical CSI.

cs.IT↗

arXiv2023

Classification of integers based on residue classes via modern deep learning algorithms

Judging whether an integer can be divided by prime numbers such as 2 or 3 may appear trivial to human beings, but can be less straightforward for computers. Here, we tested multiple deep learning architectures and feature engineering approaches on classifying integers based on their residues when divided by small prime numbers. We found that the ability of classification critically depends on the feature space. We also evaluated Automated Machine Learning (AutoML) platforms from Amazon, Google and Microsoft, and found that they failed on this task without appropriately engineered features. Furthermore, we introduced a method that utilizes linear regression on Fourier series basis vectors, and demonstrated its effectiveness. Finally, we evaluated Large Language Models (LLMs) such as GPT-4, GPT-J, LLaMA and Falcon, and demonstrated their failures. In conclusion, feature engineering remains an important task to improve performance and increase interpretability of machine-learning models, even in the era of AutoML and LLMs.

cs.LG↗

arXiv2023

The SLE Bubble Measure via Conformal Welding of Quantum Surfaces

We showed that the SLE bubble measure recently constructed by Zhan arises naturally from the conformal welding of two Liouville quantum gravity (LQG) disks. The proof relies on (1) a "quantum version" of the limiting construction of the SLE bubble, (2) the conformal welding between quantum triangles and quantum disks due to Ang, Sun and Yu, and (3) the uniform embedding techniques of Ang, Holden and Sun. As a by-product of our proof, we obtained a decomposition formula of the SLE bubble measure. Furthermore, we provided two applications of our conformal welding results. First, we computed the moments of the conformal radius of the SLE bubble on the upper half plane conditioning on surrounding i. The second application concerns the bulk-boundary correlation function in the Liouville Conformal Field Theory (LCFT). Within probabilistic frameworks, we derived a formula linking the bulk-boundary correlation function in the LCFT to the joint law of left and right quantum boundary lengths, and the quantum area of the two-pointed quantum disk. This relation is used by Ang, Remy, Sun and Zhu in a concurrent work to verify the formula of two-pointed bulk-boundary correlation function in physics predicted by Hosomichi (2001).

math.PR↗

arXiv2020

Asymptotic Enumeration of Binary Contingency Tables and Comparison with Independence Heuristic

For parameters $n,\delta,B,C$, we obtained a sharp asymptotic formula for the number of $(n+\lfloor n^\delta\rfloor)^2$-dimensional binary contingency tables with non-uniform margins taking values of $\lfloor BCn\rfloor$ and $\lfloor Cn\rfloor$. Furthermore, we compared our sharp asymptotics with the classical independence heuristic estimate and proved that the independence heuristic overestimates by a factor of $e^{\Theta(n^{2\delta})}$. Our comparison is based on the analysis of the correlation ratio and an explicit bound for the constant in $\Theta$ is also obtained.

math.CO↗

arXiv2020

A Short Note on Asymptotic Enumeration of Contingency Tables with Non-Uniform Margins

In this short note, we compute the precise asymptotics for the number of contingency tables with non-uniform margins. More precisely, for parameter $n,\delta, B,C>0$, we consider the set of matrices whose first $[n^\delta]$ rows and columns have sum $[BCn]$ and the rest $n$ rows and columns have sum $[Cn]$. We compute the precise asymptotics of the cardinality of this set when $B<B_c=1+\sqrt{1+1/C}$ using the maximal entropy methods developed by Barvinok and Hartigan. The only contribution of this note is a detailed expansion of the determinant of quadratic forms in asymptotic formulas.

math.CO↗

arXiv2020

Asymptotic Properties of Random Contingency Tables with Uniform Margin

Let $C\geq 2$ be a positive integer. Consider the set of $n\times n$ non-negative integer matrices whose row sums and column sums are all equal to $Cn$ and let $X=(X_{ij})_{1\leq i,j\leq n}$ be uniformly distributed on this set. This $X$ is called the random contingency table with uniform margin. In this paper, we study various asymptotic properties of $X=(X_{ij})_{1\leq i,j\leq n}$ as $n\to\infty$.

math.PR↗

arXiv2020

$\beta$-High Resolution ODE and Phase Transition between NAG-SC and Heavy Ball Method

In this paper, we study the convergence properties of an algorithm that can be viewed as an interpolation between two gradient based optimization methods, Nesterov's acceleration method for strongly convex functions $(NAG$-$SC)$ and Polyak's heavy ball method. Recent Progress has been made on using High-Resolution ordinary differential equations (ODEs) to distinguish these two fundamentally different methods. The key difference between them can be attributed to the gradient correction term, which is reflected by the Hessian term in the High-Resolution ODE. Our goal is to understand how this term can affect the convergence rate and the choice of our step size. To achieve this goal, we introduce the notion of $\beta$-High Resolution ODE, $0\leq \beta\leq 1$ and prove that within certain range of step size, there is a phase transition happening at $\beta_c$. When $\beta_c\leq\beta\leq 1$, the algorithm associated with $\beta$-High Resolution ODE have the same convergence rate as NAG-SC. When $0\leq \beta\leq \beta_c$, this algorithm will have the slower convergence rate than NAG-SC.

math.OC↗

arXiv2020

On Properties of Random Binary Contingency Tables with Non-Uniform Margin

We study the random binary contingency tables with non-uniform margin. More precisely, for parameters $n,\delta,B,C$, we consider $X=(X_{ij})$ with $X_{ij}\in \lbrace 0,1\rbrace$, the random binary contingency tables whose first $[n^\delta]$ rows and columns have margin $[BCn]$ and the rest columns and rows have margin $[Cn]$. We study various asymptotic properties of $X$ as $n\to \infty$. This answers a question posted by Barvinok.

math.PR↗