arXiv subjects

Xu Zuo

Publications and source records attributed to Xu Zuo.

13 recordsLinked to original sources

arXiv2024

Information Extraction from Clinical Notes: Are We Ready to Switch to Large Language Models?

Backgrounds: Information extraction (IE) is critical in clinical natural language processing (NLP). While large language models (LLMs) excel on generative tasks, their performance on extractive tasks remains debated. Methods: We investigated Named Entity Recognition (NER) and Relation Extraction (RE) using 1,588 clinical notes from four sources (UT Physicians, MTSamples, MIMIC-III, and i2b2). We developed an annotated corpus covering 4 clinical entities and 16 modifiers, and compared instruction-tuned LLaMA-2 and LLaMA-3 against BERT in terms of performance, generalizability, computational resources, and throughput to BERT. Results: LLaMA models outperformed BERT across datasets. With sufficient training data, LLaMA showed modest improvements (1% on NER, 1.5-3.7% on RE); improvements were larger with limited training data. On unseen i2b2 data, LLaMA-3-70B outperformed BERT by 7% (F1) on NER and 4% on RE. However, LLaMA models required more computing resources and ran up to 28 times slower. We implemented "Kiwi," a clinical IE package featuring both models, available at https://kiwi.clinicalnlp.org/. Conclusion: This study is among the first to develop and evaluate a comprehensive clinical IE system using open-source LLMs. Results indicate that LLaMA models outperform BERT for clinical NER and RE but with higher computational costs and lower throughputs. These findings highlight that choosing between LLMs and traditional deep learning methods for clinical IE applications should remain task-specific, taking into account both performance metrics and practical considerations such as available computing resources and the intended use case scenarios.

cs.CL↗

arXiv2024

Two-dimensional Rashba semiconductors and inversion-asymmetric topological insulators in monolayer Janus MAA'ZxZ'(4-x) family

The Rashba effect in Janus structures, accompanied by nontrivial topology, plays an important role in spintronics and even photovoltaic applications. Herein, through first-principles calculations, we systematically investigate the geometric stability and electronic structures of 135 kinds of Janus MAA'ZxZ'(4-x) family derived from two-dimensional MA2Z4 (M=Mg, Ga, Sr; A=Al, Ga; Z=S, Se, Te) monolayers, and design numerous Rashba semiconductors and inversion-asymmetric topological insulators. Specifically, there are a total of 26 Rashba semiconductors with isolated spin splitting bands contributed by Se/Te-pz orbitals at conduction band minimum, and the magnitude of the Rashba constant correlates strongly with both the intrinsic electric field and the strength of spin-orbit coupling (SOC). As the atomic number increases, the bandgap of Janus MAA'ZxZ'(4-x) continually decreases until it shrinks to a point where, when SOC is considered, band inversion occurs, leading to a reopening of the bandgap with nontrivial topological phases. In conjunction with band inversion, pz orbitals near the Fermi level can introduce double Rashba splitting featuring a distinctive hybrid spin texture, which can be further effectively adjusted through small biaxial strains and show a continuous evolution of topological to non-topological accompanied by different spin textures. This work provides significant insights into Rashba and topology physics and further presents indispensable inversion asymmetry materials for the development of nonlinear optoelectronics.

physics.app-ph↗

arXiv2024

Improving Entity Recognition Using Ensembles of Deep Learning and Fine-tuned Large Language Models: A Case Study on Adverse Event Extraction from Multiple Sources

Adverse event (AE) extraction following COVID-19 vaccines from text data is crucial for monitoring and analyzing the safety profiles of immunizations. Traditional deep learning models are adept at learning intricate feature representations and dependencies in sequential data, but often require extensive labeled data. In contrast, large language models (LLMs) excel in understanding contextual information, but exhibit unstable performance on named entity recognition tasks, possibly due to their broad but unspecific training. This study aims to evaluate the effectiveness of LLMs and traditional deep learning models in AE extraction, and to assess the impact of ensembling these models on performance. In this study, we utilized reports and posts from the VAERS (n=621), Twitter (n=9,133), and Reddit (n=131) as our corpora. Our goal was to extract three types of entities: "vaccine", "shot", and "ae". We explored and fine-tuned (except GPT-4) multiple LLMs, including GPT-2, GPT-3.5, GPT-4, and Llama-2, as well as traditional deep learning models like RNN and BioBERT. To enhance performance, we created ensembles of the three models with the best performance. For evaluation, we used strict and relaxed F1 scores to evaluate the performance for each entity type, and micro-average F1 was used to assess the overall performance. The ensemble model achieved the highest performance in "vaccine", "shot", and "ae" with strict F1-scores of 0.878, 0.930, and 0.925, respectively, along with a micro-average score of 0.903. In conclusion, this study demonstrates the effectiveness and robustness of ensembling fine-tuned traditional deep learning models and LLMs, for extracting AE-related information. This study contributes to the advancement of biomedical natural language processing, providing valuable insights into improving AE extraction from text data for pharmacovigilance and public health surveillance.

cs.CL↗

arXiv2024

Learning Long-form Video Prior via Generative Pre-Training

Concepts involved in long-form videos such as people, objects, and their interactions, can be viewed as following an implicit prior. They are notably complex and continue to pose challenges to be comprehensively learned. In recent years, generative pre-training (GPT) has exhibited versatile capacities in modeling any kind of text content even visual locations. Can this manner work for learning long-form video prior? Instead of operating on pixel space, it is efficient to employ visual locations like bounding boxes and keypoints to represent key information in videos, which can be simply discretized and then tokenized for consumption by GPT. Due to the scarcity of suitable data, we create a new dataset called \textbf{Storyboard20K} from movies to serve as a representative. It includes synopses, shot-by-shot keyframes, and fine-grained annotations of film sets and characters with consistent IDs, bounding boxes, and whole body keypoints. In this way, long-form videos can be represented by a set of tokens and be learned via generative pre-training. Experimental results validate that our approach has great potential for learning long-form video prior. Code and data will be released at \url{https://github.com/showlab/Long-form-Video-Prior}.

cs.CV↗

arXiv2024

Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations

In acupuncture therapy, the accurate location of acupoints is essential for its effectiveness. The advanced language understanding capabilities of large language models (LLMs) like Generative Pre-trained Transformers (GPT) present a significant opportunity for extracting relations related to acupoint locations from textual knowledge sources. This study aims to compare the performance of GPT with traditional deep learning models (Long Short-Term Memory (LSTM) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT)) in extracting acupoint-related location relations and assess the impact of pretraining and fine-tuning on GPT's performance. We utilized the World Health Organization Standard Acupuncture Point Locations in the Western Pacific Region (WHO Standard) as our corpus, which consists of descriptions of 361 acupoints. Five types of relations ('direction_of,' 'distance_of,' 'part_of,' 'near_acupoint,' and 'located_near') (n= 3,174) between acupoints were annotated. Five models were compared: BioBERT, LSTM, pre-trained GPT-3.5, fine-tuned GPT-3.5, as well as pre-trained GPT-4. Performance metrics included micro-average exact match precision, recall, and F1 scores. Our results demonstrate that fine-tuned GPT-3.5 consistently outperformed other models in F1 scores across all relation types. Overall, it achieved the highest micro-average F1 score of 0.92. This study underscores the effectiveness of LLMs like GPT in extracting relations related to acupoint locations, with implications for accurately modeling acupuncture knowledge and promoting standard implementation in acupuncture training and practice. The findings also contribute to advancing informatics applications in traditional and complementary medicine, showcasing the potential of LLMs in natural language processing.

cs.CL↗

arXiv2023

Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering

Objective: This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks and proposes task-specific prompts to improve their performance. Materials and Methods: We evaluated these models on two clinical NER tasks: (1) to extract medical problems, treatments, and tests from clinical notes in the MTSamples corpus, following the 2010 i2b2 concept extraction shared task, and (2) identifying nervous system disorder-related adverse events from safety reports in the vaccine adverse event reporting system (VAERS). To improve the GPT models' performance, we developed a clinical task-specific prompt framework that includes (1) baseline prompts with task description and format specification, (2) annotation guideline-based prompts, (3) error analysis-based instructions, and (4) annotated samples for few-shot learning. We assessed each prompt's effectiveness and compared the models to BioClinicalBERT. Results: Using baseline prompts, GPT-3.5 and GPT-4 achieved relaxed F1 scores of 0.634, 0.804 for MTSamples, and 0.301, 0.593 for VAERS. Additional prompt components consistently improved model performance. When all four components were used, GPT-3.5 and GPT-4 achieved relaxed F1 socres of 0.794, 0.861 for MTSamples and 0.676, 0.736 for VAERS, demonstrating the effectiveness of our prompt framework. Although these results trail BioClinicalBERT (F1 of 0.901 for the MTSamples dataset and 0.802 for the VAERS), it is very promising considering few training samples are needed. Conclusion: While direct application of GPT models to clinical NER tasks falls short of optimal performance, our task-specific prompt framework, incorporating medical knowledge and training samples, significantly enhances GPT models' feasibility for potential clinical applications.

cs.CL↗

arXiv2021

Emerging Oscillating Reactions at the Insulator/Semiconductor Solid/Solid Interface via Proton Implantation

Most oscillating reactions (ORs) happen in solutions. Few existing solid-based ORs either happen on solid/gas (e.g., oxidation or corrosion) or solid/liquid interfaces, or at the all-solid interfaces neighboring to metals or ionic conductors (e.g., electrolysis or electroplate). We report in this paper a new type of all-solid based OR that happens at the insulator (amorphous SiO$_2$)/semiconductor (Si) interface with the interfacial point defects as the oscillating species. This OR is the first example of the point-defect coupled ORs (PDC-ORs) proposed by H. Schmalzried et al. and J. Janek et al. decades ago. We use proton implantation as the driving force of the oscillation, and employ techniques common in semiconductor device characterization to monitor the oscillation in situ. This approach not only overcomes the difficulties associated with detecting reactions in solids, but also accurately measure the oscillating ultra-low concentration ($10^{10}\sim10^{11}$ cm$^{-2}$) of the interfacial charged point-defects. We propose a mechanism for the reported PDC-OR based on the Brusselator model by identifying the interfacial reactions.

physics.chem-ph↗

arXiv2021

Transverse Rashba Effect and Unconventional Magnetocrystalline Anisotropy in Double-Gd-adsorbed Zigzag Graphene Nanoribbon

The transverse Rashba effect is proposed and investigated by the first-principle calculations based on density functional theory in a quasi-one-dimensional antiferromagnet with a strong perpendicular magnetocrystalline anisotropy, which is materialized by the Gd-adsorbed graphene nanoribbon with a centric symmetry. The Rashba effect in this system is associated with the local dipole field transverse to and in the plane of the nanoribbon. That dipole field is induced by the off-center adsorption of the Gd adatom above the hex-carbon ring near the nanoribbon edges. The transverse Rashba effect at the two Gd adatoms enhances each other in the antiferromagnetic (AFM) ground state and cancels each other in the ferromagnetic (FM) meta-stable state, because of the centrosymmetric atomic structure. The transverse Rashba parameter is 1.51 eV A. This system shows a strong perpendicular magnetocrystalline anisotropy (MCA), which is 1.4 meV per Gd atom in the AFM state or 2.2 meV per Gd atom in the FM state. The origin of the perpendicular MCA is analyzed in k-space by filtering out the contribution of the transverse Rashba effect from the band structures perturbed by the spin-orbit coupling interactions. The first-order perturbation of the orbit and spin angular momentum coupling is the major source of the MCA, which is associated with the one-dimensionality of the system. The transverse Rashba effect and the strong perpendicular magnetization hosted simultaneously by the proposed AFM Gd-adsorbed graphene nanoribbon lock the up- (or down-) spin quantization direction to the forward (or backward) movement. This finding offers a magnetic approach to a high coherency spin propagation in one-dimensionality, and open a new door to manipulating spin transportation in graphene-based spintronics.

physics.comp-ph↗

arXiv2020

Universal Analytic Model of Irradiation Defect Dynamics in Silica-Silicon Structures

Irradiation damage is a key physics issue for semiconductor devices under extreme environments. For decades, the ionization-irradiation-induced damage in transistors with silica-silicon structures under constant dose rate is modeled by a uniform generation of $E'$ centers in the bulk silica region and their irreversible conversion to $P_b$ centers at the silica-silicon interface. But, the traditional model fails to explain experimentally observed dependence of the defect concentrations on dose, especially at low dose rate. Here, we propose that, the generation of $E'$ is decelerated due to the dispersive diffusion of induced holes in the disordered silica and the conversion of $P_b$ is reversible due to recombination-enhanced defect reactions under irradiation. It is shown that the derived analytic model based on these new understandings can consistently explain the fundamental but puzzling dependence of the defect concentrations on dose and dose rate in a wide range.

physics.app-ph↗

arXiv2017

Unconventional magnetic anisotropy in one-dimentional Rashba system realized by adsorbing Gd atom on zigzag graphene nanoribbons

The Rashba effect, a spin splitting in electronic band structure, attracts much attention for the potential applications in spintronics with no requirement of external magnetic field. Realizing one-dimensional (1D) Rashba system is a big challenge due to the difficulties of growing high-quality heavy-metal nanowires or introducing strong spin-orbit coupling (SOC) and broken inversion symmetry in flexible materials. Here, based on first-principles calculations, we propose a pathway to realize the Rashba spin-split by adsorbing Gd atom on zigzag graphene nanoribbons (Gd-ZGNR) and further investigate the magnetic anisotropy energy (MAE). Perpendicular MAE and unconventional MAE contributions in k-space are found in the self-assembled Gd-ZGNR system, which present a remarkable Rashba effect (the estimated strength is 1.89 eV Å) attributed to strong SOC (~65.6 meV) and the asymmetric adsorption site at the nanoribbons edge. Moreover, first-order MAE is connected to the intrinsic Rashba effect beyond the traditional second-order MAE, which is confirmed based on the analysis of electronic structures perturbed with SOC in comparison with metastable Gd-ZGNR at central symmetric adsorption site. The dependence of first-order MAE as well as Rashba effect of Gd-ZGNRs on the ribbon width are also examined. This work offers new perspective to achieve 1D Rashba system and provides fundamental understanding on the magnetic anisotropy, which would be of great significance for searching Majorana fermions and promoting the potential applications in spintronics.

physics.comp-ph↗

arXiv2013

Manipulating the magnetic anisotropy of cobalt doped titanium dioxide by carrier accumulation

Based on first-principles calculations, we predict that the magnetic anisotropy energy (MAE) of Co-doped TiO$_2$ sensitively depends on carrier accumulation. This magnetoelectric phenomenon provides a promising route to directly manipulate the magnetization direction of diluted magnetic semiconductor by external electric-fields. We calculate the band structures and reveal the origin of carrier-dependent MAE in k-space. In fact, the carrier accumulation shifts the Fermi energy and regulates the competing contributions to MAE. The first-principles calculations provide a straightforward way to design spintronics materials with electrically controllable spin direction.

cond-mat.mtrl-sci↗

arXiv2013

Ab initio study of magnetic anisotropy in cobalt doped zinc oxide with electron-filling

Based on first-principles calculation, it has been predicted that the magnetic anisotropy energy (MAE) in Co-doped ZnO (Co:ZnO) depends on electron-filling. Results show that the charge neutral Co:ZnO presents a "easy plane" magnetic state. While modifying the total number of electrons, the easy axis rotates from in-plane to out-of-plane. The alternation of the MAE is considered to be the change of the ground state of Co ion, resulting from the relocating of electrons on Co d-orbitals with electron-filling.

cond-mat.mtrl-sci↗

arXiv2012

Ab initio calculation of the local magnetic moment in titanium doped zinc oxide with a corrected-band-gap scheme

The local magnetic moment of Ti:ZnO is calculated from first principles by using the corrected-band-gap scheme (CBGS). The results shows that the system is magnetic with the magnetization of 0.699 $μ_B$ per dopant. The origin of the local magnetic moment is considered to be the impurity band partially occupied by the donor electrons in the conduction band. Further, the impacts of applying Hubbard U to Ti-d orbital on the magnetic moment have been investigated.

cond-mat.mtrl-sci↗