arXiv subjects

Zhichao Yang

Publications and source records attributed to Zhichao Yang.

63 records · Page 4Linked to original sources

arXiv2022

Multi-label Few-shot ICD Coding as Autoregressive Generation with Prompt

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with an average of 3,000+ tokens. This task is challenging due to the high-dimensional space of multi-label assignment (155,000+ ICD code candidates) and the long-tail challenge - Many ICD codes are infrequently assigned yet infrequent ICD codes are important clinically. This study addresses the long-tail challenge by transforming this multi-label classification task into an autoregressive generation task. Specifically, we first introduce a novel pretraining objective to generate free text diagnoses and procedure using the SOAP structure, the medical logic physicians use for note documentation. Second, instead of directly predicting the high dimensional space of ICD codes, our model generates the lower dimension of text descriptions, which then infer ICD codes. Third, we designed a novel prompt template for multi-label classification. We evaluate our Generation with Prompt model with the benchmark of all code assignment (MIMIC-III-full) and few shot ICD code assignment evaluation benchmark (MIMIC-III-few). Experiments on MIMIC-III-few show that our model performs with a marco F1 30.2, which substantially outperforms the previous MIMIC-III-full SOTA model (marco F1 4.3) and the model specifically designed for few/zero shot setting (marco F1 18.7). Finally, we design a novel ensemble learner, a cross attention reranker with prompts, to integrate previous SOTA and our best few-shot coding predictions. Experiments on MIMIC-III-full show that our ensemble learner substantially improves both macro and micro F1, from 10.4 to 14.6 and from 58.2 to 59.1, respectively.

cs.CL↗

arXiv2022

An Automatic SOAP Classification System Using Weakly Supervision And Transfer Learning

In this paper, we introduce a comprehensive framework for developing a machine learning-based SOAP (Subjective, Objective, Assessment, and Plan) classification system without manually SOAP annotated training data or with less manually SOAP annotated training data. The system is composed of the following two parts: 1) Data construction, 2) A neural network-based SOAP classifier, and 3) Transfer learning framework. In data construction, since a manual construction of a large size training dataset is expensive, we propose a rule-based weak labeling method utilizing the structured information of an EHR note. Then, we present a SOAP classifier composed of a pre-trained language model and bi-directional long-short term memory with conditional random field (Bi-LSTM-CRF). Finally, we propose a transfer learning framework that re-uses the trained parameters of the SOAP classifier trained with the weakly labeled dataset for datasets collected from another hospital. The proposed weakly label-based learning model successfully performed SOAP classification (89.99 F1-score) on the notes collected from the target hospital. Otherwise, in the notes collected from other hospitals and departments, the performance dramatically decreased. Meanwhile, we verified that the transfer learning framework is advantageous for inter-hospital adaptation of the model increasing the models' performance in every cases. In particular, the transfer learning approach was more efficient when the manually annotated data size was smaller. We showed that SOAP classification models trained with our weakly labeling algorithm can perform SOAP classification without manually annotated data on the EHR notes from the same hospital. The transfer learning framework helps SOAP classification model's inter-hospital migration with a minimal size of the manually annotated dataset.

cs.CL↗

arXiv2022

Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Context

Language Models (LMs) have performed well on biomedical natural language processing applications. In this study, we conducted some experiments to use prompt methods to extract knowledge from LMs as new knowledge Bases (LMs as KBs). However, prompting can only be used as a low bound for knowledge extraction, and perform particularly poorly on biomedical domain KBs. In order to make LMs as KBs more in line with the actual application scenarios of the biomedical domain, we specifically add EHR notes as context to the prompt to improve the low bound in the biomedical domain. We design and validate a series of experiments for our Dynamic-Context-BioLAMA task. Our experiments show that the knowledge possessed by those language models can distinguish the correct knowledge from the noise knowledge in the EHR notes, and such distinguishing ability can also be used as a new metric to evaluate the amount of knowledge possessed by the model.

cs.IR↗

arXiv2022

Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with average length of 3,000+ tokens. This task is challenging due to a high-dimensional space of multi-label assignment (tens of thousands of ICD codes) and the long-tail challenge: only a few codes (common diseases) are frequently assigned while most codes (rare diseases) are infrequently assigned. This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics, which has been shown to be effective under few-shot setting. To further enhance the performance in medical domain, we propose a knowledge-enhanced longformer by injecting three domain-specific knowledge: hierarchy, synonym, and abbreviation with additional pretraining using contrastive learning. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-the-art method in 14.5% in marco F1 (from 10.3 to 11.8, P<0.001). To further test our model on few-shot setting, we created a new rare diseases coding dataset, MIMIC-III-rare50, on which our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.

cs.CL↗

arXiv2022

High Spatial and Temporal Resolution NIR-IIb Gastrointestinal Imaging in Mice

Conventional biomedical imaging modalities, including endoscopy, X-rays, and magnetic resonance, are invasive and cannot provide sufficient spatial and temporal resolutions for regular imaging of gastrointestinal (GI) tract to guide prognosis and therapy of GI diseases. Here we report a non-invasive method for optical imaging of GI tract. It is based on a new type of lanthanide-doped nanocrystal with near-infrared (NIR) excitation at 980 nm and second NIR window (NIR-IIb) (1500~1700 nm) fluorescence emission at around 1530 nm. The rational design and controlled synthesis of nanocrystals with high brightness have led to an absolute quantum yield (QY) up to 48.6%. Further benefitting from the minimized scattering through the NIR-IIb window, we enhanced the spatial resolution by 3 times compared with the other NIR-IIa (1000~1500 nm) contract agents for GI tract imaging. The approach also led to a high temporal resolution of 8 frames per second, so that the moment of mice intestinal peristalsis happened in one minute can be captured. Furthermore, with a light-sheet imaging system, we demonstrated a three-dimensional (3D) imaging of the stereoscopic structure of the GI tract. Moreover, we successfully translate these advances to diagnose inflammatory bowel disease (IBD) in a pre-clinical model of mice colitis.

physics.med-ph↗

arXiv2021

Axially Overlapped Multi-Focus Light Sheet with Enlarged Field of View

Light sheet fluorescence microscopy provides optical sectioning and is widely used in volumetric imaging of large specimens. However, the axial resolution and the lateral Field of View (FoV) of the system, defined by the light sheet, typically limit each other due to the spatial band product of the excitation objective. Here, we develop a simple multi-focus scheme to extend the FoV, where a Gaussian light sheet can be focused at three or more consecutive positions. Axially overlapped multiple light sheets significantly enlarge the FoV with improved uniformity and negligible loss in axial resolution. By measuring the point spread function of fluorescent beads, we demonstrated that the obtained light sheet has a FoV of 450 um and a maximum axial FWHM of 7.5 um. Compared with the conventional single-focus one, the multi-focus Gaussian light sheet displays a significantly improved optical sectioning ability over the full FoV when imaging cells and zebrafish.

physics.optics↗

arXiv2019

Generating Classical Chinese Poems from Vernacular Chinese

Classical Chinese poetry is a jewel in the treasure house of Chinese culture. Previous poem generation models only allow users to employ keywords to interfere the meaning of generated poems, leaving the dominion of generation to the model. In this paper, we propose a novel task of generating classical Chinese poems from vernacular, which allows users to have more control over the semantic of generated poems. We adapt the approach of unsupervised machine translation (UMT) to our task. We use segmentation-based padding and reinforcement learning to address under-translation and over-translation respectively. According to experiments, our approach significantly improve the perplexity and BLEU compared with typical UMT models. Furthermore, we explored guidelines on how to write the input vernacular to generate better poems. Human evaluation showed our approach can generate high-quality poems which are comparable to amateur poems.

cs.CL↗

arXiv2018

Word Embedding Perturbation for Sentence Classification

In this technique report, we aim to mitigate the overfitting problem of natural language by applying data augmentation methods. Specifically, we attempt several types of noise to perturb the input word embedding, such as Gaussian noise, Bernoulli noise, and adversarial noise, etc. We also apply several constraints on different types of noise. By implementing these proposed data augmentation methods, the baseline models can gain improvements on several sentence classification tasks.

cs.CL↗

arXiv2013

Ohmic Contact Formation Between Metal and AlGaN/GaN Heterostructure via Graphene Insertion

A simple method for the creation of Ohmic contact to 2-D electron gas (2DEG) in AlGaN/GaN high electron-mobility transistors (HEMTs) using Cr/Graphene layer is demonstrated. A weak temperature dependence of this Ohmic contact observed in the range 77 to 300 K precludes thermionic emission or trap-assisted hopping as possible carrier-transport mechanisms. It is suggested that the Cr/Graphene combination acts akin to a doped n-type semiconductor in contact with AlGaN/GaN heterostructure, and promotes carrier transport along percolating Al-lean paths through the AlGaN layer. This new use of graphene offers a simple and reliable method for making Ohmic contacts to AlGaN/GaN heterostructures, circumventing complex additional processing steps involving high temperatures. These results could have important implications for the fabrication and manufacturing of AlGaN/GaN-based microelectronic and optoelectronic devices/sensors of the future.

cond-mat.mtrl-sci↗