Searcharxiv⌕ Search

arXiv subjects

Xin Gao

Publications and source records attributed to Xin Gao.

At least 253 records · Page 14Linked to original sources

Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning

Federated Learning is a powerful machine learning paradigm to cooperatively train a global model with highly distributed data. A major bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) algorithm for large-scale Federated Learning is the communication overhead on pushing local gradients and pulling global model. In this paper, to reduce the communication complexity of Federated Learning, a novel approach named Pulling Reduction with Local Compensation (PRLC) is proposed. Specifically, each training node intermittently pulls the global model from the server in SGD iterations, resulting in that it is sometimes unsynchronized with the server. In such a case, it will use its local update to compensate the gap between the local model and the global model. Our rigorous theoretical analysis of PRLC achieves two important findings. First, we prove that the convergence rate of PRLC preserves the same order as the classical synchronous SGD for both strongly-convex and non-convex cases with good scalability due to the linear speedup with respect to the number of training nodes. Second, we show that PRLC admits lower pulling frequency than the existing pulling reduction method without local compensation. We also conduct extensive experiments on various machine learning models to validate our theoretical results. Experimental results show that our approach achieves a significant pulling reduction over the state-of-the-art methods, e.g., PRLC requiring only half of the pulling operations of LAG.

cs.LG↗

Extending the Geometry of Heterotic Spectral Cover Constructions

In this work we extend the well-known spectral cover construction first developed by Friedman, Morgan, and Witten to describe more general vector bundles on elliptically fibered Calabi-Yau geometries. In particular, we consider the case in which the Calabi-Yau fibration is not in Weierstrass form, but can rather contain fibral divisors or multiple sections (i.e. a higher rank Mordell-Weil group). In these cases, general vector bundles defined over such Calabi-Yau manifolds cannot be described by ordinary spectral data. To accomplish this we employ well established tools from the mathematics literature of Fourier-Mukai functors. We also generalize existing tools for explicitly computing Fourier-Mukai transforms of stable bundles on elliptic Calabi-Yau manifolds. As an example of these new tools we produce novel examples of chirality changing small instanton transitions. The goal of this work is to provide a geometric formalism that can substantially increase the understood regimes of heterotic/F-theory duality.

hep-th↗

AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration

Deep learning-based health status representation learning and clinical prediction have raised much research interest in recent years. Existing models have shown superior performance, but there are still several major issues that have not been fully taken into consideration. First, the historical variation pattern of the biomarker in diverse time scales plays a vital role in indicating the health status, but it has not been explicitly extracted by existing works. Second, key factors that strongly indicate the health risk are different among patients. It is still challenging to adaptively make use of the features for patients in diverse conditions. Third, using prediction models as the black box will limit the reliability in clinical practice. However, none of the existing works can provide satisfying interpretability and meanwhile achieve high prediction performance. In this work, we develop a general health status representation learning model, named AdaCare. It can capture the long and short-term variations of biomarkers as clinical features to depict the health status in multiple time scales. It also models the correlation between clinical features to enhance the ones which strongly indicate the health status and thus can maintain a state-of-the-art performance in terms of prediction accuracy while providing qualitative interpretability. We conduct a health risk prediction experiment on two real-world datasets. Experiment results indicate that AdaCare outperforms state-of-the-art approaches and provides effective interpretability, which is verifiable by clinical experts.

cs.LG↗

ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context

Predicting the patient's clinical outcome from the historical electronic medical records (EMR) is a fundamental research problem in medical informatics. Most deep learning-based solutions for EMR analysis concentrate on learning the clinical visit embedding and exploring the relations between visits. Although those works have shown superior performances in healthcare prediction, they fail to explore the personal characteristics during the clinical visits thoroughly. Moreover, existing works usually assume that the more recent record weights more in the prediction, but this assumption is not suitable for all conditions. In this paper, we propose ConCare to handle the irregular EMR data and extract feature interrelationship to perform individualized healthcare prediction. Our solution can embed the feature sequences separately by modeling the time-aware distribution. ConCare further improves the multi-head self-attention via the cross-head decorrelation, so that the inter-dependencies among dynamic features and static baseline information can be effectively captured to form the personal health context. Experimental results on two real-world EMR datasets demonstrate the effectiveness of ConCare. The medical findings extracted by ConCare are also empirically confirmed by human experts and medical literature.

cs.LG↗

On Missing Bianchi Identities in Cohomology Formulation

In this article, we perform a deep analysis of the Bianchi identities in the two known formulations developed for the four-dimensional effective type IIA supergravity theory with (non-)geometric fluxes. In what we call the `first formulation', fluxes are expressed in the real six-dimensional indices while in the `second formulation', fluxes are written in the cohomology form. We find that the set of flux constraints arising from these two known formulations are not equivalent, and there are missing identities in the cohomology version which need to be supplemented to match with the first formulation. By analyzing two explicit examples, we conjecture a model independent form for (the most of) the missing identities. These identities have been mostly overlooked in the previous attempts of studying moduli stabilization, particularly for the models developed in the beyond toroidal frameworks, where they could play some important role.

hep-th↗

Heterotic/Heterotic and Heterotic/F-theory Duality

We consider heterotic target space dual (0,2) GLSMs on elliptically fibered Calabi-Yau manifolds. In this context, each half of the "dual" heterotic theories must in turn have an F-theory dual. Moreover, the apparent relationship between two heterotic compactifications seen in (0,2) heterotic target space dual pairs should, in principle, induce some putative correspondence between the dual F-theory geometries. It has previously been conjectured in the literature that (0,2) target space duality might manifest in F-theory as multiple K3-fibrations of the same elliptically fibered Calabi-Yau manifold. In this work we investigate this conjecture in the context of both 6-dimensional and 4-dimensional effective theories and demonstrate that in general, (0,2) target space duality cannot be explained by such a simple phenomenon alone. In all cases, we provide evidence that non-geometric data in F-theory must play at least some role in the induced F-theory correspondence, while leaving the full determination of the putative new F-theory duality to future work.

hep-th↗

Ship Instance Segmentation From Remote Sensing Images Using Sequence Local Context Module

The performance of object instance segmentation in remote sensing images has been greatly improved through the introduction of many landmark frameworks based on convolutional neural network. However, the object densely issue still affects the accuracy of such segmentation frameworks. Objects of the same class are easily confused, which is most likely due to the close docking between objects. We think context information is critical to address this issue. So, we propose a novel framework called SLCMASK-Net, in which a sequence local context module (SLC) is introduced to avoid confusion between objects of the same class. The SLC module applies a sequence of dilation convolution blocks to progressively learn multi-scale context information in the mask branch. Besides, we try to add SLC module to different locations in our framework and experiment with the effect of different parameter settings. Comparative experiments are conducted on remote sensing images acquired by QuickBird with a resolution of $0.5m-1m$ and the results show that the proposed method achieves state-of-the-art performance.

cs.CV↗

Deep learning in bioinformatics: introduction, application, and perspective in big data era

Deep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at \url{https://github.com/lykaust15/Deep_learning_examples}.

q-bio.QM↗

On the Decision Boundary of Deep Neural Networks

While deep learning models and techniques have achieved great empirical success, our understanding of the source of success in many aspects remains very limited. In an attempt to bridge the gap, we investigate the decision boundary of a production deep learning architecture with weak assumptions on both the training data and the model. We demonstrate, both theoretically and empirically, that the last weight layer of a neural network converges to a linear SVM trained on the output of the last hidden layer, for both the binary case and the multi-class case with the commonly used cross-entropy loss. Furthermore, we show empirically that training a neural network as a whole, instead of only fine-tuning the last weight layer, may result in better bias constant for the last weight layer, which is important for generalization. In addition to facilitating the understanding of deep learning, our result can be helpful for solving a broad range of practical problems of deep learning, such as catastrophic forgetting and adversarial attacking. The experiment codes are available at https://github.com/lykaust15/NN_decision_boundary

cs.NE↗

SupportNet: solving catastrophic forgetting in class incremental learning with support data

A plain well-trained deep learning model often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as catastrophic forgetting. Here we propose a novel method, SupportNet, to efficiently and effectively solve the catastrophic forgetting problem in the class incremental learning scenario. SupportNet combines the strength of deep learning and support vector machine (SVM), where SVM is used to identify the support data from the old data, which are fed to the deep learning model together with the new data for further training so that the model can review the essential information of the old data when learning the new information. Two powerful consolidation regularizers are applied to stabilize the learned representation and ensure the robustness of the learned model. We validate our method with comprehensive experiments on various tasks, which show that SupportNet drastically outperforms the state-of-the-art incremental learning methods and even reaches similar performance as the deep learning model trained from scratch on both old and new data. Our program is accessible at: https://github.com/lykaust15/SupportNet

cs.NE↗

PromID: human promoter prediction by deep learning

Computational identification of promoters is notoriously difficult as human genes often have unique promoter sequences that provide regulation of transcription and interaction with transcription initiation complex. While there are many attempts to develop computational promoter identification methods, we have no reliable tool to analyze long genomic sequences. In this work we further develop our deep learning approach that was relatively successful to discriminate short promoter and non-promoter sequences. Instead of focusing on the classification accuracy, in this work we predict the exact positions of the TSS inside the genomic sequences testing every possible location. We studied human promoters to find effective regions for discrimination and built corresponding deep learning models. These models use adaptively constructed negative set which iteratively improves the models discriminative ability. The developed promoter identification models significantly outperform the previously developed promoter prediction programs by considerably reducing the number of false positive predictions. The best model we have built has recall 0.76, precision 0.77 and MCC 0.76, while the next best tool FPROM achieved precision 0.48 and MCC 0.60 for the recall of 0.75. Our method is available at http://www.cbrc.kaust.edu.sa/PromID/.

q-bio.GN↗

Accelerating Flash Calculation through Deep Learning Methods

In the past two decades, researchers have made remarkable progress in accelerating flash calculation, which is very useful in a variety of engineering processes. In this paper, general phase splitting problem statements and flash calculation procedures using the Successive Substitution Method are reviewed, while the main shortages are pointed out. Two acceleration methods, Newton's method and the Sparse Grids Method are presented afterwards as a comparison with the deep learning model proposed in this paper. A detailed introduction from artificial neural networks to deep learning methods is provided here with the authors' own remarks. Factors in the deep learning model are investigated to show their effect on the final result. A selected model based on that has been used in a flash calculation predictor with comparison with other methods mentioned above. It is shown that results from the optimized deep learning model meet the experimental data well with the shortest CPU time. More comparison with experimental data has been conducted to show the robustness of our model.

cs.CE↗

DLBI: Deep learning guided Bayesian inference for structure reconstruction of super-resolution fluorescence microscopy

Super-resolution fluorescence microscopy, with a resolution beyond the diffraction limit of light, has become an indispensable tool to directly visualize biological structures in living cells at a nanometer-scale resolution. Despite advances in high-density super-resolution fluorescent techniques, existing methods still have bottlenecks, including extremely long execution time, artificial thinning and thickening of structures, and lack of ability to capture latent structures. Here we propose a novel deep learning guided Bayesian inference approach, DLBI, for the time-series analysis of high-density fluorescent images. Our method combines the strength of deep learning and statistical inference, where deep learning captures the underlying distribution of the fluorophores that are consistent with the observed time-series fluorescent images by exploring local features and correlation along time-axis, and statistical inference further refines the ultrastructure extracted by deep learning and endues physical meaning to the final image. Comprehensive experimental results on both real and simulated datasets demonstrate that our method provides more accurate and realistic local patch and large-field reconstruction than the state-of-the-art method, the 3B analysis, while our method is more than two orders of magnitude faster. The main program is available at https://github.com/lykaust15/DLBI

cs.CV↗

Symplectic formulation of the type IIA nongeometric scalar potential

We study the four-dimensional (4D) scalar potential arising from a generalized type IIA flux superpotential including the (non-)geometric fluxes. First, we show that using a set of peculiar flux combinations, the 4D scalar potential can be formulated into a very compact form. This is what we call as the `symplectic formulation' from which one could easily anticipate the ten-dimensional origin of the effective scalar potential. We support our formulation through an alternate derivation of the scalar potential via considering the Double Field Theory (DFT) reduction on a generic Calabi Yau orientifold. In addition, we also exemplify the insights of our formulation with explicit computations for two concrete toroidal examples using orientifolds of the complex threefolds ${\mathbb T}^6/{({\mathbb Z}_2 \times {\mathbb Z}_2)}$ and ${\mathbb T}^6/{\mathbb Z}_4$.

hep-th↗

OPA2Vec: combining formal and informal content of biomedical ontologies to improve similarity-based prediction

Motivation: Ontologies are widely used in biology for data annotation, integration, and analysis. In addition to formally structured axioms, ontologies contain meta-data in the form of annotation axioms which provide valuable pieces of information that characterize ontology classes. Annotations commonly used in ontologies include class labels, descriptions, or synonyms. Despite being a rich source of semantic information, the ontology meta-data are generally unexploited by ontology-based analysis methods such as semantic similarity measures. Results: We propose a novel method, OPA2Vec, to generate vector representations of biological entities in ontologies by combining formal ontology axioms and annotation axioms from the ontology meta-data. We apply a Word2Vec model that has been pre-trained on PubMed abstracts to produce feature vectors from our collected data. We validate our method in two different ways: first, we use the obtained vector representations of proteins as a similarity measure to predict protein-protein interaction (PPI) on two different datasets. Second, we evaluate our method on predicting gene-disease associations based on phenotype similarity by generating vector representations of genes and diseases using a phenotype ontology, and applying the obtained vectors to predict gene-disease associations. These two experiments are just an illustration of the possible applications of our method. OPA2Vec can be used to produce vector representations of any biomedical entity given any type of biomedical ontology. Availability: https://github.com/bio-ontology-research-group/opa2vec Contact: robert.hoehndorf@kaust.edu.sa and xin.gao@kaust.edu.sa.

cs.CL↗

RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning

Anonymized electronic medical records are an increasingly popular source of research data. However, these datasets often lack race and ethnicity information. This creates problems for researchers modeling human disease, as race and ethnicity are powerful confounders for many health exposures and treatment outcomes; race and ethnicity are closely linked to population-specific genetic variation. We showed that deep neural networks generate more accurate estimates for missing racial and ethnic information than competing methods (e.g., logistic regression, random forest). RIDDLE yielded significantly better classification performance across all metrics that were considered: accuracy, cross-entropy loss (error), and area under the curve for receiver operating characteristic plots (all $p < 10^{-6}$). We made specific efforts to interpret the trained neural network models to identify, quantify, and visualize medical features which are predictive of race and ethnicity. We used these characterizations of informative features to perform a systematic comparison of differential disease patterns by race and ethnicity. The fact that clinical histories are informative for imputing race and ethnicity could reflect (1) a skewed distribution of blue- and white-collar professions across racial and ethnic groups, (2) uneven accessibility and subjective importance of prophylactic health, (3) possible variation in lifestyle, such as dietary habits, and (4) differences in background genetic variation which predispose to diseases.

q-bio.QM↗

Realization of low-energy Dirac fermions in (Ir$_{1-x}$Pt$_x$)Te$_2$ superconductors

Superconducting and topological states are two distinct quantum states of matter. Superconductors with topologically nontrivial electronic states provide a platform to study the interplay between them. Recent study revealed that a bulk superconductor PdTe2 is a topological Dirac semimetal with type-II Dirac fermions. However, the Dirac fermions in PdTe2 have no contribution to the superconducting pair because the Dirac points reside far below the Fermi level (EF). Here, we show that in the isostructural compounds (Ir1-xPtx)Te2, the type-II Dirac points can be adjusted to EF by element substitution at x ~ 0.1, for which the bulk superconductivity appears near 2 K. The (Ir1-xPtx)Te2 superconductor with the Dirac points at EF paves the way for studying the relevant exotic physical phenomena such as topological superconductivity in Dirac semimetals.

cond-mat.mtrl-sci↗