arXiv subjects

Li Wang

Publications and source records attributed to Li Wang.

At least 433 records · Page 24Linked to original sources

arXiv2020

ASFGNN: Automated Separated-Federated Graph Neural Network

Graph Neural Networks (GNNs) have achieved remarkable performance by taking advantage of graph data. The success of GNN models always depends on rich features and adjacent relationships. However, in practice, such data are usually isolated by different data owners (clients) and thus are likely to be Non-Independent and Identically Distributed (Non-IID). Meanwhile, considering the limited network status of data owners, hyper-parameters optimization for collaborative learning approaches is time-consuming in data isolation scenarios. To address these problems, we propose an Automated Separated-Federated Graph Neural Network (ASFGNN) learning paradigm. ASFGNN consists of two main components, i.e., the training of GNN and the tuning of hyper-parameters. Specifically, to solve the data Non-IID problem, we first propose a separated-federated GNN learning model, which decouples the training of GNN into two parts: the message passing part that is done by clients separately, and the loss computing part that is learnt by clients federally. To handle the time-consuming parameter tuning problem, we leverage Bayesian optimization technique to automatically tune the hyper-parameters of all the clients. We conduct experiments on benchmark datasets and the results demonstrate that ASFGNN significantly outperforms the naive federated GNN, in terms of both accuracy and parameter-tuning efficiency.

cs.LG↗

arXiv2020

Multi-Accent Adaptation based on Gate Mechanism

When only a limited amount of accented speech data is available, to promote multi-accent speech recognition performance, the conventional approach is accent-specific adaptation, which adapts the baseline model to multiple target accents independently. To simplify the adaptation procedure, we explore adapting the baseline model to multiple target accents simultaneously with multi-accent mixed data. Thus, we propose using accent-specific top layer with gate mechanism (AST-G) to realize multi-accent adaptation. Compared with the baseline model and accent-specific adaptation, AST-G achieves 9.8% and 1.9% average relative WER reduction respectively. However, in real-world applications, we can't obtain the accent category label for inference in advance. Therefore, we apply using an accent classifier to predict the accent label. To jointly train the acoustic model and the accent classifier, we propose the multi-task learning with gate mechanism (MTL-G). As the accent label prediction could be inaccurate, it performs worse than the accent-specific adaptation. Yet, in comparison with the baseline model, MTL-G achieves 5.1% average relative WER reduction.

eess.AS↗

arXiv2020

Domain Adaptation Using Class Similarity for Robust Speech Recognition

When only limited target domain data is available, domain adaptation could be used to promote performance of deep neural network (DNN) acoustic model by leveraging well-trained source model and target domain data. However, suffering from domain mismatch and data sparsity, domain adaptation is very challenging. This paper proposes a novel adaptation method for DNN acoustic model using class similarity. Since the output distribution of DNN model contains the knowledge of similarity among classes, which is applicable to both source and target domain, it could be transferred from source to target model for the performance improvement. In our approach, we first compute the frame level posterior probabilities of source samples using source model. Then, for each class, probabilities of this class are used to compute a mean vector, which we refer to as mean soft labels. During adaptation, these mean soft labels are used in a regularization term to train the target model. Experiments showed that our approach outperforms fine-tuning using one-hot labels on both accent and noise adaptation task, especially when source and target domain are highly mismatched.

eess.AS↗

arXiv2020

Semibricks in extriangulated categories

Let $\mathcal{X}$ be a semibrick in an extriangulated category $\mathscr{C}$. Let $\mathcal{T}$ be the filtration subcategory generated by $\mathcal{X}$. We give a one-to-one correspondence between simple semibricks and length wide subcategories in $\mathscr{C}$. This generalizes a bijection given by Ringel in module categories, which has been generalized by Enomoto to exact categories. Moreover, we also give a one-to-one correspondence between cotorsion pairs in $\mathcal{T}$ and certain subsets of $\mathcal{X}$. Applying to the simple minded systems of an triangulated category, we recover a result given by Dugas.

math.RT↗

arXiv2020

Orthogonal Multi-view Analysis by Successive Approximations via Eigenvectors

We propose a unified framework for multi-view subspace learning to learn individual orthogonal projections for all views. The framework integrates the correlations within multiple views, supervised discriminant capacity, and distance preservation in a concise and compact way. It not only includes several existing models as special cases, but also inspires new novel models. To demonstrate its versatility to handle different learning scenarios, we showcase three new multi-view discriminant analysis models and two new multi-view multi-label classification ones under this framework. An efficient numerical method based on successive approximations via eigenvectors is presented to solve the associated optimization problem. The method is built upon an iterative Krylov subspace method which can easily scale up for high-dimensional datasets. Extensive experiments are conducted on various real-world datasets for multi-view discriminant analysis and multi-view multi-label classification. The experimental results demonstrate that the proposed models are consistently competitive to and often better than the compared methods that do not learn orthogonal projections.

cs.LG↗

arXiv2020

Privacy-preserving Transfer Learning via Secure Maximum Mean Discrepancy

The success of machine learning algorithms often relies on a large amount of high-quality data to train well-performed models. However, data is a valuable resource and are always held by different parties in reality. An effective solution to such a data isolation problem is to employ federated learning, which allows multiple parties to collaboratively train a model. In this paper, we propose a Secure version of the widely used Maximum Mean Discrepancy (SMMD) based on homomorphic encryption to enable effective knowledge transfer under the data federation setting without compromising the data privacy. The proposed SMMD is able to avoid the potential information leakage in transfer learning when aligning the source and target data distribution. As a result, both the source domain and target domain can fully utilize their data to build more scalable models. Experimental results demonstrate that our proposed SMMD is secure and effective.

cs.LG↗

arXiv2020

A Comprehensive Analysis of Information Leakage in Deep Transfer Learning

Transfer learning is widely used for transferring knowledge from a source domain to the target domain where the labeled data is scarce. Recently, deep transfer learning has achieved remarkable progress in various applications. However, the source and target datasets usually belong to two different organizations in many real-world scenarios, potential privacy issues in deep transfer learning are posed. In this study, to thoroughly analyze the potential privacy leakage in deep transfer learning, we first divide previous methods into three categories. Based on that, we demonstrate specific threats that lead to unintentional privacy leakage in each category. Additionally, we also provide some solutions to prevent these threats. To the best of our knowledge, our study is the first to provide a thorough analysis of the information leakage issues in deep transfer learning methods and provide potential solutions to the issue. Extensive experiments on two public datasets and an industry dataset are conducted to show the privacy leakage under different deep transfer learning settings and defense solution effectiveness.

cs.CL↗

arXiv2020

Quantitative estimates for homogenization of nonlinear elliptic operators in perforated domains

This paper was devoted to study the quantitative homogenization problems for nonlinear elliptic operators in perforated domains. We obtained a sharp error estimate $O(\varepsilon)$ when the problem was anchored in the reference domain $\varepsilonω$. If concerning a bounded perforated domain, one will see a bad influence from the boundary layers, which leads to the loss of the convergence rate by $O(\varepsilon^{1/2})$. Equipped with the error estimates, we developed both interior and boundary Lipschitz estimates at large-scales. As an application, we received the so-called quenched Calderón-Zygumund estimates by Shen's real arguments. To overcome some difficulties, we improved the extension theory from (\cite[Theorem 4.3]{OSY}) to $L^p$-versions with $\frac{2d}{d+1}-ε<p<\frac{2d}{d-1}+ε$ and $0<ε\ll1$. Appealing to this, we established Poincaré-Sobolev inequalities of local type on perforated domains. Some of results in the present literature are new even for related linear elliptic models.

math.AP↗

arXiv2020

Generation and manipulation of chiral terahertz waves emitted from the three-dimensional topological insulator Bi2Te3

Arbitrary manipulation of broadband terahertz waves with flexible polarization shaping at the source has great potential in expanding real applications such as imaging, information encryption, and all-optically coherent control of terahertz nonlinear phenomena. Topological insulators featuring unique spin-momentum locked surface state have already exhibited very promising prospects in terahertz emission, detection and modulation, which may lay a foundation for future on-chip topological insulator-based terahertz systems. However, polarization shaped terahertz emission with prescribed manners of arbitrarily manipulated temporal evolution of the amplitude and electric-field vector direction based on topological insulators have not yet been explored. Here we systematically investigated the terahertz radiation from topological insulator Bi2Te3 nanofilms driven by femtosecond laser pulses, and successfully realized the generation of efficient chiral terahertz waves with controllable chirality, ellipticity, and principle axis. The convenient engineering of the chiral terahertz waves was interpreted by photogalvanic effect induced photocurrent, while the linearly polarized terahertz waves originated from linear photogalvanic effect induced shift currents. We believe our works not only help further understanding femtosecond coherent control of ultrafast spin currents in light-matter interaction but also provide an effective way to generate spin-polarized terahertz waves and accelerate the proliferation of twisting the terahertz waves at the source.

physics.optics↗

arXiv2020

A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization

In this paper, we propose a deep learning approach to tackle the automatic summarization tasks by incorporating topic information into the convolutional sequence-to-sequence (ConvS2S) model and using self-critical sequence training (SCST) for optimization. Through jointly attending to topics and word-level alignment, our approach can improve coherence, diversity, and informativeness of generated summaries via a biased probability generation mechanism. On the other hand, reinforcement training, like SCST, directly optimizes the proposed model with respect to the non-differentiable metric ROUGE, which also avoids the exposure bias during inference. We carry out the experimental evaluation with state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets. The empirical results demonstrate the superiority of our proposed method in the abstractive summarization.

cs.CL↗

arXiv2020

Direct Observation of One-Dimensional Peierls-Type Charge Density Wave in Twin Boundaries of Monolayer MoTe$_2$

One-dimensional (1D) metallic mirror-twin boundaries (MTBs) in monolayer transition metal dichalcogenides (TMDCs) exhibit a periodic charge modulation and provide an ideal platform for exploring collective electron behavior in the confined system. The underlying mechanism of the charge modulation and how the electrons travel in 1D structures remain controversial. Here, for the first time, we observed atomic-scale structures of the charge distribution within one period in MTB of monolayer MoTe2 by using scanning tunneling microscopy/spectroscopy (STM/STS). The coexisting apparent periodic lattice distortions and U-shaped energy gap clearly demonstrate a Peierls-type charge density wave (CDW). Equidistant quantized energy levels with varied periodicity are further discovered outside the CDW gap along the metallic MTB. Density functional theory (DFT) calculations are in good agreement with the gapped electronic structures and reveal they originate mainly from Mo 4d orbital. Our work presents hallmark evidence of the 1D Peierls-type CDW on the metallic MTBs and offers opportunities to study the underlying physics of 1D charge modulation.

cond-mat.mtrl-sci↗

arXiv2020

InGaAs/InP single-photon detectors with 60% detection efficiency at 1550 nm

InGaAs/InP single-photon detectors (SPDs) are widely used for near-infrared photon counting in practical applications. Photon detection efficiency (PDE) is one of the most important parameters for SPD characterization, and therefore increasing PDE consistently plays a central role in both industrial development and academic research. Here we present the implementation of high-frequency gating InGaAs/InP SPD with a PDE as high as 60% at 1550 nm. On one hand, we optimize the structure design and device fabrication of InGaAs/InP single-photon avalanche diode with an additional dielectric-metal reflection layer to relatively increase the absorption efficiency of incident photons by ~ 20%. On the other hand, we develop a monolithic readout circuit of weak avalanche extraction to minimize the parasitic capacitance for the suppression of the afterpulsing effect. With 1.25 GHz sine wave gating and optimized gate amplitude and operation temperature, the SPD is characterized to reach a PDE of 60% with a dark count rate (DCR) of 340 kcps. For practical use, given 3 kcps DCR as a reference the PDE reaches ~ 40% PDE with an afterpulse probability of 5.5%, which can significantly improve the performance for the near-infrared SPD based applications.

physics.app-ph↗

arXiv2020

Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge

To better understand early brain growth patterns in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; however, one of major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. Training/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participating in iSeg-2019. We review the 8 top-ranked teams by detailing their pipelines/implementations, presenting experimental results and evaluating performance in terms of the whole brain, regions of interest, and gyral landmark curves. We also discuss their limitations and possible future directions for the multi-site issue. We hope that the multi-site dataset in iSeg-2019 and this review article will attract more researchers on the multi-site issue.

eess.IV↗

arXiv2020

Multi-view Orthonormalized Partial Least Squares: Regularizations and Deep Extensions

We establish a family of subspace-based learning method for multi-view learning using the least squares as the fundamental basis. Specifically, we investigate orthonormalized partial least squares (OPLS) and study its important properties for both multivariate regression and classification. Building on the least squares reformulation of OPLS, we propose a unified multi-view learning framework to learn a classifier over a common latent space shared by all views. The regularization technique is further leveraged to unleash the power of the proposed framework by providing three generic types of regularizers on its inherent ingredients including model parameters, decision values and latent projected points. We instantiate a set of regularizers in terms of various priors. The proposed framework with proper choices of regularizers not only can recast existing methods, but also inspire new models. To further improve the performance of the proposed framework on complex real problems, we propose to learn nonlinear transformations parameterized by deep networks. Extensive experiments are conducted to compare various methods on nine data sets with different numbers of views in terms of both feature extraction and cross-modal retrieval.

cs.LG↗

arXiv2020

Idempotent completion of extriangulated categories

Extriangulated categories were introduced by Nakaoka and Palu as a simultaneous generalization of exact categories and triangulated categories. In this paper, we show that the idempotent completion of an extriangulated category admits a natural extriangulated structure. As applications, we prove that cotorsion pairs in an extriangulated category induce cotorsion pairs in its idempotent completion under certain condition, and the idempotent completion of a recollement of extriangulated categories is still a recollement.

math.CT↗

arXiv2020

An Auto-Context Deformable Registration Network for Infant Brain MRI

Deformable image registration is fundamental to longitudinal and population analysis. Geometric alignment of the infant brain MR images is challenging, owing to rapid changes in image appearance in association with brain development. In this paper, we propose an infant-dedicated deep registration network that uses the auto-context strategy to gradually refine the deformation fields to obtain highly accurate correspondences. Instead of training multiple registration networks, our method estimates the deformation fields by invoking a single network multiple times for iterative deformation refinement. The final deformation field is obtained by the incremental composition of the deformation fields. Experimental results in comparison with state-of-the-art registration methods indicate that our method achieves higher accuracy while at the same time preserves the smoothness of the deformation fields. Our implementation is available online.

cs.CV↗

arXiv2020

A Proximal-Gradient Algorithm for Crystal Surface Evolution

As a counterpoint to recent numerical methods for crystal surface evolution, which agree well with microscopic dynamics but suffer from significant stiffness that prevents simulation on fine spatial grids, we develop a new numerical method based on the macroscopic partial differential equation, leveraging its formal structure as the gradient flow of the total variation energy, with respect to a weighted $H^{-1}$ norm. This gradient flow structure relates to several metric space gradient flows of recent interest, including 2-Wasserstein flows and their generalizations to nonlinear mobilities. We develop a novel semi-implicit time discretization of the gradient flow, inspired by the classical minimizing movements scheme (known as the JKO scheme in the 2-Wasserstein case). We then use a primal dual hybrid gradient (PDHG) method to compute each element of the semi-implicit scheme. In one dimension, we prove convergence of the PDHG method to the semi-implicit scheme, under general integrability assumptions on the mobility and its reciprocal. Finally, by taking finite difference approximations of our PDHG method, we arrive at a fully discrete numerical algorithm, with iterations that converge at a rate independent of the spatial discretization: in particular, the convergence properties do not deteriorate as we refine our spatial grid. We close with several numerical examples illustrating the properties of our method, including facet formation at local maxima, pinning at local minima, and convergence as the spatial and temporal discretizations are refined.

math.NA↗

arXiv2020

Glioma Grade Prediction Using Wavelet Scattering-Based Radiomics

Glioma grading before surgery is very critical for the prognosis prediction and treatment plan making. We present a novel wavelet scattering-based radiomic method to predict noninvasively and accurately the glioma grades. The method consists of wavelet scattering feature extraction, dimensionality reduction, and glioma grade prediction. The dimensionality reduction was achieved using partial least squares (PLS) regression and the glioma grade prediction using support vector machine (SVM), logistic regression (LR) and random forest (RF). The prediction obtained on multimodal magnetic resonance images of 285 patients with well-labeled intratumoral and peritumoral regions showed that the area under the receiver operating characteristic curve (AUC) of glioma grade prediction was increased up to 0.99 when considering both intratumoral and peritumoral features in multimodal images, which represents an increase of about 13% compared to traditional radiomics. In addition, the features extracted from peritumoral regions further increase the accuracy of glioma grading.

cs.LG↗