arXiv subjects

Chao Li

Publications and source records attributed to Chao Li.

At least 433 records · Page 24Linked to original sources

arXiv2021

Erbium-doped lithium niobate thin film waveguide amplifier with 16 dB internal net gain

Erbium-doped lithium niobate on insulator (Er:LNOI) has attracted enormous interest as it provides gain and enables integrated amplifiers and lasers on the lithium niobate on insulator (LNOI) platform. We demonstrate a highly efficient waveguide amplifier on Er:LNOI. The 2.58-cm long amplifier can achieve 27.94 dB signal enhancement, 16.0 dB internal net gain (6.20 dB/cm), -8.84 dBm saturation power, 4.59 dB/mW power conversion efficiency, and 4.49 dB noise figure at 1531.6 nm. Besides, thorough investigation on the pumping wavelength, pumping scheme, output power and noise figure have been performed to provide a comprehensive understanding on this novel waveguide amplifier. This work will benefit the development of a powerful gain platform and can pave the way for a fully integrated photonic system on LNOI platform.

physics.optics↗

arXiv2021

A Channel-Aware Routing Protocol With Nearest Neighbor Regression For Underwater Sensor Networks

The underwater acoustic channel is one of the most challenging communication channels. Due to periodical tidal and daily climatic variation, underwater noise is periodically fluctuating, which result in the periodical changing of acoustic channel quality in long-term. Also, time-variant channel quality leads to routing failure. Routing protocols with acoustic channel estimation, namely underwater channel-aware routing protocols are recently proposed to maintain the routing performance. However, channel estimation algorithms for these routing protocols are mostly linear and rarely consider periodicity of acoustic channels. In this paper, we introduce acoustic channel estimation based on nearest neighbor regression for underwater acoustic networks. We extend nearest neighbor regression for SNR (Signal-to-Noise Ratio) time series prediction, providing an outstanding prediction accuracy for intricately periodical and fluctuating received SNR time series. Moreover, we propose a quick search algorithm and use statistical storage compression to optimize the time and space complexity of the algorithm. In contrast with linear methods, this algorithm significantly improves channel prediction accuracy (over three times at most) on both simulation and sea trial data sets. With this channel estimation method, we then propose a Depth-Based Channel-Aware Routing protocol (DBCAR). Taking advantage of depth-greedy forwarding and channel-aware reliable communication, DBCAR has an outstanding network performance on packet delivery ratio, average energy consumption and average transmission delay which is validated through extensive simulations.

cs.NI↗

arXiv2021

A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Acceleration

Distributed training is an effective way to accelerate the training process of large-scale deep learning models. However, the parameter exchange and synchronization of distributed stochastic gradient descent introduce a large amount of communication overhead. Gradient compression is an effective method to reduce communication overhead. In synchronization SGD compression methods, many Top-k sparsification based gradient compression methods have been proposed to reduce the communication. However, the centralized method based on the parameter servers has the single point of failure problem and limited scalability, while the decentralized method with global parameter exchanging may reduce the convergence rate of training. In contrast with Top-$k$ based methods, we proposed a gradient compression method with globe gradient vector sketching, which uses the Count-Sketch structure to store the gradients to reduce the loss of the accuracy in the training process, named global-sketching SGD (gs-SGD). The gs-SGD has better convergence efficiency on deep learning models and a communication complexity of O($\log d*\log P$), where $d$ is the number of model parameters and P is the number of workers. We conducted experiments on GPU clusters to verify that our method has better convergence efficiency than global Top-$k$ and Sketching-based methods. In addition, gs-SGD achieves 1.3-3.1x higher throughput compared with gTop-$k$, and 1.1-1.2x higher throughput compared with original Sketched-SGD.

cs.DC↗

arXiv2021

End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23\% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-$k$ similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and $k$ items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a locality-sensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1\% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.

cs.IR↗

arXiv2021

Semi-supervised classification of radiology images with NoTeacher: A Teacher that is not Mean

Deep learning models achieve strong performance for radiology image classification, but their practical application is bottlenecked by the need for large labeled training datasets. Semi-supervised learning (SSL) approaches leverage small labeled datasets alongside larger unlabeled datasets and offer potential for reducing labeling cost. In this work, we introduce NoTeacher, a novel consistency-based SSL framework which incorporates probabilistic graphical models. Unlike Mean Teacher which maintains a teacher network updated via a temporal ensemble, NoTeacher employs two independent networks, thereby eliminating the need for a teacher network. We demonstrate how NoTeacher can be customized to handle a range of challenges in radiology image classification. Specifically, we describe adaptations for scenarios with 2D and 3D inputs, uni and multi-label classification, and class distribution mismatch between labeled and unlabeled portions of the training data. In realistic empirical evaluations on three public benchmark datasets spanning the workhorse modalities of radiology (X-Ray, CT, MRI), we show that NoTeacher achieves over 90-95% of the fully supervised AUROC with less than 5-15% labeling budget. Further, NoTeacher outperforms established SSL methods with minimal hyperparameter tuning, and has implications as a principled and practical option for semisupervised learning in radiology applications.

cs.CV↗

arXiv2021

BrainNetGAN: Data augmentation of brain connectivity using generative adversarial network for dementia classification

Alzheimer's disease (AD) is the most common age-related dementia. It remains a challenge to identify the individuals at risk of dementia for precise management. Brain MRI offers a noninvasive biomarker to detect brain aging. Previous evidence shows that the brain structural change detected by diffusion MRI is associated with dementia. Mounting studies has conceptualised the brain as a complex network, which has shown the utility of this approach in characterising various neurological and psychiatric disorders. Therefore, the structural connectivity shows promise in dementia classification. The proposed BrainNetGAN is a generative adversarial network variant to augment the brain structural connectivity matrices for binary dementia classification tasks. Structural connectivity matrices between separated brain regions are constructed using tractography on diffusion MRI data. The BrainNetGAN model is trained to generate fake brain connectivity matrices, which are expected to reflect latent distribution of the real brain network data. Finally, a convolutional neural network classifier is proposed for binary dementia classification. Numerical results show that the binary classification performance in the testing set was improved using the BrainNetGAN augmented dataset. The proposed methodology allows quick synthesis of an arbitrary number of augmented connectivity matrices and can be easily transferred to similar classification tasks.

cs.LG↗

arXiv2021

SilentDelivery: Practical Timed-delivery of Private Information using Smart Contracts

This paper proposes SilentDelivery, a secure, scalable and cost-efficient protocol for implementing timed information delivery service in a decentralized blockchain network. SilentDelivery employs a novel combination of threshold secret sharing and decentralized smart contracts. The protocol maintains shares of the decryption key of the private information of an information sender using a group of mailmen recruited in a blockchain network before the specified future time-frame and restores the information to the information recipient at the required time-frame. To tackle the key challenges that limit the security and scalability of the protocol, SilentDelivery incorporates two novel countermeasure strategies. The first strategy, namely silent recruitment, enables a mailman to get recruited by a sender silently without the knowledge of any third party. The second strategy, namely dual-mode execution, makes the protocol run in a lightweight mode by default, where the cost of running smart contracts is significantly reduced. We rigorously analyze the security of SilentDelivery and implement the protocol over the Ethereum official test network. The results demonstrate that SilentDelivery is more secure and scalable compared to the state of the art and reduces the cost of running smart contracts by 85%.

cs.CR↗

arXiv2021

ZIPPER: Exploiting Tile- and Operator-level Parallelism for General and Scalable Graph Neural Network Acceleration

Graph neural networks (GNNs) start to gain momentum after showing significant performance improvement in a variety of domains including molecular science, recommendation, and transportation. Turning such performance improvement of GNNs into practical applications relies on effective and efficient execution, especially for inference. However, neither CPU nor GPU can meet these needs if considering both performance and energy efficiency. That's because accelerating GNNs is challenging due to their excessive memory usage and arbitrary interleaving of diverse operations. Besides, the semantics gap between the high-level GNN programming model and efficient hardware makes it difficult in accelerating general-domain GNNs. To address the challenge, we propose Zipper, an efficient yet general acceleration system for GNNs. The keys to Zipper include a graph-native intermediate representation (IR) and the associated compiler. By capturing GNN primitive operations and representing with GNN IR, Zipper is able to fit GNN semantics into hardware structure for efficient execution. The IR also enables GNN-specific optimizations including sparse graph tiling and redundant operation elimination. We further present an hardware architecture design consisting of dedicated blocks for different primitive operations, along with a run-time scheduler to map a IR program to the hardware blocks. Our evaluation shows that Zipper achieves 93.6x speedup and 147x energy reduction over Intel Xeon CPU, and 1.56x speedup and 4.85x energy reduction over NVIDIA V100 GPU on averages.

cs.AR↗

arXiv2021

Expectation-Maximization Regularized Deep Learning for Weakly Supervised Tumor Segmentation for Glioblastoma

We present an Expectation-Maximization (EM) Regularized Deep Learning (EMReDL) model for weakly supervised tumor segmentation. The proposed framework is tailored to glioblastoma, a type of malignant tumor characterized by its diffuse infiltration into the surrounding brain tissue, which poses significant challenge to treatment target and tumor burden estimation using conventional structural MRI. Although physiological MRI provides more specific information regarding tumor infiltration, the relatively low resolution hinders a precise full annotation. This has motivated us to develop a weakly supervised deep learning solution that exploits the partial labelled tumor regions. EMReDL contains two components: a physiological prior prediction model and EM-regularized segmentation model. The physiological prior prediction model exploits the physiological MRI by training a classifier to generate a physiological prior map. This map is passed to the segmentation model for regularization using the EM algorithm. We evaluated the model on a glioblastoma dataset with the pre-operative multiparametric and recurrence MRI available. EMReDL showed to effectively segment the infiltrated tumor from the partially labelled region of potential infiltration. The segmented core tumor and infiltrated tumor demonstrated high consistency with the tumor burden labelled by experts. The performance comparisons showed that EMReDL achieved higher accuracy than published state-of-the-art models. On MR spectroscopy, the segmented region displayed more aggressive features than other partial labelled region. The proposed model can be generalized to other segmentation tasks that rely on partial labels, with the CNN architecture flexible in the framework.

eess.IV↗

arXiv2021

Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy

Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study a crucial question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. Our work investigates how many times the PS should query clients and how many times each client should reply the PS. We investigate two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.

cs.LG↗

arXiv2021

On the arithmetic Siegel--Weil formula for GSpin Shimura varieties

We formulate and prove a local arithmetic Siegel--Weil formula for GSpin Rapoport--Zink spaces, which is a precise identity between the arithmetic intersection numbers of special cycles on GSpin Rapoport--Zink spaces and the derivatives of local representation densities of quadratic forms. As a first application, we prove a semi-global arithmetic Siegel--Weil formula as conjectured by Kudla, which relates the arithmetic intersection numbers of special cycles on GSpin Shimura varieties at a place of good reduction and the central derivatives of nonsingular Fourier coefficients of incoherent Siegel Eisenstein series.

math.NT↗

arXiv2021

A single-frequency single-resonator laser on erbium-doped lithium niobate on insulator

Erbium-doped lithium niobate on insulator (Er:LNOI) is a promising platform for photonic integrated circuits as it adds gain to the LNOI system and enables on-chip lasers and amplifiers. A challenge for Er:LNOI laser is to increase its output power while maintaining single-frequency and single (-transverse)-mode operation. In this work, we demonstrate that single-frequency and single-mode operation can be achieved even in a single multi-mode Er:LNOI microring by introducing mode-dependent loss and gain competition. In a single microring with a free spectral range of 192 GHz, we have achieved single-mode lasing with an output power of 2.1 microwatt, a side-mode suppression of 35.5 dB, and a linewidth of 1.27 MHz.

physics.optics↗

arXiv2021

A blind zone-suppressed hybrid beam steering for solid-state Lidar

We demonstrate a blind zone-suppressed and flash-emitting solid-state Lidar based on lens-assisted beam steering (LABS) technology. As a proof-of-concept demonstration, with a design of subwavelength-gap one-dimensional (1D) long-emitter array and multi-wavelength flash beam emitting, the device was measured to have 5%-blind zone suppression, 0.06°/point-deflection step and 4.2 microsecond-scanning speed. In time-of-flight (TOF) ranging experiments, Lidar systems have field of view of 11.3°* 8.1° (normal device) or 0.9°*8.1° (blind-zone suppressed device), far-field number of resolved points of 192 and a detection distance of 10 m. This work demonstrates the possibility that a new integrated beam-steering technology can be implemented in a Lidar without sacrificing other performance.

physics.app-ph↗

arXiv2021

A new heat source model for selective laser melting simulations based on energy distribution on the powder layer and the surface of substrate

In order to predict the more accurate shape information of the melt pool in Selective Laser Melting (SLM), a new finite element temperature field simulations model is proposed. The simulations use a new heat source model that takes into account the influence of the powder layout, the surface of the substrate and the changes in the thickness of the powder layer after fusion on the energy distribution. In order to construct this new heat source model, firstly an improved optimization method based on the gradient descent and the univariate search technique is proposed to simulate the powder layout, and then the laser beam propagation between the powder and the surface of the substrate is tracked and recorded to obtain the energy distribution. Finally, according to the distribution of laser energy between the powder layer and the surface of the substrate, the heat source model is divided into two parts: one is the surface of substrate heat source model being the Gaussian distribution, the other one is the powder layer heat source model-satisfying the Gaussian distribution on the horizontal plane, changes in the depth direction according to the functional relationship obtained by the fitting. In addition, the thickness change of the powder layer after fusion is analyzed, and is taken into account in the heat source model. The powder simulation results are compared with the powder scattering experiment results to verify the effectiveness of the powder model. Comparing the temperature field simulation with the experiment, the results show that the predicted molten pool width relative error is 6.4%, and the connect width error is 9.6%, which has better accuracy and verifies the validity of the temperature field simulation model.

physics.app-ph↗

arXiv2021

Self-Supervised Nonlinear Transform-Based Tensor Nuclear Norm for Multi-Dimensional Image Recovery

In this paper, we study multi-dimensional image recovery. Recently, transform-based tensor nuclear norm minimization methods are considered to capture low-rank tensor structures to recover third-order tensors in multi-dimensional image processing applications. The main characteristic of such methods is to perform the linear transform along the third mode of third-order tensors, and then compute tensor nuclear norm minimization on the transformed tensor so that the underlying low-rank tensors can be recovered. The main aim of this paper is to propose a nonlinear multilayer neural network to learn a nonlinear transform via the observed tensor data under self-supervision. The proposed network makes use of low-rank representation of transformed tensors and data-fitting between the observed tensor and the reconstructed tensor to construct the nonlinear transformation. Extensive experimental results on tensor completion, background subtraction, robust tensor completion, and snapshot compressive imaging are presented to demonstrate that the performance of the proposed method is better than that of state-of-the-art methods.

eess.IV↗

arXiv2021

Regularity of free boundary minimal surfaces in locally polyhedral domains

We prove an Allard-type regularity theorem for free-boundary minimal surfaces in Lipschitz domains locally modelled on convex polyhedra. We show that if such a minimal surface is sufficiently close to an appropriate free-boundary plane, then the surface is $C^{1,α}$ graphical over this plane. We apply our theorem to prove partial regularity results for free-boundary minimizing hypersurfaces, and relative isoperimetric regions.

math.DG↗

arXiv2021

Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking

Chinese Spell Checking (CSC) aims to detect and correct erroneous characters for user-generated text in the Chinese language. Most of the Chinese spelling errors are misused semantically, phonetically or graphically similar characters. Previous attempts noticed this phenomenon and try to use the similarity for this task. However, these methods use either heuristics or handcrafted confusion sets to predict the correct character. In this paper, we propose a Chinese spell checker called ReaLiSe, by directly leveraging the multimodal information of the Chinese characters. The ReaLiSe model tackles the CSC task by (1) capturing the semantic, phonetic and graphic information of the input characters, and (2) selectively mixing the information in these modalities to predict the correct output. Experiments on the SIGHAN benchmarks show that the proposed model outperforms strong baselines by a large margin.

cs.CL↗

arXiv2021

Improving BERT with Syntax-aware Local Attention

Pre-trained Transformer-based neural language models, such as BERT, have achieved remarkable results on varieties of NLP tasks. Recent works have shown that attention-based models can benefit from more focused attention over local regions. Most of them restrict the attention scope within a linear span, or confine to certain tasks such as machine translation and question answering. In this paper, we propose a syntax-aware local attention, where the attention scopes are restrained based on the distances in the syntactic structure. The proposed syntax-aware local attention can be integrated with pretrained language models, such as BERT, to render the model to focus on syntactically relevant words. We conduct experiments on various single-sentence benchmarks, including sentence classification and sequence labeling tasks. Experimental results show consistent gains over BERT on all benchmark datasets. The extensive studies verify that our model achieves better performance owing to more focused attention over syntactically relevant words.

cs.CL↗