arXiv subjects

Yi Zhou

Publications and source records attributed to Yi Zhou.

At least 451 records · Page 25Linked to original sources

arXiv2020

The Complexity of the Partition Coloring Problem

Given a simple undirected graph $G=(V,E)$ and a partition of the vertex set $V$ into $p$ parts, the \textsc{Partition Coloring Problem} asks if we can select one vertex from each part of the partition such that the chromatic number of the subgraph induced on the $p$ selected vertices is bounded by $k$. PCP is a generalized problem of the classical \textsc{Vertex Coloring Problem} and has applications in many areas, such as scheduling and encoding etc. In this paper, we show the complexity status of the \textsc{Partition Coloring Problem} with three parameters: the number of colors, the number of parts of the partition, and the maximum size of each part of the partition. Furthermore, we give a new exact algorithm for this problem.

cs.DS↗

arXiv2020

History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms

Variance-reduced algorithms, although achieve great theoretical performance, can run slowly in practice due to the periodic gradient estimation with a large batch of data. Batch-size adaptation thus arises as a promising approach to accelerate such algorithms. However, existing schemes either apply prescribed batch-size adaption rule or exploit the information along optimization path via additional backtracking and condition verification steps. In this paper, we propose a novel scheme, which eliminates backtracking line search but still exploits the information along optimization path by adapting the batch size via history stochastic gradients. We further theoretically show that such a scheme substantially reduces the overall complexity for popular variance-reduced algorithms SVRG and SARAH/SPIDER for both conventional nonconvex optimization and reinforcement learning problems. To this end, we develop a new convergence analysis framework to handle the dependence of the batch size on history stochastic gradients. Extensive experiments validate the effectiveness of the proposed batch-size adaptation scheme.

math.OC↗

arXiv2020

IBM Federated Learning: an Enterprise Framework White Paper V0.1

Federated Learning (FL) is an approach to conduct machine learning without centralizing training data in a single place, for reasons of privacy, confidentiality or data volume. However, solving federated machine learning problems raises issues above and beyond those of centralized machine learning. These issues include setting up communication infrastructure between parties, coordinating the learning process, integrating party results, understanding the characteristics of the training data sets of different participating parties, handling data heterogeneity, and operating with the absence of a verification data set. IBM Federated Learning provides infrastructure and coordination for federated learning. Data scientists can design and run federated learning jobs based on existing, centralized machine learning models and can provide high-level instructions on how to run the federation. The framework applies to both Deep Neural Networks as well as ``traditional'' approaches for the most common machine learning libraries. {\proj} enables data scientists to expand their scope from centralized to federated machine learning, minimizing the learning curve at the outset while also providing the flexibility to deploy to different compute environments and design custom fusion algorithms.

cs.LG↗

arXiv2020

Motion-Attentive Transition for Zero-Shot Video Object Segmentation

In this paper, we present a novel Motion-Attentive Transition Network (MATNet) for zero-shot video object segmentation, which provides a new way of leveraging motion information to reinforce spatio-temporal object representation. An asymmetric attention block, called Motion-Attentive Transition (MAT), is designed within a two-stream encoder, which transforms appearance features into motion-attentive representations at each convolutional stage. In this way, the encoder becomes deeply interleaved, allowing for closely hierarchical interactions between object motion and appearance. This is superior to the typical two-stream architecture, which treats motion and appearance separately in each stream and often suffers from overfitting to appearance information. Additionally, a bridge network is proposed to obtain a compact, discriminative and scale-sensitive representation for multi-level encoder features, which is further fed into a decoder to achieve segmentation results. Extensive experiments on three challenging public benchmarks (i.e. DAVIS-16, FBMS and Youtube-Objects) show that our model achieves compelling performance against the state-of-the-arts.

cs.CV↗

arXiv2020

Understanding the Impact of Model Incoherence on Convergence of Incremental SGD with Random Reshuffle

Although SGD with random reshuffle has been widely-used in machine learning applications, there is a limited understanding of how model characteristics affect the convergence of the algorithm. In this work, we introduce model incoherence to characterize the diversity of model characteristics and study its impact on convergence of SGD with random reshuffle under weak strong convexity. Specifically, minimizer incoherence measures the discrepancy between the global minimizers of a sample loss and those of the total loss and affects the convergence error of SGD with random reshuffle. In particular, we show that the variable sequence generated by SGD with random reshuffle converges to a certain global minimizer of the total loss under full minimizer coherence. The other curvature incoherence measures the quality of condition numbers of the sample losses and determines the convergence rate of SGD. With model incoherence, our results show that SGD has a faster convergence rate and smaller convergence error under random reshuffle than those under random sampling, and hence provide justifications to the superior practical performance of SGD with random reshuffle.

math.OC↗

arXiv2020

Chinese Named Entity Recognition Augmented with Lexicon Memory

Inspired by a concept of content-addressable retrieval from cognitive science, we propose a novel fragment-based model augmented with a lexicon-based memory for Chinese NER, in which both the character-level and word-level features are combined to generate better feature representations for possible name candidates. It is observed that locating the boundary information of entity names is useful in order to classify them into pre-defined categories. Position-dependent features, including prefix and suffix are introduced for NER in the form of distributed representation. The lexicon-based memory is used to help generate such position-dependent features and deal with the problem of out-of-vocabulary words. Experimental results showed that the proposed model, called LEMON, achieved state-of-the-art on four datasets.

cs.CL↗

arXiv2020

Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble

Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples. In this paper, we propose Dirichlet Neighborhood Ensemble (DNE), a randomized smoothing method for training a robust model to defense substitution-based attacks. During training, DNE forms virtual sentences by sampling embedding vectors for each word in an input sentence from a convex hull spanned by the word and its synonyms, and it augments them with the training data. In such a way, the model is robust to adversarial attacks while maintaining the performance on the original clean data. DNE is agnostic to the network architectures and scales to large models for NLP applications. We demonstrate through extensive experimentation that our method consistently outperforms recently proposed defense methods by a significant margin across different network architectures and multiple data sets.

cs.CL↗

arXiv2020

On the Continuity of Rotation Representations in Neural Networks

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

cs.LG↗

arXiv2020

Recognizing Chinese Judicial Named Entity using BiLSTM-CRF

Named entity recognition (NER) plays an essential role in natural language processing systems. Judicial NER is a fundamental component of judicial information retrieval, entity relation extraction, and knowledge map building. However, Chinese judicial NER remains to be more challenging due to the characteristics of Chinese and high accuracy requirements in the judicial filed. Thus, in this paper, we propose a deep learning-based method named BiLSTM-CRF which consists of bi-directional long short-term memory (BiLSTM) and conditional random fields (CRF). For further accuracy promotion, we propose to use Adaptive moment estimation (Adam) for optimization of the model. To validate our method, we perform experiments on judgment documents including commutation, parole and temporary service outside prison, which is acquired from China Judgments Online. Experimental results achieve the accuracy of 0.876, recall of 0.856 and F1 score of 0.855, which suggests the superiority of the proposed BiLSTM-CRF with Adam optimizer.

cs.CL↗

arXiv2020

Generative Tweening: Long-term Inbetweening of 3D Human Motions

The ability to generate complex and realistic human body animations at scale, while following specific artistic constraints, has been a fundamental goal for the game and animation industry for decades. Popular techniques include key-framing, physics-based simulation, and database methods via motion graphs. Recently, motion generators based on deep learning have been introduced. Although these learning models can automatically generate highly intricate stylized motions of arbitrary length, they still lack user control. To this end, we introduce the problem of long-term inbetweening, which involves automatically synthesizing complex motions over a long time interval given very sparse keyframes by users. We identify a number of challenges related to this problem, including maintaining biomechanical and keyframe constraints, preserving natural motions, and designing the entire motion sequence holistically while considering all constraints. We introduce a biomechanically constrained generative adversarial network that performs long-term inbetweening of human motions, conditioned on keyframe constraints. This network uses a novel two-stage approach where it first predicts local motion in the form of joint angles, and then predicts global motion, i.e. the global path that the character follows. Since there are typically a number of possible motions that could satisfy the given user constraints, we also enable our network to generate a variety of outputs with a scheme that we call Motion DNA. This approach allows the user to manipulate and influence the output content by feeding seed motions (DNA) to the network. Trained with 79 classes of captured motion data, our network performs robustly on a variety of highly complex motion styles.

cs.CV↗

arXiv2020

Non-invasive detection of nanoscale structural changes in cornea associated with cross-linking treatment

Corneal cross-linking (CXL) using UVA irradiation with a riboflavin photosensitizer has grown from an interesting concept to a practical clinical treatment for corneal ectatic diseases globally, such as keratoconus. To characterize the corneal structural changes, existing methods such as X-ray microscopy, transmission electron microscopy (TEM), histology and optical coherence tomography have been used. However, these methods have various drawbacks such as invasive detection, the impossibility for in vivo measurement, or limited resolution and sensitivity to structural alterations. Here, we report the application of over-sampling nano-sensitive optical coherence tomography (nsOCT) method for probing the corneal structural alterations. The results indicate that the spatial period increases slightly after 30 minutes riboflavin instillation but decreases significantly after 30 min UVA irradiation following the Dresden protocol. The proposed non-invasive method can be implemented using existing OCT system, without any additional components, for detecting nanoscale changes with the potential to assist diagnostic assessment during CXL treatment, and possibly to be a real-time monitoring tool in clinics.

physics.med-ph↗

arXiv2020

Nanoscale structure detection and monitoring of tumour growth with optical coherence tomography

Approximately 90% of cancers have their origins in epithelial tissues and this leads to epithelial thickening, but the ultrastructural changes and underlying architecture is less well known. Depth resolved label free visualization of nanoscale tissue morphology is required to reveal the extent and distribution of ultrastructural changes in underlying tissue, but is difficult to achieve with existing imaging modalities. We developed a nanosensitive optical coherence tomography (nsOCT) approach to provide such imaging based on dominant axial structure with a few nanometre detection accuracy. nsOCT maps the distribution of axial structural sizes an order of magnitude smaller than the axial resolution of the system. We validated nsOCT methodology by detecting synthetic axial structure via numerical simulations. Subsequently, we validated the nsOCT technique experimentally by detecting known structures from a commercially fabricated sample. nsOCT reveals scaling with different depth of dominant submicron structural changes associated with carcinoma which may inform the origins of the disease, its progression and improve diagnosis.

physics.med-ph↗

arXiv2020

Nanosensitive optical coherence tomography to assess wound healing within the cornea

Optical Coherence Tomography (OCT) is a non-invasive depth resolved optical imaging modality, that enables high resolution, cross-sectional imaging in biological tissues and materials at clinically relevant depths. Though OCT offers high resolution imaging, the best ultra-high-resolution OCT systems are limited to imaging structural changes with a resolution of one micron on a single B-scan within very limited depth. Nanosensitive OCT (nsOCT) is a recently developed technique that is capable of providing enhanced sensitivity of OCT to structural changes. Improving the sensitivity of OCT to detect structural changes at the nanoscale level, to a depth typical for conventional OCT, could potentially improve the diagnostic capability of OCT in medical applications. In this paper, we demonstrate the capability of nsOCT to detect structural changes deep in the rat cornea following superficial corneal injury.

physics.med-ph↗

arXiv2020

Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images

Coronavirus Disease 2019 (COVID-19) spread globally in early 2020, causing the world to face an existential health crisis. Automated detection of lung infections from computed tomography (CT) images offers a great potential to augment the traditional healthcare strategy for tackling COVID-19. However, segmenting infected regions from CT slices faces several challenges, including high variation in infection characteristics, and low intensity contrast between infections and normal tissues. Further, collecting a large amount of data is impractical within a short time period, inhibiting the training of a deep model. To address these challenges, a novel COVID-19 Lung Infection Segmentation Deep Network (Inf-Net) is proposed to automatically identify infected regions from chest CT slices. In our Inf-Net, a parallel partial decoder is used to aggregate the high-level features and generate a global map. Then, the implicit reverse attention and explicit edge-attention are utilized to model the boundaries and enhance the representations. Moreover, to alleviate the shortage of labeled data, we present a semi-supervised segmentation framework based on a randomly selected propagation strategy, which only requires a few labeled images and leverages primarily unlabeled data. Our semi-supervised framework can improve the learning ability and achieve a higher performance. Extensive experiments on our COVID-SemiSeg and real CT volumes demonstrate that the proposed Inf-Net outperforms most cutting-edge segmentation models and advances the state-of-the-art performance.

eess.IV↗

arXiv2020

SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms

SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization. However, SPIDER uses an accuracy-dependent stepsize that slows down the convergence in practice, and cannot handle objective functions that involve nonsmooth regularizers. In this paper, we propose SpiderBoost as an improved scheme, which allows to use a much larger constant-level stepsize while maintaining the same near-optimal oracle complexity, and can be extended with proximal mapping to handle composite optimization (which is nonsmooth and nonconvex) with provable convergence guarantee. In particular, we show that proximal SpiderBoost achieves an oracle complexity of $\mathcal{O}(\min\{n^{1/2}ε^{-2},ε^{-3}\})$ in composite nonconvex optimization, improving the state-of-the-art result by a factor of $\mathcal{O}(\min\{n^{1/6},ε^{-1/3}\})$. We further develop a novel momentum scheme to accelerate SpiderBoost for composite optimization, which achieves the near-optimal oracle complexity in theory and substantial improvement in experiments.

math.OC↗

arXiv2020

Momentum with Variance Reduction for Nonconvex Composition Optimization

Composition optimization is widely-applied in nonconvex machine learning. Various advanced stochastic algorithms that adopt momentum and variance reduction techniques have been developed for composition optimization. However, these algorithms do not fully exploit both techniques to accelerate the convergence and are lack of convergence guarantee in nonconvex optimization. This paper complements the existing literature by developing various momentum schemes with SPIDER-based variance reduction for non-convex composition optimization. In particular, our momentum design requires less number of proximal mapping evaluations per-iteration than that required by the existing Katyusha momentum. Furthermore, our algorithm achieves near-optimal sample complexity results in both non-convex finite-sum and online composition optimization and achieves a linear convergence rate under the gradient dominant condition. Numerical experiments demonstrate that our algorithm converges significantly faster than existing algorithms in nonconvex composition optimization.

cs.LG↗

arXiv2020

Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization

Various types of parameter restart schemes have been proposed for accelerated gradient algorithms to facilitate their practical convergence in convex optimization. However, the convergence properties of accelerated gradient algorithms under parameter restart remain obscure in nonconvex optimization. In this paper, we propose a novel accelerated proximal gradient algorithm with parameter restart (named APG-restart) for solving nonconvex and nonsmooth problems. Our APG-restart is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization. Numerical experiments demonstrate the effectiveness of our proposed algorithm.

math.OC↗

arXiv2020

Efficient tensor network representation for Gutzwiller projected states of paired fermions

Recent work by Wu {\em et al.} [arXiv:1910.11011] proposed a numerical method, so-called matrix product operator-matrix product state (MPO-MPS) method, by which several types of quantum many-body wave functions, in particular, the projected Fermi sea state, can be efficiently represented as a tensor network. In this paper, we generalize the MPO-MPS method to study Gutzwiller projected paired states of fermions, where the maximally localized Wannier orbitals for Bogoliubov quasiparticles/quasiholes have been adapted to improve the computational performance. The study of $SO(3)$-symmetric spin-1 chains reveals that this new method has better performance than variational Monte Carlo for gapped states and similar performance for gapless states. Moreover, we demonstrate that dynamic correlation functions can be easily evaluated by this method cooperating with other MPS-based accurate approaches, such as the Chebyshev MPS method.

cond-mat.str-el↗