arXiv subjects

Zhizhong Li

Publications and source records attributed to Zhizhong Li.

25 records · Page 2Linked to original sources

arXiv2019

Biased Estimates of Advantages over Path Ensembles

The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or against risks. On top of this formulation, we systematically study the impacts of different methods for estimating advantages. Our findings reveal that biased estimates, when chosen appropriately, can result in significant benefits. In particular, for the environments with sparse rewards, optimistic estimates would lead to more efficient exploration of the policy space; while for those where individual actions can have critical impacts, conservative estimates are preferable. On various benchmarks, including MuJoCo continuous control, Terrain locomotion, Atari games, and sparse-reward environments, the proposed biased estimation schemes consistently demonstrate improvement over mainstream methods, not only accelerating the learning process but also obtaining substantial performance gains.

cs.LG↗

arXiv2019

A hierarchical Lyapunov-based cascade adaptive control scheme for lower-limb exoskeleton

This paper proposes a hierarchical Lyapunov-based adaptive cascade control scheme for a lower-limb exoskeleton with control saturation. The proposed approach is composed by two control levels with cascade structure. At the higher layer of the structure, a Lyapunov-based back-stepping regulator including adaptive estimation of uncertain parameters and friction force is designed for the leg dynamics, to minimize the deviation of the joint position and its reference value. At the lower layer, a Lyapunov-based neural network adaptive controller is in charge of computing control action for the hydraulic servo system, to follow the force reference computed at the high level, also to compensate for model uncertainty, nonlinearity, and control saturation. The proposed approach shows to be capable in minimizing the interaction torque between machine and human, and suitable for possible imprecise models. The robustness of the closed-loop system is discussed under input constraint. Simulation experiments are reported, which shows that the proposed scheme is effective in imposing smaller interaction torque with respect to PD controller, and in control of models with uncertainty and nonlinearity.

eess.SY↗

arXiv2019

Deep Nearest Class Mean Model for Incremental Odor Classification

In recent years, more machine learning algorithms have been applied to odor classification. These odor classification algorithms usually assume that the training datasets are static. However, for some odor recognition tasks, new odor classes continually emerge. That is, the odor datasets are dynamically growing while both training samples and number of classes are increasing over time. Motivated by this concern, this paper proposes a Deep Nearest Class Mean (DNCM) model based on the deep learning framework and nearest class mean method. The proposed model not only leverages deep neural network to extract deep features, but is also able to dynamically integrate new classes over time. In our experiments, the DNCM model was initially trained with 10 classes, then 25 new classes are integrated. Experiment results demonstrate that the proposed model is very efficient for incremental odor classification, especially for new classes with only a small number of training examples.

cs.LG↗

arXiv2018

Complete 3D Scene Parsing from an RGBD Image

One major goal of vision is to infer physical models of objects, surfaces, and their layout from sensors. In this paper, we aim to interpret indoor scenes from one RGBD image. Our representation encodes the layout of orthogonal walls and the extent of objects, modeled with CAD-like 3D shapes. We parse both the visible and occluded portions of the scene and all observable objects, producing a complete 3D parse. Such a scene interpretation is useful for robotics and visual reasoning, but difficult to produce due to the well-known challenge of segmentation, the high degree of occlusion, and the diversity of objects in indoor scenes. We take a data-driven approach, generating sets of potential object regions, matching to regions in training images, and transferring and aligning associated 3D models while encouraging fit to observations and spatial consistency. We use support inference to aid interpretation and propose a retrieval scheme that uses convolutional neural networks (CNNs) to classify regions and retrieve objects with similar shapes. We demonstrate the performance of our method on our newly annotated NYUd v2 dataset with detailed 3D shapes.

cs.CV↗

arXiv2017

Integrating Specialized Classifiers Based on Continuous Time Markov Chain

Specialized classifiers, namely those dedicated to a subset of classes, are often adopted in real-world recognition systems. However, integrating such classifiers is nontrivial. Existing methods, e.g. weighted average, usually implicitly assume that all constituents of an ensemble cover the same set of classes. Such methods can produce misleading predictions when used to combine specialized classifiers. This work explores a novel approach. Instead of combining predictions from individual classifiers directly, it first decomposes the predictions into sets of pairwise preferences, treating them as transition channels between classes, and thereon constructs a continuous-time Markov chain, and use the equilibrium distribution of this chain as the final prediction. This way allows us to form a coherent picture over all specialized predictions. On large public datasets, the proposed method obtains considerable improvement compared to mainstream ensemble methods, especially when the classifier coverage is highly unbalanced.

cs.LG↗

arXiv2017

PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition. In our study, however, we observed difficulties along both directions. On one hand, the pursuit for very deep networks is met with a diminishing return and increased training difficulty; on the other hand, widening a network would result in a quadratic growth in both computational cost and memory demand. These difficulties motivate us to explore structural diversity in designing deep networks, a new dimension beyond just depth and width. Specifically, we present a new family of modules, namely the PolyInception, which can be flexibly inserted in isolation or in a composition as replacements of different parts of a network. Choosing PolyInception modules with the guidance of architectural efficiency can improve the expressive power while preserving comparable computational cost. The Very Deep PolyNet, designed following this direction, demonstrates substantial improvements over the state-of-the-art on the ILSVRC 2012 benchmark. Compared to Inception-ResNet-v2, it reduces the top-5 validation error on single crops from 4.9% to 4.25%, and that on multi-crops from 3.7% to 3.45%.

cs.CV↗

arXiv2017

Learning without Forgetting

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

cs.CV↗