arXiv subjects

Zhi Tian

Publications and source records attributed to Zhi Tian.

At least 73 records · Page 4Linked to original sources

arXiv2020

Conditional Convolutions for Instance Segmentation

We propose a simple yet effective instance segmentation framework, termed CondInst (conditional convolutions for instance segmentation). Top-performing instance segmentation methods such as Mask R-CNN rely on ROI operations (typically ROIPool or ROIAlign) to obtain the final instance masks. In contrast, we propose to solve instance segmentation from a new perspective. Instead of using instance-wise ROIs as inputs to a network of fixed weights, we employ dynamic instance-aware networks, conditioned on instances. CondInst enjoys two advantages: 1) Instance segmentation is solved by a fully convolutional network, eliminating the need for ROI cropping and feature alignment. 2) Due to the much improved capacity of dynamically-generated conditional convolutions, the mask head can be very compact (e.g., 3 conv. layers, each having only 8 channels), leading to significantly faster inference. We demonstrate a simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed. On the COCO dataset, we outperform a few recent methods including well-tuned Mask RCNN baselines, without longer training schedules needed. Code is available: https://github.com/aim-uofa/adet

cs.CV↗

arXiv2020

Mask Encoding for Single Shot Instance Segmentation

To date, instance segmentation is dominated by twostage methods, as pioneered by Mask R-CNN. In contrast, one-stage alternatives cannot compete with Mask R-CNN in mask AP, mainly due to the difficulty of compactly representing masks, making the design of one-stage methods very challenging. In this work, we propose a simple singleshot instance segmentation framework, termed mask encoding based instance segmentation (MEInst). Instead of predicting the two-dimensional mask directly, MEInst distills it into a compact and fixed-dimensional representation vector, which allows the instance segmentation task to be incorporated into one-stage bounding-box detectors and results in a simple yet efficient instance segmentation framework. The proposed one-stage MEInst achieves 36.4% in mask AP with single-model (ResNeXt-101-FPN backbone) and single-scale testing on the MS-COCO benchmark. We show that the much simpler and flexible one-stage instance segmentation method, can also achieve competitive performance. This framework can be easily adapted for other instance-level recognition tasks. Code is available at: https://git.io/AdelaiDet

cs.CV↗

arXiv2020

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

Instance segmentation is one of the fundamental vision tasks. Recently, fully convolutional instance segmentation methods have drawn much attention as they are often simpler and more efficient than two-stage approaches like Mask R-CNN. To date, almost all such approaches fall behind the two-stage Mask R-CNN method in mask precision when models have similar computation complexity, leaving great room for improvement. In this work, we achieve improved mask prediction by effectively combining instance-level information with semantic information with lower-level fine-granularity. Our main contribution is a blender module which draws inspiration from both top-down and bottom-up instance segmentation approaches. The proposed BlendMask can effectively predict dense per-pixel position-sensitive instance features with very few channels, and learn attention maps for each instance with merely one convolution layer, thus being fast in inference. BlendMask can be easily incorporated with the state-of-the-art one-stage detection frameworks and outperforms Mask R-CNN under the same training schedule while being 20% faster. A light-weight version of BlendMask achieves $ 34.2% $ mAP at 25 FPS evaluated on a single 1080Ti GPU card. Because of its simplicity and efficacy, we hope that our BlendMask could serve as a simple yet strong baseline for a wide range of instance-wise prediction tasks. Code is available at https://git.io/AdelaiDet

cs.CV↗

arXiv2020

DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data

We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation, thus preserving accurate shapes of a scene. Previous methods that predict metric depth often work well only for a specific scene. In contrast, learning relative depth (information of being closer or further) can enjoy better generalization, with the price of failing to recover the accurate geometric shape of the scene. In this work, we propose a dataset and methods to tackle this dilemma, aiming to predict accurate depth up to an affine transformation with good generalization to diverse scenes. First we construct a large-scale and diverse dataset, termed Diverse Scene Depth dataset (DiverseDepth), which has a broad range of scenes and foreground contents. Compared with previous learning objectives, i.e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes. Furthermore, in order to train the model on the complex dataset effectively, we propose a multi-curriculum learning method. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting, demonstrating the excellent generalization capacity of the learned model to diverse scenes. The reconstructed point clouds with the predicted depth show that our method can recover high-quality 3D shapes. Code and dataset are available at: https://tinyurl.com/DiverseDepth

cs.CV↗

arXiv2020

NAS-FCOS: Fast Neural Architecture Search for Object Detection

The success of deep neural networks relies on significant architecture engineering. Recently neural architecture search (NAS) has emerged as a promise to greatly reduce manual effort in network design by automatically searching for optimal architectures, although typically such algorithms need an excessive amount of computational resources, e.g., a few thousand GPU-days. To date, on challenging vision tasks such as object detection, NAS, especially fast versions of NAS, is less studied. Here we propose to search for the decoder structure of object detectors with search efficiency being taken into consideration. To be more specific, we aim to efficiently search for the feature pyramid network (FPN) as well as the prediction head of a simple anchor-free object detector, namely FCOS, using a tailored reinforcement learning paradigm. With carefully designed search space, search algorithms and strategies for evaluating network quality, we are able to efficiently search a top-performing detection architecture within 4 days using 8 V100 GPUs. The discovered architecture surpasses state-of-the-art object detection models (such as Faster R-CNN, RetinaNet and FCOS) by 1.5 to 3.5 points in AP on the COCO dataset, with comparable computation complexity and memory footprint, demonstrating the efficacy of the proposed NAS for object detection.

cs.CV↗

arXiv2020

Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation

3D point cloud semantic and instance segmentation is crucial and fundamental for 3D scene understanding. Due to the complex structure, point sets are distributed off balance and diversely, which appears as both category imbalance and pattern imbalance. As a result, deep networks can easily forget the non-dominant cases during the learning process, resulting in unsatisfactory performance. Although re-weighting can reduce the influence of the well-classified examples, they cannot handle the non-dominant patterns during the dynamic training. In this paper, we propose a memory-augmented network to learn and memorize the representative prototypes that cover diverse samples universally. Specifically, a memory module is introduced to alleviate the forgetting issue by recording the patterns seen in mini-batch training. The learned memory items consistently reflect the interpretable and meaningful information for both dominant and non-dominant categories and cases. The distorted observations and rare cases can thus be augmented by retrieving the stored prototypes, leading to better performances and generalization. Exhaustive experiments on the benchmarks, i.e. S3DIS and ScanNetV2, reflect the superiority of our method on both effectiveness and efficiency. Not only the overall accuracy but also nondominant classes have improved substantially.

cs.CV↗

arXiv2019

A Class of Distributed Event-Triggered Average Consensus Algorithms for Multi-Agent Systems

This paper proposes a class of distributed event-triggered algorithms that solve the average consensus problem in multi-agent systems. By designing events such that a specifically chosen Lyapunov function is monotonically decreasing, event-triggered algorithms succeed in reducing communications among agents while still ensuring that the entire system converges to the desired state. However, depending on the chosen Lyapunov function the transient behaviors can be very different. Moreover, performance requirements also vary from application to application. Consequently, we are instead interested in considering a class of Lyapunov functions such that each Lyapunov function produces a different event-triggered coordination algorithm to solve the multi-agent average consensus problem. The proposed class of algorithms all guarantee exponential convergence of the resulting system and exclusion of Zeno behaviors. This allows us to easily implement different algorithms that all guarantee correctness to meet varying performance needs. We show that our findings can be applied to the practical clock synchronization problem in wireless sensor networks (WSNs) and further corroborate their effectiveness with simulation results.

cs.MA↗

arXiv2019

DirectPose: Direct End-to-End Multi-Person Pose Estimation

We propose the first direct end-to-end multi-person pose estimation framework, termed DirectPose. Inspired by recent anchor-free object detectors, which directly regress the two corners of target bounding-boxes, the proposed framework directly predicts instance-aware keypoints for all the instances from a raw input image, eliminating the need for heuristic grouping in bottom-up methods or bounding-box detection and RoI operations in top-down ones. We also propose a novel Keypoint Alignment (KPAlign) mechanism, which overcomes the main difficulty: lack of the alignment between the convolutional features and predictions in this end-to-end framework. KPAlign improves the framework's performance by a large margin while still keeping the framework end-to-end trainable. With the only postprocessing non-maximum suppression (NMS), our proposed framework can detect multi-person keypoints with or without bounding-boxes in a single shot. Experiments demonstrate that the end-to-end paradigm can achieve competitive or better performance than previous strong baselines, in both bottom-up and top-down methods. We hope that our end-to-end approach can provide a new perspective for the human pose estimation task.

cs.CV↗

arXiv2019

Convolutional Character Networks

Recent progress has been made on developing a unified framework for joint text detection and recognition in natural images, but existing joint models were mostly built on two-stage framework by involving ROI pooling, which can degrade the performance on recognition task. In this work, we propose convolutional character networks, referred as CharNet, which is an one-stage model that can process two tasks simultaneously in one pass. CharNet directly outputs bounding boxes of words and characters, with corresponding character labels. We utilize character as basic element, allowing us to overcome the main difficulty of existing approaches that attempted to optimize text detection jointly with a RNN-based recognition branch. In addition, we develop an iterative character detection approach able to transform the ability of character detection learned from synthetic data to real-world images. These technical improvements result in a simple, compact, yet powerful one-stage model that works reliably on multi-orientation and curved text. We evaluate CharNet on three standard benchmarks, where it consistently outperforms the state-of-the-art approaches [25, 24] by a large margin, e.g., with improvements of 65.33%->71.08% (with generic lexicon) on ICDAR 2015, and 54.0%->69.23% on Total-Text, on end-to-end text recognition. Code is available at: https://github.com/MalongTech/research-charnet.

cs.CV↗

arXiv2019

Communication-Censored Linearized ADMM for Decentralized Consensus Optimization

In this paper, we propose a communication- and computation-efficient algorithm to solve a convex consensus optimization problem defined over a decentralized network. A remarkable existing algorithm to solve this problem is the alternating direction method of multipliers (ADMM), in which at every iteration every node updates its local variable through combining neighboring variables and solving an optimization subproblem. The proposed algorithm, called as COmmunication-censored Linearized ADMM (COLA), leverages a linearization technique to reduce the iteration-wise computation cost of ADMM and uses a communication-censoring strategy to alleviate the communication cost. To be specific, COLA introduces successive linearization approximations to the local cost functions such that the resultant computation is first-order and light-weight. Since the linearization technique slows down the convergence speed, COLA further adopts the communication-censoring strategy to avoid transmissions of less informative messages. A node is allowed to transmit only if the distance between the current local variable and its previously transmitted one is larger than a censoring threshold. COLA is proven to be convergent when the local cost functions have Lipschitz continuous gradients and the censoring threshold is summable. When the local cost functions are further strongly convex, we establish the linear (sublinear) convergence rate of COLA, given that the censoring threshold linearly (sublinearly) decays to 0. Numerical experiments corroborate with the theoretical findings and demonstrate the satisfactory communication-computation tradeoff of COLA.

math.OC↗

arXiv2019

FCOS: Fully Convolutional One-Stage Object Detection

We propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the predefined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating overlapping during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often very sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), FCOS with ResNeXt-64x4d-101 achieves 44.7% in AP with single-model and single-scale testing, surpassing previous one-stage detectors with the advantage of being much simpler. For the first time, we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code is available at:Code is available at: https://tinyurl.com/FCOSv1

cs.CV↗

arXiv2019

Super-Resolution Channel Estimation for Arbitrary Arrays in Hybrid Millimeter-Wave Massive MIMO Systems

This paper develops efficient channel estimation techniques for millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) systems under practical hardware limitations, including an arbitrary array geometry and a hybrid hardware structure. Taking on an angle-based approach, this work adopts a generalized array manifold separation approach via Jacobi-Anger approximation, which transforms a non-ideal, non-uniform array manifold into a virtual array domain with a desired uniform geometric structure to facilitate super-resolution angle estimation and channel acquisition. Accordingly, structure-based optimization techniques are developed to effectively estimate both the channel covariance and the instantaneous channel state information (CSI) within a short sensing time. In particular, the difference in time-variation of channel path angles and path gains is capitalized to design a two-step CSI estimation scheme that can quickly sense fading channels. Theoretical results are provided on the fundamental limits of the proposed technique in terms of sample efficiency. For computational efficiency, a fast iterative algorithm is developed via the alternating direction method of multipliers. Other related issues such as spurious-peak cancellation in nonuniform linear arrays and extensions to higher-dimensional cases are also discussed. Simulations testify the effectiveness of the proposed approaches in hybrid mmWave massive MIMO systems with arbitrary arrays.

eess.SP↗

arXiv2019

Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Recent semantic segmentation methods exploit encoder-decoder architectures to produce the desired pixel-wise segmentation prediction. The last layer of the decoders is typically a bilinear upsampling procedure to recover the final pixel-wise prediction. We empirically show that this oversimple and data-independent bilinear upsampling may lead to sub-optimal results. In this work, we propose a data-dependent upsampling (DUpsampling) to replace bilinear, which takes advantages of the redundancy in the label space of semantic segmentation and is able to recover the pixel-wise prediction from low-resolution outputs of CNNs. The main advantage of the new upsampling layer lies in that with a relatively lower-resolution feature map such as $\frac{1}{16}$ or $\frac{1}{32}$ of the input size, we can achieve even better segmentation accuracy, significantly reducing computation complexity. This is made possible by 1) the new upsampling layer's much improved reconstruction capability; and more importantly 2) the DUpsampling based decoder's flexibility in leveraging almost arbitrary combinations of the CNN encoders' features. Experiments demonstrate that our proposed decoder outperforms the state-of-the-art decoder, with only $\sim$20\% of computation. Finally, without any post-processing, the framework equipped with our proposed decoder achieves new state-of-the-art performance on two datasets: 88.1\% mIOU on PASCAL VOC with 30\% computation of the previously best model; and 52.5\% mIOU on PASCAL Context.

cs.CV↗

arXiv2019

Knowledge Adaptation for Efficient Semantic Segmentation

Both accuracy and efficiency are of significant importance to the task of semantic segmentation. Existing deep FCNs suffer from heavy computations due to a series of high-resolution feature maps for preserving the detailed knowledge in dense estimation. Although reducing the feature map resolution (i.e., applying a large overall stride) via subsampling operations (e.g., pooling and convolution striding) can instantly increase the efficiency, it dramatically decreases the estimation accuracy. To tackle this dilemma, we propose a knowledge distillation method tailored for semantic segmentation to improve the performance of the compact FCNs with large overall stride. To handle the inconsistency between the features of the student and teacher network, we optimize the feature similarity in a transferred latent domain formulated by utilizing a pre-trained autoencoder. Moreover, an affinity distillation module is proposed to capture the long-range dependency by calculating the non-local interactions across the whole image. To validate the effectiveness of our proposed method, extensive experiments have been conducted on three popular benchmarks: Pascal VOC, Cityscapes and Pascal Context. Built upon a highly competitive baseline, our proposed method can improve the performance of a student network by 2.5\% (mIOU boosts from 70.2 to 72.7 on the cityscapes test set) and can train a better compact model with only 8\% float operations (FLOPS) of a model that achieves comparable performances.

cs.CV↗

arXiv2018

Efficient Two-Dimensional Line Spectrum Estimation Based on Decoupled Atomic Norm Minimization

This paper presents an efficient optimization technique for gridless {2-D} line spectrum estimation, named decoupled atomic norm minimization (D-ANM). The framework of atomic norm minimization (ANM) is considered, which has been successfully applied in 1-D problems to allow super-resolution frequency estimation for correlated sources even when the number of snapshots is highly limited. The state-of-the-art 2-D ANM approach vectorizes the 2-D measurements to their 1-D equivalence, which incurs huge computational cost and may become too costly for practical applications. We develop a novel decoupled approach of 2-D ANM via semi-definite programming (SDP), which introduces a new matrix-form atom set to naturally decouple the joint observations in both dimensions without loss of optimality. Accordingly, the original large-scale 2-D problem is equivalently reformulated via two decoupled one-level Toeplitz matrices, which can be solved by simple 1-D frequency estimation with pairing. Compared with the conventional vectorized approach, the proposed D-ANM technique reduces the computational complexity by several orders of magnitude with respect to the problem size. It also retains the benefits of ANM in terms of precise signal recovery, small number of required measurements, and robustness to source correlation. The complexity benefits are particularly attractive for large-scale antenna systems such as massive MIMO, radar signal processing and radio astronomy.

eess.SP↗

arXiv2018

A Hybrid Neural Network Framework and Application to Radar Automatic Target Recognition

Deep neural networks (DNNs) have found applications in diverse signal processing (SP) problems. Most efforts either directly adopt the DNN as a black-box approach to perform certain SP tasks without taking into account of any known properties of the signal models, or insert a pre-defined SP operator into a DNN as an add-on data processing stage. This paper presents a novel hybrid-NN framework in which one or more SP layers are inserted into the DNN architecture in a coherent manner to enhance the network capability and efficiency in feature extraction. These SP layers are properly designed to make good use of the available models and properties of the data. The network training algorithm of hybrid-NN is designed to actively involve the SP layers in the learning goal, by simultaneously optimizing both the weights of the DNN and the unknown tuning parameters of the SP operators. The proposed hybrid-NN is tested on a radar automatic target recognition (ATR) problem. It achieves high validation accuracy of 96\% with 5,000 training images in radar ATR. Compared with ordinary DNN, hybrid-NN can markedly reduce the required amount of training data and improve the learning performance.

eess.SP↗

arXiv2018

ANM-PhaseLift: Structured Line Spectrum Estimation from Quadratic Measurements

PhaseLift is a noted convex optimization technique for phase retrieval that can recover a signal exactly from amplitude measurements only, with high probability. Conventional PhaseLift requires a relatively large number of samples that sometimes can be costly to acquire. % to compensate for the missing phase information and achieve effective phase retrieval. This paper focuses on some practical applications where the signal of interest is composed of a few Vandermonde components, such as line spectra.A novel phase retrieval framework, namely ANM-PhaseLift, is developed that exploits the Vandermonde structure to alleviate the sampling requirements. Specifically, the atom set of amplitude-based quadratic measurements is identified, and atomic norm minimization (ANM) is introduced into PhaseLift to considerably reduce the number of measurements that are needed for accurate phase retrieval. The benefit of ANM-PhaseLift is particularly attractive in applications where the Vandermonde structure is presented, such as massive MIMO and radar imaging.

eess.SP↗

arXiv2018

Low-complexity optimization for Two-Dimensional Direction-of-arrival Estimation via Decoupled Atomic Norm Minimization

This paper presents an efficient optimization technique for super-resolution two-dimensional (2D) direction of arrival (DOA) estimation by introducing a new formulation of atomic norm minimization (ANM). ANM allows gridless angle estimation for correlated sources even when the number of snapshots is far less than the antenna size, yet it incurs huge computational cost in 2D processing. This paper introduces a novel formulation of ANM via semi-definite programming, which expresses the original high-dimensional problem by two decoupled Toeplitz matrices in one dimension, followed by 1D angle estimation with automatic angle pairing. Compared with the state-of-the-art 2D ANM, the proposed technique reduces the computational complexity by several orders of magnitude with respect to the antenna size, while retaining the benefits of ANM in terms of super-resolution performance with use of a small number of measurements, and robustness to source correlation and noise. The complexity benefits are particularly attractive for large-scale antenna systems such as massive MIMO and radio astronomy.

eess.SP↗