arXiv subjects

Chao Xu

Publications and source records attributed to Chao Xu.

At least 343 records · Page 19Linked to original sources

arXiv2020

CARS: Continuous Evolution for Efficient Neural Architecture Search

Searching techniques in most of existing neural architecture search (NAS) algorithms are mainly dominated by differentiable methods for the efficiency reason. In contrast, we develop an efficient continuous evolutionary approach for searching neural networks. Architectures in the population that share parameters within one SuperNet in the latest generation will be tuned over the training dataset with a few epochs. The searching in the next evolution generation will directly inherit both the SuperNet and the population, which accelerates the optimal network generation. The non-dominated sorting strategy is further applied to preserve only results on the Pareto front for accurately updating the SuperNet. Several neural networks with different model sizes and performances will be produced after the continuous search with only 0.4 GPU days. As a result, our framework provides a series of networks with the number of parameters ranging from 3.7M to 5.1M under mobile settings. These networks surpass those produced by the state-of-the-art methods on the benchmark ImageNet dataset.

cs.CV↗

arXiv2020

Distilling portable Generative Adversarial Networks for Image Translation

Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage cost. Traditional network compression methods focus on visually recognition tasks, but never deal with generation tasks. Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators. An adversarial learning process is therefore established to optimize student generator and student discriminator. Qualitative and quantitative analysis by conducting experiments on benchmark datasets demonstrate that the proposed method can learn portable generative models with strong performance.

cs.CV↗

arXiv2020

AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks

Caching has been regarded as a promising technique to alleviate energy consumption of sensors in Internet of Things (IoT) networks by responding to users' requests with the data packets stored in the edge caching node (ECN). For real-time applications in caching enabled IoT networks, it is essential to develop dynamic status update strategies to strike a balance between the information freshness experienced by users and energy consumed by the sensor, which, however, is not well addressed. In this paper, we first depict the evolution of information freshness, in terms of age of information (AoI), at each user. Then, we formulate a dynamic status update optimization problem to minimize the expectation of a long term accumulative cost, which jointly considers the users' AoI and sensor's energy consumption. To solve this problem, a Markov Decision Process (MDP) is formulated to cast the status updating procedure, and a model-free reinforcement learning algorithm is proposed, with which the challenge brought by the unknown of the formulated MDP's dynamics can be addressed. Finally, simulations are conducted to validate the convergence of our proposed algorithm and its effectiveness compared with the zero-wait baseline policy.

cs.IT↗

arXiv2020

Average Age of Changed Information in the Internet of Things

The freshness of status updates is imperative in mission-critical Internet of things (IoT) applications. Recently, Age of Information (AoI) has been proposed to measure the freshness of updates at the receiver. However, AoI only characterizes the freshness over time, but ignores the freshness in the content. In this paper, we introduce a new performance metric, Age of Changed Information (AoCI), which captures both the passage of time and the change of information content. Also, we examine the AoCI in a time-slotted status update system, where a sensor samples the physical process and transmits the update packets with a cost. We formulate a Markov Decision Process (MDP) to find the optimal updating policy that minimizes the weighted sum of the AoCI and the update cost. Particularly, in a special case that the physical process is modeled by a two-state discrete time Markov chain with equal transition probability, we show that the optimal policy is of threshold type with respect to the AoCI and derive the closed-form of the threshold. Finally, simulations are conducted to exhibit the performance of the threshold policy and its superiority over the zero-wait baseline policy.

cs.IT↗

arXiv2020

Alternating Minimization Based Trajectory Generation for Quadrotor Aggressive Flight

With much research has been conducted into trajectory planning for quadrotors, planning with spatial and temporal optimal trajectories in real-time is still challenging. In this paper, we propose a framework for generating large-scale piecewise polynomial trajectories for aggressive autonomous flights, with highlights on its superior computational efficiency and simultaneous spatial-temporal optimality. Exploiting the implicitly decoupled structure of the planning problem, we conduct alternating minimization between boundary conditions and time durations of trajectory pieces. In each minimization phase, we leverage the algebraic convenience of the sub-problem to escape poor local minima and achieve the lowest time consumption. Theoretical analysis for the global/local convergence rate of our proposed method is provided. Moreover, based on polynomial theory, an extremely fast feasibility check method is designed for various kinds of constraints. By incorporating the method into our alternating structure, a constrained minimization algorithm is constructed to optimize trajectories on the premise of feasibility. Benchmark evaluation shows that our algorithm outperforms state-of-the-art methods regarding efficiency, optimality, and scalability. Aggressive flight experiments in a limited space with dense obstacles are presented to demonstrate the performance of the proposed algorithm. We release our implementation as an open-source ros-package.

cs.RO↗

arXiv2020

Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks

Deep neural networks often consist of a great number of trainable parameters for extracting powerful features from given datasets. On one hand, massive trainable parameters significantly enhance the performance of these deep networks. On the other hand, they bring the problem of over-fitting. To this end, dropout based methods disable some elements in the output feature maps during the training phase for reducing the co-adaptation of neurons. Although the generalization ability of the resulting models can be enhanced by these approaches, the conventional binary dropout is not the optimal solution. Therefore, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks and propose a feature distortion method (Disout) for addressing the aforementioned problem. In the training period, randomly selected elements in the feature maps will be replaced with specific values by exploiting the generalization error bound. The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated on several benchmark image datasets.

cs.LG↗

arXiv2020

Detailed Proofs of Alternating Minimization Based Trajectory Generation for Quadrotor Aggressive Flight

This technical report provides detailed theoretical analysis of the algorithm used in \textit{Alternating Minimization Based Trajectory Generation for Quadrotor Aggressive Flight}. An assumption is provided to ensure that settings for the objective function are meaningful. What's more, we explore the structure of the optimization problem and analyze the global/local convergence rate of the employed algorithm.

cs.RO↗

arXiv2020

On Positive-Unlabeled Classification in GAN

This paper defines a positive and unlabeled classification problem for standard GANs, which then leads to a novel technique to stabilize the training of the discriminator in GANs. Traditionally, real data are taken as positive while generated data are negative. This positive-negative classification criterion was kept fixed all through the learning process of the discriminator without considering the gradually improved quality of generated data, even if they could be more realistic than real data at times. In contrast, it is more reasonable to treat the generated data as unlabeled, which could be positive or negative according to their quality. The discriminator is thus a classifier for this positive and unlabeled classification problem, and we derive a new Positive-Unlabeled GAN (PUGAN). We theoretically discuss the global optimality the proposed model will achieve and the equivalent optimization goal. Empirically, we find that PUGAN can achieve comparable or even better performance than those sophisticated discriminator stabilization methods.

cs.LG↗

arXiv2019

Data-Free Learning of Student Networks

Learning portable neural networks is very essential for computer vision for the purpose that pre-trained heavy deep models can be well applied on edge devices such as mobile phones and micro sensors. Most existing deep neural network compression and speed-up methods are very effective for training compact deep models, when we can directly access the training dataset. However, training data for the given deep network are often unavailable due to some practice problems (e.g. privacy, legal issue, and transmission), and the architecture of the given network are also unknown except some interfaces. To this end, we propose a novel framework for training efficient deep neural networks by exploiting generative adversarial networks (GANs). To be specific, the pre-trained teacher networks are regarded as a fixed discriminator and the generator is utilized for derivating training samples which can obtain the maximum response on the discriminator. Then, an efficient network with smaller model size and computational complexity is trained using the generated data and the teacher network, simultaneously. Efficient student networks learned using the proposed Data-Free Learning (DAFL) method achieve 92.22% and 74.47% accuracies using ResNet-18 without any training data on the CIFAR-10 and CIFAR-100 datasets, respectively. Meanwhile, our student network obtains an 80.56% accuracy on the CelebA benchmark.

cs.LG↗

arXiv2019

Bringing Giant Neural Networks Down to Earth with Unlabeled Data

Compressing giant neural networks has gained much attention for their extensive applications on edge devices such as cellphones. During the compressing process, one of the most important procedures is to retrain the pre-trained models using the original training dataset. However, due to the consideration of security, privacy or commercial profits, in practice, only a fraction of sample training data are made available, which makes the retraining infeasible. To solve this issue, this paper proposes to resort to unlabeled data in hand that can be cheaper to acquire. Specifically, we exploit the unlabeled data to mimic the classification characteristics of giant networks, so that the original capacity can be preserved nicely. Nevertheless, there exists a dataset bias between the labeled and unlabeled data, which may disturb the training and degrade the performance. We thus fix this bias by an adversarial loss to make an alignment on the distributions of their low-level feature representations. We further provide theoretical discussions about how the unlabeled data help compressed networks to generalize better. Experimental results demonstrate that the unlabeled data can significantly improve the performance of the compressed networks.

cs.LG↗

arXiv2019

Optimizing Information Freshness in Computing enabled IoT Networks

Internet of Things (IoT) has emerged as one of the key features of the next generation wireless networks, where timely delivery of status update packets is essential for many real-time IoT applications. To provide users with context-aware services and lighten the transmission burden, the raw data usually needs to be preprocessed before being transmitted to the destination. However, the effect of computing on the overall information freshness is not well understood. In this work, we first develop an analytical framework to investigate the information freshness, in terms of peak age of information (PAoI), of a computing enabled IoT system with multiple sensors. Specifically, we model the procedure of computing and transmission as a tandem queue, and derive the analytical expressions of the average PAoI for different sensors. Based on the theoretical results, we formulate a min-max optimization problem to minimize the maximum average PAoI of different sensors. We further design a derivative-free algorithm to find the optimal updating frequency, with which the complexity for checking the convexity of the formulated problem or obtaining the derivatives of the object function can be largely reduced. The accuracy of our analysis and effectiveness of the proposed algorithm are verified with extensive simulation results.

cs.IT↗

arXiv2019

Energy Efficient Transmission Based on Grouped Spatial Modulation for upstream DSL Systems

The digital Subscriber Line (DSL) remains an important component of heterogeneous networking, especially in historic city-centers, where using optical fibre is less realistic. Recently, the power consumption has become an important performance metric in telecommunication due to the associated environmental issues. In the recent bonding model, customer sites have been equipped with two/four copper pairs, which may be exploited for designing grouped spatial modulation (SM) aiming for reducing the power consumption and mitigating the stubborn crosstalk in DSL communications. Explicitly, we view the two pair copper pairs equipped for each user as a group and propose an energy efficient transmission scheme based on grouped SM strategy for the upstream DSL systems, which is capable of reducing the power consumption of the upstream transmitters by activating a single copper line of each user. More especially, in order to compensate for the potential bit-rate reduction imposed by reducing the number of activated lines, the proposed scheme implicitly delivers ``virtual bits" via activating/deactivating the lines in addition to the classic modulation scheme. This is particularly beneficial in the DSL context, because the cross-talk imposed by activating several lines may swamp the desired signal. Furthermore, a pair of near-optimal soft turbo detection schemes are proposed for exploiting the unique properties of the DSL channel in order to eliminate the error propagation problem of SM detection routinely encountered in wireless channels. Both the attainable energy-efficiency and the achievable Bit Error Ratio (BER) are investigated. Our simulation results demonstrate that the proposed group-based SM is capable of outperforming the vectoring scheme both in terms of its energy efficiency for all the examined loop lengths and transmit powers.

eess.SP↗

arXiv2019

Exciton gas transport through nano-constrictions

An indirect exciton is a bound state of an electron and a hole in spatially separated layers. Two-dimensional indirect excitons can be created optically in heterostructures containing double quantum wells or atomically thin semiconductors. We study theoretically transmission of such bosonic quasiparticles through nano-constrictions. We show that quantum transport phenomena, e.g., conductance quantization, single-slit diffraction, two-slit interference, and the Talbot effect, are experimentally realizable in systems of indirect excitons. We discuss similarities and differences between these phenomena and their counterparts in electronic devices.

cond-mat.mes-hall↗

arXiv2019

On the computational complexity of the probabilistic label tree algorithms

Label tree-based algorithms are widely used to tackle multi-class and multi-label problems with a large number of labels. We focus on a particular subclass of these algorithms that use probabilistic classifiers in the tree nodes. Examples of such algorithms are hierarchical softmax (HSM), designed for multi-class classification, and probabilistic label trees (PLTs) that generalize HSM to multi-label problems. If the tree structure is given, learning of PLT can be solved with provable regret guaranties [Wydmuch et.al. 2018]. However, to find a tree structure that results in a PLT with a low training and prediction computational costs as well as low statistical error seems to be a very challenging problem, not well-understood yet. In this paper, we address the problem of finding a tree structure that has low computational cost. First, we show that finding a tree with optimal training cost is NP-complete, nevertheless there are some tractable special cases with either perfect approximation or exact solution that can be obtained in linear time in terms of the number of labels $m$. For the general case, we obtain $O(\log m)$ approximation in linear time too. Moreover, we prove an upper bound on the expected prediction cost expressed in terms of the expected training cost. We also show that under additional assumptions the prediction cost of a PLT is $O(\log m)$.

cs.LG↗

arXiv2019

Decomposition-Based Transfer Distance Metric Learning for Image Classification

Distance metric learning (DML) is a critical factor for image analysis and pattern recognition. To learn a robust distance metric for a target task, we need abundant side information (i.e., the similarity/dissimilarity pairwise constraints over the labeled data), which is usually unavailable in practice due to the high labeling cost. This paper considers the transfer learning setting by exploiting the large quantity of side information from certain related, but different source tasks to help with target metric learning (with only a little side information). The state-of-the-art metric learning algorithms usually fail in this setting because the data distributions of the source task and target task are often quite different. We address this problem by assuming that the target distance metric lies in the space spanned by the eigenvectors of the source metrics (or other randomly generated bases). The target metric is represented as a combination of the base metrics, which are computed using the decomposed components of the source metrics (or simply a set of random bases); we call the proposed method, decomposition-based transfer DML (DTDML). In particular, DTDML learns a sparse combination of the base metrics to construct the target metric by forcing the target metric to be close to an integration of the source metrics. The main advantage of the proposed method compared with existing transfer metric learning approaches is that we directly learn the base metric coefficients instead of the target metric. To this end, far fewer variables need to be learned. We therefore obtain more reliable solutions given the limited side information and the optimization tends to be faster. Experiments on the popular handwritten image (digit, letter) classification and challenge natural image annotation tasks demonstrate the effectiveness of the proposed method.

cs.CV↗

arXiv2019

Multi-View Matrix Completion for Multi-Label Image Classification

There is growing interest in multi-label image classification due to its critical role in web-based image analytics-based applications, such as large-scale image retrieval and browsing. Matrix completion has recently been introduced as a method for transductive (semi-supervised) multi-label classification, and has several distinct advantages, including robustness to missing data and background noise in both feature and label space. However, it is limited by only considering data represented by a single-view feature, which cannot precisely characterize images containing several semantic concepts. To utilize multiple features taken from different views, we have to concatenate the different features as a long vector. But this concatenation is prone to over-fitting and often leads to very high time complexity in MC based image classification. Therefore, we propose to weightedly combine the MC outputs of different views, and present the multi-view matrix completion (MVMC) framework for transductive multi-label image classification. To learn the view combination weights effectively, we apply a cross validation strategy on the labeled set. In the learning process, we adopt the average precision (AP) loss, which is particular suitable for multi-label image classification. A least squares loss formulation is also presented for the sake of efficiency, and the robustness of the algorithm based on the AP loss compared with the other losses is investigated. Experimental evaluation on two real world datasets (PASCAL VOC' 07 and MIR Flickr) demonstrate the effectiveness of MVMC for transductive (semi-supervised) multi-label image classification, and show that MVMC can exploit complementary properties of different features and output-consistent labels for improved multi-label image classification.

stat.ML↗

arXiv2019

Large Margin Multi-modal Multi-task Feature Extraction for Image Classification

The features used in many image analysis-based applications are frequently of very high dimension. Feature extraction offers several advantages in high-dimensional cases, and many recent studies have used multi-task feature extraction approaches, which often outperform single-task feature extraction approaches. However, most of these methods are limited in that they only consider data represented by a single type of feature, even though features usually represent images from multiple modalities. We therefore propose a novel large margin multi-modal multi-task feature extraction (LM3FE) framework for handling multi-modal features for image classification. In particular, LM3FE simultaneously learns the feature extraction matrix for each modality and the modality combination coefficients. In this way, LM3FE not only handles correlated and noisy features, but also utilizes the complementarity of different modalities to further help reduce feature redundancy in each modality. The large margin principle employed also helps to extract strongly predictive features so that they are more suitable for prediction (e.g., classification). An alternating algorithm is developed for problem optimization and each sub-problem can be efficiently solved. Experiments on two challenging real-world image datasets demonstrate the effectiveness and superiority of the proposed method.

stat.ML↗

arXiv2019

Multi-view Vector-valued Manifold Regularization for Multi-label Image Classification

In computer vision, image datasets used for classification are naturally associated with multiple labels and comprised of multiple views, because each image may contain several objects (e.g. pedestrian, bicycle and tree) and is properly characterized by multiple visual features (e.g. color, texture and shape). Currently available tools ignore either the label relationship or the view complementary. Motivated by the success of the vector-valued function that constructs matrix-valued kernels to explore the multi-label structure in the output space, we introduce multi-view vector-valued manifold regularization (MV$\mathbf{^3}$MR) to integrate multiple features. MV$\mathbf{^3}$MR exploits the complementary property of different features and discovers the intrinsic local geometry of the compact support shared by different features under the theme of manifold regularization. We conducted extensive experiments on two challenging, but popular datasets, PASCAL VOC' 07 (VOC) and MIR Flickr (MIR), and validated the effectiveness of the proposed MV$\mathbf{^3}$MR for image classification.

stat.ML↗