Searcharxiv⌕ Search

arXiv subjects

Tao Huang

Publications and source records attributed to Tao Huang.

At least 253 records · Page 14Linked to original sources
arXiv2021

Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images

In the last few years, image denoising has benefited a lot from the fast development of neural networks. However, the requirement of large amounts of noisy-clean image pairs for supervision limits the wide use of these models. Although there have been a few attempts in training an image denoising model with only single noisy images, existing self-supervised denoising approaches suffer from inefficient network training, loss of useful information, or dependence on noise modeling. In this paper, we present a very simple yet effective method named Neighbor2Neighbor to train an effective image denoising model with only noisy images. Firstly, a random neighbor sub-sampler is proposed for the generation of training image pairs. In detail, input and target used to train a network are images sub-sampled from the same noisy image, satisfying the requirement that paired pixels of paired images are neighbors and have very similar appearance with each other. Secondly, a denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance. The proposed Neighbor2Neighbor framework is able to enjoy the progress of state-of-the-art supervised denoising networks in network architecture design. Moreover, it avoids heavy dependence on the assumption of the noise distribution. We explain our approach from a theoretical perspective and further validate it through extensive experiments, including synthetic experiments with different noise distributions in sRGB space and real-world experiments on a denoising benchmark dataset in raw-RGB space.

eess.IV↗
arXiv2021

Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging

In coded aperture snapshot spectral imaging (CASSI) system, the real-world hyperspectral image (HSI) can be reconstructed from the captured compressive image in a snapshot. Model-based HSI reconstruction methods employed hand-crafted priors to solve the reconstruction problem, but most of which achieved limited success due to the poor representation capability of these hand-crafted priors. Deep learning based methods learning the mappings between the compressive images and the HSIs directly achieved much better results. Yet, it is nontrivial to design a powerful deep network heuristically for achieving satisfied results. In this paper, we propose a novel HSI reconstruction method based on the Maximum a Posterior (MAP) estimation framework using learned Gaussian Scale Mixture (GSM) prior. Different from existing GSM models using hand-crafted scale priors (e.g., the Jeffrey's prior), we propose to learn the scale prior through a deep convolutional neural network (DCNN). Furthermore, we also propose to estimate the local means of the GSM models by the DCNN. All the parameters of the MAP estimation algorithm and the DCNN parameters are jointly optimized through end-to-end training. Extensive experimental results on both synthetic and real datasets demonstrate that the proposed method outperforms existing state-of-the-art methods. The code is available at https://see.xidian.edu.cn/faculty/wsdong/Projects/DGSM-SCI.htm.

eess.IV↗
arXiv2021

Prioritized Architecture Sampling with Monto-Carlo Tree Search

One-shot neural architecture search (NAS) methods significantly reduce the search cost by considering the whole search space as one network, which only needs to be trained once. However, current methods select each operation independently without considering previous layers. Besides, the historical information obtained with huge computation cost is usually used only once and then discarded. In this paper, we introduce a sampling strategy based on Monte Carlo tree search (MCTS) with the search space modeled as a Monte Carlo tree (MCT), which captures the dependency among layers. Furthermore, intermediate results are stored in the MCT for the future decision and a better exploration-exploitation balance. Concretely, MCT is updated using the training loss as a reward to the architecture performance; for accurately evaluating the numerous nodes, we propose node communication and hierarchical node selection methods in the training and search stages, respectively, which make better uses of the operation rewards and hierarchical information. Moreover, for a fair comparison of different NAS methods, we construct an open-source NAS benchmark of a macro search space evaluated on CIFAR-10, namely NAS-Bench-Macro. Extensive experiments on NAS-Bench-Macro and ImageNet demonstrate that our method significantly improves search efficiency and performance. For example, by only searching $20$ architectures, our obtained architecture achieves $78.0\%$ top-1 accuracy with 442M FLOPs on ImageNet. Code (Benchmark) is available at: \url{https://github.com/xiusu/NAS-Bench-Macro}.

cs.CV↗
arXiv2021

Locally Free Weight Sharing for Network Width Search

Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluator to rank the performance \wrt~different width. Nevertheless, current methods mainly follow a manually fixed weight sharing pattern, which is limited to distinguish the performance gap of different width. In this paper, to better evaluate each width, we propose a locally free weight sharing strategy (CafeNet) accordingly. In CafeNet, weights are more freely shared, and each width is jointly indicated by its base channels and free channels, where free channels are supposed to loCAte FrEely in a local zone to better represent each width. Besides, we propose to further reduce the search space by leveraging our introduced FLOPs-sensitive bins. As a result, our CafeNet can be trained stochastically and get optimized within a min-min strategy. Extensive experiments on ImageNet, CIFAR-10, CelebA and MS COCO dataset have verified our superiority comparing to other state-of-the-art baselines. For example, our method can further boost the benchmark NAS network EfficientNet-B0 by 0.41\% via searching its width more delicately.

cs.CV↗
arXiv2020

MG-GCN: Fast and Effective Learning with Mix-grained Aggregators for Training Large Graph Convolutional Networks

Graph convolutional networks (GCNs) have been employed as a kind of significant tool on many graph-based applications recently. Inspired by convolutional neural networks (CNNs), GCNs generate the embeddings of nodes by aggregating the information of their neighbors layer by layer. However, the high computational and memory cost of GCNs due to the recursive neighborhood expansion across GCN layers makes it infeasible for training on large graphs. To tackle this issue, several sampling methods during the process of information aggregation have been proposed to train GCNs in a mini-batch Stochastic Gradient Descent (SGD) manner. Nevertheless, these sampling strategies sometimes bring concerns about insufficient information collection, which may hinder the learning performance in terms of accuracy and convergence. To tackle the dilemma between accuracy and efficiency, we propose to use aggregators with different granularities to gather neighborhood information in different layers. Then, a degree-based sampling strategy, which avoids the exponential complexity, is constructed for sampling a fixed number of nodes. Combining the above two mechanisms, the proposed model, named Mix-grained GCN (MG-GCN) achieves state-of-the-art performance in terms of accuracy, training speed, convergence speed, and memory cost through a comprehensive set of experiments on four commonly used benchmark datasets and a new Ethereum dataset.

cs.LG↗
arXiv2020

Data Agnostic Filter Gating for Efficient Deep Networks

To deploy a well-trained CNN model on low-end computation edge devices, it is usually supposed to compress or prune the model under certain computation budget (e.g., FLOPs). Current filter pruning methods mainly leverage feature maps to generate important scores for filters and prune those with smaller scores, which ignores the variance of input batches to the difference in sparse structure over filters. In this paper, we propose a data agnostic filter pruning method that uses an auxiliary network named Dagger module to induce pruning and takes pretrained weights as input to learn the importance of each filter. In addition, to help prune filters with certain FLOPs constraints, we leverage an explicit FLOPs-aware regularization to directly promote pruning filters toward target FLOPs. Extensive experimental results on CIFAR-10 and ImageNet datasets indicate our superiority to other state-of-the-art filter pruning methods. For example, our 50\% FLOPs ResNet-50 can achieve 76.1\% Top-1 accuracy on ImageNet dataset, surpassing many other filter pruning methods.

cs.CV↗
arXiv2020

Inverse Solidification Induced by Active Janus Particles

Crystals melt when thermal excitations or the concentration of defects in the lattice is sufficiently high. Upon melting, the crystalline long-range order vanishes, turning the solid to a fluid. In contrast to this classical scenario of solid melting, here we demonstrate a counter-intuitive behavior of the occurrence of crystalline long-range order in an initially disordered matrix. This unusual solidification is demonstrated in a system of passive colloidal particles accommodating chemically active defects -- photocatalytic Janus particles. The observed crystallization occurs when the amount of active-defect-induced fluctuations (which is the measure of the effective temperature) reaches critical value. The driving mechanism behind this unusual behavior is purely internal and resembles a blast-induced solidification. Here the role of "internal micro-blasts" is played by the photochemical activity of defects residing in the colloidal matrix. The defect-induced solidification occurs under non-equilibrium conditions: the resulting solid exists as long as a constant supply of energy in the form of ion flow is provided by the catalytic photochemical reaction at the surface of active Janus particle defects. Our findings could be useful for understanding of the phase transitions of matter under extreme conditions far from thermodynamic equilibrium.

cond-mat.soft↗
arXiv2020

Unified statistical inference for a novel nonlinear dynamic functional/longitudinal data model

In light of recent work studying massive functional/longitudinal data, such as the resulting data from the COVID-19 pandemic, we propose a novel functional/longitudinal data model which is a combination of the popular varying coefficient (VC) model and additive model. We call it Semi-VCAM in which the response could be a functional/longitudinal variable, and the explanatory variables could be a mixture of functional/longitudinal and scalar variables. Notably some of the scalar variables could be categorical variables as well. The Semi-VCAM simultaneously allows for both substantial flexibility and the maintaining of one-dimensional rates of convergence. A local linear smoothing with the aid of an initial B spline series approximation is developed to estimate the unknown functional effects in the model. To avoid the subjective choice between the sparse and dense cases of the data, we establish the asymptotic theories of the resultant Pilot Estimation Based Local Linear Estimators (PEBLLE) on a unified framework of sparse, dense and ultra-dense cases of the data. Moreover, we construct unified consistent tests to justify whether a parsimony submodel is sufficient or not. These test methods also avoid the subjective choice between the sparse, dense and ultra dense cases of the data. Extensive Monte Carlo simulation studies investigating the finite sample performance of the proposed methodologies confirm our asymptotic results. We further illustrate our methodologies via analyzing the COVID-19 data from China and the CD4 data.

stat.ME↗
arXiv2020

Weak compactness of simplified nematic liquid flows in 2D

For any bounded, smooth domain $Ω\subset \R^2$, %(or $Ω=\R^2$), we will establish the weak compactness property of solutions to the simplified Ericksen-Leslie system for both uniaxial and biaxial nematics, and the convergence of weak solutions of the Ginzburg-Landau type nematic liquid crystal flow to a weak solution of the simplified Ericksen-Leslie system as the parameter tends to zero. This is based on the compensated compactness property of the Ericksen stress tensors, which is obtained by the $L^p$-estimate of the Hopf differential for the Ericksen-Leslie system and the Pohozaev type argument for the Ginzburg-Landau type nematic liquid crystal flow.

math.AP↗
arXiv2020

GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet

Training a supernet matters for one-shot neural architecture search (NAS) methods since it serves as a basic performance estimator for different architectures (paths). Current methods mainly hold the assumption that a supernet should give a reasonable ranking over all paths. They thus treat all paths equally, and spare much effort to train paths. However, it is harsh for a single supernet to evaluate accurately on such a huge-scale search space (e.g., $7^{21}$). In this paper, instead of covering all paths, we ease the burden of supernet by encouraging it to focus more on evaluation of those potentially-good ones, which are identified using a surrogate portion of validation data. Concretely, during training, we propose a multi-path sampling strategy with rejection, and greedily filter the weak paths. The training efficiency is thus boosted since the training space has been greedily shrunk from all paths to those potentially-good ones. Moreover, we further adopt an exploration and exploitation policy by introducing an empirical candidate path pool. Our proposed method GreedyNAS is easy-to-follow, and experimental results on ImageNet dataset indicate that it can achieve better Top-1 accuracy under same search space and FLOPs or latency level, but with only $\sim$60\% of supernet training cost. By searching on a larger space, our GreedyNAS can also obtain new state-of-the-art architectures.

cs.CV↗
arXiv2020

Grant-Free Non-Orthogonal Multiple Access: A Key Enabler for 6G-IoT

The proliferating number of devices with short payloads as well as low power budget has already driven researchers away from classical grant-based access schemes that are notorious for their large signalling overhead as well as power-consuming retransmissions. Instead, light-weight random access protocols have been re-investigated and their throughput has been improved in orders of magnitude with sophisticated yet still low-complex transceiver algorithms. In fact, grant-free access has been identified as a key medium access control technique for providing massive connectivity in machine type communications in cellular networks. In this paper, we show that grant-free access combined with non-orthogonal transmission schemes is a promising solution for 6G Internet of Things (IoT). We present novel and promising results for deep learning (DL)-based techniques for joint user detection and decoding. Then, we propose a multi-layered model for GF-NOMA for power-efficient communications. We also discuss resource allocation issues to enable the co-existence of GF-NOMA with other orthogonal or even grant-based schemes. Finally, we conclude with proposed research directions for medium access towards enabling 6G-IoT.

cs.IT↗
arXiv2019

Robust Data Preprocessing for Machine-Learning-Based Disk Failure Prediction in Cloud Production Environments

To provide proactive fault tolerance for modern cloud data centers, extensive studies have proposed machine learning (ML) approaches to predict imminent disk failures for early remedy and evaluated their approaches directly on public datasets (e.g., Backblaze SMART logs). However, in real-world production environments, the data quality is imperfect (e.g., inaccurate labeling, missing data samples, and complex failure types), thereby degrading the prediction accuracy. We present RODMAN, a robust data preprocessing pipeline that refines data samples before feeding them into ML models. We start with a large-scale trace-driven study of over three million disks from Alibaba Cloud's data centers, and motivate the practical challenges in ML-based disk failure prediction. We then design RODMAN with three data preprocessing echniques, namely failure-type filtering, spline-based data filling, and automated pre-failure backtracking, that are applicable for general ML models. Evaluation on both the Alibaba and Backblaze datasets shows that RODMAN improves the prediction accuracy compared to without data preprocessing under various settings.

cs.LG↗
arXiv2019

Poiseuille flow of nematic liquid crystals via the full Ericksen-Leslie model

In this paper, we study the Cauchy problem of the Poiseuille flow of full Ericksen-Leslie model for nematic liquid crystals. The model is a coupled system of a parabolic equation for the velocity and a quasilinear wave equation for the director. For a particular choice of several physical parameter values, we construct solutions with smooth initial data and finite energy that produce, in finite time, cusp singularities - blowups of gradients. The formation of cusp singularity is due to local interactions of wave-like characteristics of solutions, which is different from the mechanism of finite time singularity formations for the parabolic Ericksen-Leslie system. The finite time singularity formation for the physical model might raise some concerns for purposes of applications. This is, however, resolved satisfactorily; more precisely, we are able to establish the global existence of weak solutions that are Hölder continuous and have bounded energy. One major contribution of this paper is our identification of the effect of the flux density of the velocity on the director and the reveal of a singularity cancellation - the flux density remains uniformly bounded while its two components approach infinity at formations of cusp singularities.

math.AP↗
arXiv2018

Mean Field Control and Mean Field Game Models with Several Populations

In this paper, we investigate the interaction of two populations with a large number of indistinguishable agents. The problem consists in two levels: the interaction between agents of a same population, and the interaction between the two populations. In the spirit of mean field type control (MFC) problems and mean field games (MFG), each population is approximated by a continuum of infinitesimal agents. We define four different problems in a general context and interpret them in the framework of MFC or MFG. By calculus of variations, we derive formally in each case the adjoint equations for the necessary conditions of optimality. Importantly, we find that in the case of a competition between two coalitions, one needs to rely on a system of Master equations in order to describe the equilibrium. Examples are provided, in particular linear-quadratic models for which we obtain systems of ODEs that can be related to Riccati equations.

math.OC↗
arXiv2018

The Ratio $\mathcal{R}(D)$ and the $D$-meson Distribution Amplitude

In this paper, we calculate the $B\to D$ transition form factors (TFFs) within the light-cone sum rules (LCSR) and predict the ratio $\mathcal{R}(D)$. More accurate $D$-meson distribution amplitudes (DAs) are essential to get a more accurate theoretical prediction. We construct a new model for the twist-3 DAs $ϕ^p_{3;D}$ and $ϕ^σ_{3;D}$ based on the QCD sum rules under the background field theory for their moments as we have done for constructing the leading-twist DA $ϕ_{2;D}$. As an application, we observe that the twist-3 contributions are sizable in whole $q^2$-region. Taking the twist-2 and twist-3 DAs into consideration, we obtain $f^{B\to D}_{+,0}(0) = 0.659^{+0.029}_{-0.032}$. As a combination of the Lattice QCD and the QCD LCSR predictions on the TFFs $f^{B\to D}_{+,0}(q^2)$, we predict $\mathcal{R}(D) = 0.320^{+0.018}_{-0.021}$, which improves is about $1.5σ$ deviation from the HFAG average of the Belle and BABAR data. At present the data are still of large errors, and we need further accurate measurements of the experiment to confirm whether there is signal of new physics from the ratio $\mathcal{R}(D)$.

hep-ph↗
arXiv2018

The Dimension of Eigenvariety of Nonnegative Tensors Associated with Spectral Radius

For a nonnegative weakly irreducible tensor, its spectral radius is an eigenvalue corresponding to a unique positive eigenvector up to a scalar called the Perron vector. But including the Perron vector, it may have more than one eigenvector corresponding to the spectral radius. The projective eigenvariety associated with the spectral radius is the set of the eigenvectors corresponding to the spectral radius considered in the complex projective space. In this paper we prove that the dimension of the above projective eigenvariety is zero, i.e. there are finite many eigenvectors associated with the spectral radius up to a scalar. For a general nonnegative tensor, we characterize the nonnegative combinatorially symmetric tensor for which the dimension of projective eigenvariety associated with spectral radius is greater than zero. Finally we apply those results to the adjacency tensors of uniform hypergraphs.

math.CO↗
arXiv2018

Eigenvariety of Nonnegative Symmetric Weakly Irreducible Tensors Associated with Spectral Radius and Its Application to Hypergraphs

For a nonnegative symmetric weakly irreducible tensor, its spectral radius is an eigenvalue corresponding to a unique positive eigenvector up to a scalar called the Perron vector. But including the Perron vector, there may have more than one eigenvector corresponding to the spectral radius. The projective eigenvariety associated with the spectral radius is the set of the eigenvectors corresponding to the spectral radius considered in the complex projective space. In this paper we proved that such projective eigenvariety admits a module structure, which is determined by the support of the tensor and can be characterized explicitly by solving the Smith normal form of the incidence matrix of the tensor. We introduced two parameters: the stabilizing index and the stabilizing dimension of the tensor, where the former is exactly the cardinality of the projective eigenvariety and the latter is the composition length of the projective eigenvariety as a module. We give some upper bounds for the two parameters, and characterize the case that there is only one eigenvector of the tensor corresponding to the spectral radius, i.e. the Perron vector. By applying the above results to the adjacency tensor of a connected uniform hypergraph, we give some upper bounds for the two parameters in terms of the structural parameters of the hypergraph such as path cover number, matching number and the maximum length of paths.

math.CO↗