arXiv subjects

Fan Zhou

Publications and source records attributed to Fan Zhou.

At least 163 records · Page 9Linked to original sources

arXiv2020

Metric learning by Similarity Network for Deep Semi-Supervised Learning

Deep semi-supervised learning has been widely implemented in the real-world due to the rapid development of deep learning. Recently, attention has shifted to the approaches such as Mean-Teacher to penalize the inconsistency between two perturbed input sets. Although these methods may achieve positive results, they ignore the relationship information between data instances. To solve this problem, we propose a novel method named Metric Learning by Similarity Network (MLSN), which aims to learn a distance metric adaptively on different domains. By co-training with the classification network, similarity network can learn more information about pairwise relationships and performs better on some empirical tasks than state-of-art methods.

stat.ML↗

arXiv2020

A Heterogeneous Dynamical Graph Neural Networks Approach to Quantify Scientific Impact

Quantifying and predicting the long-term impact of scientific writings or individual scholars has important implications for many policy decisions, such as funding proposal evaluation and identifying emerging research fields. In this work, we propose an approach based on Heterogeneous Dynamical Graph Neural Network (HDGNN) to explicitly model and predict the cumulative impact of papers and authors. HDGNN extends heterogeneous GNNs by incorporating temporally evolving characteristics and capturing both structural properties of attributed graph and the growing sequence of citation behavior. HDGNN is significantly different from previous models in its capability of modeling the node impact in a dynamic manner while taking into account the complex relations among nodes. Experiments conducted on a real citation dataset demonstrate its superior performance of predicting the impact of both papers and authors.

cs.SI↗

arXiv2020

Deep Active Learning: Unified and Principled Method for Query and Training

In this paper, we are proposing a unified and principled method for both the querying and training processes in deep batch active learning. We are providing theoretical insights from the intuition of modeling the interactive procedure in active learning as distribution matching, by adopting the Wasserstein distance. As a consequence, we derived a new training loss from the theoretical analysis, which is decomposed into optimizing deep neural network parameters and batch query selection through alternative optimization. In addition, the loss for training a deep neural network is naturally formulated as a min-max optimization problem through leveraging the unlabeled data information. Moreover, the proposed principles also indicate an explicit uncertainty-diversity trade-off in the query batch selection. Finally, we evaluate our proposed method on different benchmarks, consistently showing better empirical performances and a better time-efficient query strategy compared to the baselines.

cs.LG↗

arXiv2020

Relational State-Space Model for Stochastic Multi-Object Systems

Real-world dynamical systems often consist of multiple stochastic subsystems that interact with each other. Modeling and forecasting the behavior of such dynamics are generally not easy, due to the inherent hardness in understanding the complicated interactions and evolutions of their constituents. This paper introduces the relational state-space model (R-SSM), a sequential hierarchical latent variable model that makes use of graph neural networks (GNNs) to simulate the joint state transitions of multiple correlated objects. By letting GNNs cooperate with SSM, R-SSM provides a flexible way to incorporate relational information into the modeling of multi-object dynamics. We further suggest augmenting the model with normalizing flows instantiated for vertex-indexed random variables and propose two auxiliary contrastive objectives to facilitate the learning. The utility of R-SSM is empirically evaluated on synthetic and real time-series datasets.

cs.LG↗

arXiv2019

A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction

Reducing communication in training large-scale machine learning applications on distributed platform is still a big challenge. To address this issue, we propose a distributed hierarchical averaging stochastic gradient descent (Hier-AVG) algorithm with infrequent global reduction by introducing local reduction. As a general type of parallel SGD, Hier-AVG can reproduce several popular synchronous parallel SGD variants by adjusting its parameters. We show that Hier-AVG with infrequent global reduction can still achieve standard convergence rate for non-convex optimization problems. In addition, we show that more frequent local averaging with more participants involved can lead to faster training convergence. By comparing Hier-AVG with another popular distributed training algorithm K-AVG, we show that through deploying local averaging with fewer number of global averaging, Hier-AVG can still achieve comparable training speed while frequently get better test accuracy. This indicates that local averaging can serve as an alternative remedy to effectively reduce communication overhead when the number of learners is large. Experimental results of Hier-AVG with several state-of-the-art deep neural nets on CIFAR-10 and IMAGENET-1K are presented to validate our analysis and show its superiority.

cs.LG↗

arXiv2019

Implicit Dual-domain Convolutional Network for Robust Color Image Compression Artifact Reduction

Several dual-domain convolutional neural network-based methods show outstanding performance in reducing image compression artifacts. However, they suffer from handling color images because the compression processes for gray-scale and color images are completely different. Moreover, these methods train a specific model for each compression quality and require multiple models to achieve different compression qualities. To address these problems, we proposed an implicit dual-domain convolutional network (IDCN) with the pixel position labeling map and the quantization tables as inputs. Specifically, we proposed an extractor-corrector framework-based dual-domain correction unit (DCU) as the basic component to formulate the IDCN. A dense block was introduced to improve the performance of extractor in DRU. The implicit dual-domain translation allows the IDCN to handle color images with the discrete cosine transform (DCT)-domain priors. A flexible version of IDCN (IDCN-f) was developed to handle a wide range of compression qualities. Experiments for both objective and subjective evaluations on benchmark datasets show that IDCN is superior to the state-of-the-art methods and IDCN-f exhibits excellent abilities to handle a wide range of compression qualities with little performance sacrifice and demonstrates great potential for practical applications.

cs.CV↗

arXiv2019

Meta-GNN: On Few-shot Node Classification in Graph Meta-learning

Meta-learning has received a tremendous recent attention as a possible approach for mimicking human intelligence, i.e., acquiring new knowledge and skills with little or even no demonstration. Most of the existing meta-learning methods are proposed to tackle few-shot learning problems such as image and text, in rather Euclidean domain. However, there are very few works applying meta-learning to non-Euclidean domains, and the recently proposed graph neural networks (GNNs) models do not perform effectively on graph few-shot learning problems. Towards this, we propose a novel graph meta-learning framework -- Meta-GNN -- to tackle the few-shot node classification problem in graph meta-learning settings. It obtains the prior knowledge of classifiers by training on many similar few-shot learning tasks and then classifies the nodes from new classes with only few labeled samples. Additionally, Meta-GNN is a general model that can be straightforwardly incorporated into any existing state-of-the-art GNN. Our experiments conducted on three benchmark datasets demonstrate that our proposed approach not only improves the node classification performance by a large margin on few-shot learning problems in meta-learning paradigm, but also learns a more general and flexible model for task adaption.

cs.LG↗

arXiv2019

Nonparametric Estimation of Low Rank Matrix Valued Function

Let $A:[0,1]\rightarrow\mathbb{H}_m$ (the space of Hermitian matrices) be a matrix valued function which is low rank with entries in Hölder class $Σ(β,L)$. The goal of this paper is to study statistical estimation of $A$ based on the regression model $\mathbb{E}(Y_j|τ_j,X_j) = \langle A(τ_j), X_j \rangle,$ where $τ_j$ are i.i.d. uniformly distributed in $[0,1]$, $X_j$ are i.i.d. matrix completion sampling matrices, $Y_j$ are independent bounded responses. We propose an innovative nuclear norm penalized local polynomial estimator and establish an upper bound on its point-wise risk measured by Frobenius norm. Then we extend this estimator globally and prove an upper bound on its integrated risk measured by $L_2$-norm. We also propose another new estimator based on bias-reducing kernels to study the case when $A$ is not necessarily low rank and establish an upper bound on its risk measured by $L_{\infty}$-norm. We show that the obtained rates are all optimal up to some logarithmic factor in minimax sense. Finally, we propose an adaptive estimation procedure based on Lepskii's method and model selection with data splitting which is computationally efficient and can be easily implemented and parallelized.

stat.ML↗

arXiv2019

The Sup-norm Perturbation of HOSVD and Low Rank Tensor Denoising

The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces and the low rank tensors are attainable in the Euclidean norm. In this article, we analyze the sup-norm perturbation bounds of HOSVD and introduce estimators of the singular subspaces with sharp deviation bounds in the sup-norm. We also investigate a low rank tensor denoising estimator and demonstrate its fast convergence rate with respect to the entry-wise errors. The sup-norm perturbation bounds reveal unconventional phase transitions for statistical learning applications such as the exact clustering in high dimensional Gaussian mixture model and the exact support recovery in sub-tensor localizations. In addition, the bounds established for HOSVD also elaborate the one-sided sup-norm perturbation bounds for the singular subspaces of unbalanced (or fat) matrices.

math.ST↗

arXiv2018

ORACLE: Optimized Radio clAssification through Convolutional neuraL nEtworks

This paper describes the architecture and performance of ORACLE, an approach for detecting a unique radio from a large pool of bit-similar devices (same hardware, protocol, physical address, MAC ID) using only IQ samples at the physical layer. ORACLE trains a convolutional neural network (CNN) that balances computational time and accuracy, showing 99\% classification accuracy for a 16-node USRP X310 SDR testbed and an external database of $>$100 COTS WiFi devices. Our work makes the following contributions: (i) it studies the hardware-centric features within the transmitter chain that causes IQ sample variations; (ii) for an idealized static channel environment, it proposes a CNN architecture requiring only raw IQ samples accessible at the front-end, without channel estimation or prior knowledge of the communication protocol; (iii) for dynamic channels, it demonstrates a principled method of feedback-driven transmitter-side modifications that uses channel estimation at the receiver to increase differentiability for the CNN classifier. The key innovation here is to intentionally introduce controlled imperfections on the transmitter side through software directives, while minimizing the change in bit error rate. Unlike previous work that imposes constant environmental conditions, ORACLE adopts the `train once deploy anywhere' paradigm with near-perfect device classification accuracy.

eess.SP↗

arXiv2018

The Voronoi formula on GL(3) with ramification

Firstly we prove that the Voronoi formula of Miller-Schmid type applies to automorphic forms on GL(3) for the congruence subgroup $Γ_0(N)$, when the conductor of the additive character in the formula is a multiple of $N$. As an application, we produce a result about the functional equation of $L$-function of the automorphic form on GL(3) twisted by Dirichlet characters. Secondly we prove that a similar formula applies to automorphic forms on GL(3) for the congruence subgroup $Γ_0(N)$, when the conductor of the additive character in the formula is coprime with $N$.

math.NT↗

arXiv2018

On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization

Despite their popularity, the practical performance of asynchronous stochastic gradient descent methods (ASGD) for solving large scale machine learning problems are not as good as theoretical results indicate. We adopt and analyze a synchronous K-step averaging stochastic gradient descent algorithm which we call K-AVG. We establish the convergence results of K-AVG for nonconvex objectives and explain why the K-step delay is necessary and leads to better performance than traditional parallel stochastic gradient descent which is a special case of K-AVG with $K=1$. We also show that K-AVG scales better than ASGD. Another advantage of K-AVG over ASGD is that it allows larger stepsizes. On a cluster of $128$ GPUs, K-AVG is faster than ASGD implementations and achieves better accuracies and faster convergence for \cifar dataset.

cs.LG↗

arXiv2017

The balanced Voronoi formulas for GL(n)

In this paper we show how the GL(N) Voronoi summation formula of [MiSc2] can be rewritten to incorporate hyper-Kloosterman sums of various dimensions on both sides. This generalizes a formula for GL(4) with ordinary Kloosterman sums on both sides that was considered by Xiaoqing Li and the first-named author, and later by the second-named author in [Zho].

math.NT↗

arXiv2016

Plancherel distribution of Satake parameters of Maass cusp forms on $GL_3$

We prove an equidistribution result for the Satake parameters of Maass cusp forms on $GL_3$ with respect to the $p$-adic Plancherel measure by using an application of the Kuznetsov trace formula. The techniques developed in this paper deal with the removal of arithmetic weight $L(1,F,Ad)^{-1}$ in the Kuznetsov trace formula on $GL_3$.

math.NT↗

arXiv2016

The LAMOST spectroscopic survey of star clusters in M31. II. Metallicities, ages and masses

We select from Paper I a sample of 306 massive star clusters observed with the Large Sky Area Multi-Object Fibre Spectroscopic Telescope (LAMOST) in the vicinity fields of M31 and M33 and determine their metallicities, ages and masses. Metallicities and ages are estimated by fitting the observed integrated spectra with stellar synthesis population (SSP) models with a pixel-to-pixel spectral fitting technique. Ages for most young clusters are also derived by fitting the multi-band photometric measurements with model spectral energy distributions (SEDs). The estimated cluster ages span a wide range, from several million years to the age of the universe. The numbers of clusters younger and older than 1 Gyr are respectively 46 and 260. With ages and metallicities determined, cluster masses are then estimated by comparing the multi-band photometric measurements with SSP model SEDs. The derived masses range from $\sim 10^{3}$ to $\sim 10^7$ $M_{\odot}$, peaking at $\sim 10^{4.3}$ and $\sim 10^{5.7}$ $M_{\odot}$ for young ($< 1$ Gyr) and old ($>1$ Gyr) clusters, respectively. Our estimated metallicities, ages and masses are in good agreement with available literature values. Old clusters richer than [Fe/H] $\sim -0.7$ dex have a wide range of ages. Those poorer than [Fe/H] $\sim -0.7$ dex seem to be composed of two groups, as previously found for Galactic GCs -- one of the oldest ages with all values of metallicity down to $\sim -2$ dex and another with metallicity increasing with decreasing age. The old clusters in the inner disk of M\,31 (0 -- 30 kpc) show a clear metallicity gradient measured at $-0.038\pm0.023$ dex/kpc.

astro-ph.SR↗

arXiv2016

The Voronoi formula and double Dirichlet series

We prove a Voronoi formula for coefficients of a large class of $L$-functions including Maass cusp forms, Rankin-Selberg convolutions, and certain isobaric sums. Our proof is based on the functional equations of $L$-functions twisted by Dirichlet characters and does not directly depend on automorphy. Hence it has wider application than previous proofs. The key ingredient is the construction of a double Dirichlet series.

math.NT↗

arXiv2015

Ultrafast All-optical Modulation Exploiting the Vibrational Dynamic of Metallic Meta-atoms

Optical control over elementary molecular vibration establishes fundamental capabilities for exploiting the broad range of optical linear and nonlinear phenomena. However, experimental demonstration of the coherently driven molecular vibration remains a challenge task due to the weak optical force imposed on natural materials. Here we report the design of "meta-atom" that exhibits giant artificial optical nonlinearity. These "meta-atoms" support co-localized magnetic resonance at optical frequency and vibration resonance at GHz frequency with a deep-sub-diffraction-limit spatial confinement ($λ^2/100$). The coherent coupling of those two distinct resonances manifests a strong optical force, which is fundamentally different from the commonly studied form of radiation forces, the gradient forces, or photo-thermal induced deformation. It results in a giant third-order susceptibility $χ^{(3)}$ of $10^{-13}$ $m^2$/$V^2$, which is more than six orders of magnitude larger than that found in natural materials. The all-optical modulation at the frequency well above 1 GHz has thus been demonstrated experimentally.

physics.optics↗

arXiv2015

Voronoi Summation Formulae on GL(n)

We discover new Voronoi formulae for automorphic forms on GL($n$) for $n\geq 4$. There are $[n/2]$ different Voronoi formulae on GL($n$), which are Poisson summation formulae weighted by Fourier coefficients of the automorphic form with twists by some hyper-Kloosterman sums.

math.NT↗