arXiv subjects

Zhan Gao

Publications and source records attributed to Zhan Gao.

48 records · Page 3Linked to original sources

arXiv2021

Stability of Graph Convolutional Neural Networks to Stochastic Perturbations

Graph convolutional neural networks (GCNNs) are nonlinear processing tools to learn representations from network data. A key property of GCNNs is their stability to graph perturbations. Current analysis considers deterministic perturbations but fails to provide relevant insights when topological changes are random. This paper investigates the stability of GCNNs to stochastic graph perturbations induced by link losses. In particular, it proves the expected output difference between the GCNN over random perturbed graphs and the GCNN over the nominal graph is upper bounded by a factor that is linear in the link loss probability. We perform the stability analysis in the graph spectral domain such that the result holds uniformly for any graph. This result also shows the role of the nonlinearity and the architecture width and depth, and allows identifying handle to improve the GCNN robustness. Numerical simulations on source localization and robot swarm control corroborate our theoretical findings.

cs.LG↗

arXiv2021

Stochastic Graph Neural Networks

Graph neural networks (GNNs) model nonlinear representations in graph data with applications in distributed agent coordination, control, and planning among others. Current GNN architectures assume ideal scenarios and ignore link fluctuations that occur due to environment, human factors, or external attacks. In these situations, the GNN fails to address its distributed task if the topological randomness is not considered accordingly. To overcome this issue, we put forth the stochastic graph neural network (SGNN) model: a GNN where the distributed graph convolution module accounts for the random network changes. Since stochasticity brings in a new learning paradigm, we conduct a statistical analysis on the SGNN output variance to identify conditions the learned filters should satisfy for achieving robust transference to perturbed scenarios, ultimately revealing the explicit impact of random link losses. We further develop a stochastic gradient descent (SGD) based learning process for the SGNN and derive conditions on the learning rate under which this learning process converges to a stationary point. Numerical results corroborate our theoretical findings and compare the benefits of SGNN robust transference with a conventional GNN that ignores graph perturbations during learning.

eess.SP↗

arXiv2021

Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping

Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data. Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data, referred to as over-fitting. This paper proposes Topology Adaptive Edge Dropping (TADropEdge) method as an adaptive data augmentation technique to improve generalization performance and learn robust GNN models. We start by explicitly analyzing how random edge dropping increases the data diversity during training, while indicating i.i.d. edge dropping does not account for graph structural information and could result in noisy augmented data degrading performance. To overcome this issue, we consider graph connectivity as the key property that captures graph topology. TADropEdge incorporates this factor into random edge dropping such that the edge-dropped subgraphs maintain similar topology as the underlying graph, yielding more satisfactory data augmentation. In particular, TADropEdge first leverages the graph spectrum to assign proper weights to graph edges, which represent their criticality for establishing the graph connectivity. It then normalizes the edge weights and drops graph edges adaptively based on their normalized weights. Besides improving generalization performance, TADropEdge reduces variance for efficient training and can be applied as a generic method modular to different GNN models. Intensive experiments on real-life and synthetic datasets corroborate theory and verify the effectiveness of the proposed method.

cs.LG↗

arXiv2021

Resource Allocation via Model-Free Deep Learning in Free Space Optical Communications

This paper investigates the general problem of resource allocation for mitigating channel fading effects in Free Space Optical (FSO) communications. The resource allocation problem is modeled as the constrained stochastic optimization framework, which covers a variety of FSO scenarios involving power adaptation, relay selection and their joint allocation. Under this framework, we propose two algorithms that solve FSO resource allocation problems. We first present the Stochastic Dual Gradient (SDG) algorithm that is shown to solve the problem exactly by exploiting the strong duality but whose implementation necessarily requires explicit and accurate system models. As an alternative we present the Primal-Dual Deep Learning (PDDL) algorithm based on the SDG algorithm, which parametrizes the resource allocation policy with Deep Neural Networks (DNNs) and optimizes the latter via a primal-dual method. The parametrized resource allocation problem incurs only a small loss of optimality due to the strong representational power of DNNs, and can be moreover implemented without knowledge of system models. A wide set of numerical experiments are performed to corroborate the proposed algorithms in FSO resource allocation problems. We demonstrate their superior performance and computational efficiency compared to the baseline methods in both continuous power allocation and binary relay selection settings.

eess.SP↗

arXiv2021

Spherical Convolutional Neural Networks: Stability to Perturbations in SO(3)

Spherical convolutional neural networks (Spherical CNNs) learn nonlinear representations from 3D data by exploiting the data structure and have shown promising performance in shape analysis, object classification, and planning among others. This paper investigates the properties that Spherical CNNs exhibit as they pertain to the rotational structure inherent in spherical signals. We build upon the rotation equivariance of spherical convolutions to show that Spherical CNNs are stable to general structure perturbations. In particular, we model arbitrary structure perturbations as diffeomorphism perturbations, and define the rotation distance that measures how far from rotations these perturbations are. We prove that the output change of a Spherical CNN induced by the diffeomorphism perturbation is bounded proportionally by the perturbation size under the rotation distance. This stability property coupled with the rotation equivariance provide theoretical guarantees that underpin the practical observations that Spherical CNNs exploit the rotational structure, maintain performance under structure perturbations that are close to rotations, and offer good generalization and faster learning.

cs.LG↗

arXiv2021

On LASSO for Predictive Regression

Explanatory variables in a predictive regression typically exhibit low signal strength and various degrees of persistence. Variable selection in such a context is of great importance. In this paper, we explore the pitfalls and possibilities of the LASSO methods in this predictive regression framework. In the presence of stationary, local unit root, and cointegrated predictors, we show that the adaptive LASSO cannot asymptotically eliminate all cointegrating variables with zero regression coefficients. This new finding motivates a novel post-selection adaptive LASSO, which we call the twin adaptive LASSO (TAlasso), to restore variable selection consistency. Accommodating the system of heterogeneous regressors, TAlasso achieves the well-known oracle property. In contrast, conventional LASSO fails to attain coefficient estimation consistency and variable screening in all components simultaneously. We apply these LASSO methods to evaluate the short- and long-horizon predictability of S\&P 500 excess returns.

econ.EM↗

arXiv2020

Wide and Deep Graph Neural Networks with Distributed Online Learning

Graph neural networks (GNNs) learn representations from network data with naturally distributed architectures, rendering them well-suited candidates for decentralized learning. Oftentimes, this decentralized graph support changes with time due to link failures or topology variations. These changes create a mismatch between the graphs on which GNNs were trained and the ones on which they are tested. Online learning can be used to retrain GNNs at testing time, overcoming this issue. However, most online algorithms are centralized and work on convex problems (which GNNs rarely lead to). This paper proposes the Wide and Deep GNN (WD-GNN), a novel architecture that can be easily updated with distributed online learning mechanisms. The WD-GNN comprises two components: the wide part is a bank of linear graph filters and the deep part is a GNN. At training time, the joint architecture learns a nonlinear representation from data. At testing time, the deep part (nonlinear) is left unchanged, while the wide part is retrained online, leading to a convex problem. We derive convergence guarantees for this online retraining procedure and further propose a decentralized alternative. Experiments on the robot swarm control for flocking corroborate theory and show potential of the proposed architecture for distributed online learning.

cs.LG↗

arXiv2020

Balancing Rates and Variance via Adaptive Batch-Size for Stochastic Optimization Problems

Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is required for exact asymptotic convergence with the fact that constant step-size learns faster in finite time up to an error. To do so, rather than fixing the mini-batch and the step-size at the outset, we propose a strategy to allow parameters to evolve adaptively. Specifically, the batch-size is set to be a piecewise-constant increasing sequence where the increase occurs when a suitable error criterion is satisfied. Moreover, the step-size is selected as that which yields the fastest convergence. The overall algorithm, two scale adaptive (TSA) scheme, is developed for both convex and non-convex stochastic optimization problems. It inherits the exact asymptotic convergence of stochastic gradient method. More importantly, the optimal error decreasing rate is achieved theoretically, as well as an overall reduction in computational cost. Experimentally, we observe that TSA attains a favorable tradeoff relative to standard SGD that fixes the mini-batch and the step-size, or simply allowing one to increase or decrease respectively.

eess.SP↗

arXiv2020

Resource Allocation via Graph Neural Networks in Free Space Optical Fronthaul Networks

This paper investigates the optimal resource allocation in free space optical (FSO) fronthaul networks. The optimal allocation maximizes an average weighted sum-capacity subject to power limitation and data congestion constraints. Both adaptive power assignment and node selection are considered based on the instantaneous channel state information (CSI) of the links. By parameterizing the resource allocation policy, we formulate the problem as an unsupervised statistical learning problem. We consider the graph neural network (GNN) for the policy parameterization to exploit the FSO network structure with small-scale training parameters. The GNN is shown to retain the permutation equivariance that matches with the permutation equivariance of resource allocation policy in networks. The primal-dual learning algorithm is developed to train the GNN in a model-free manner, where the knowledge of system models is not required. Numerical simulations present the strong performance of the GNN relative to a baseline policy with equal power assignment and random node selection.

eess.SP↗

arXiv2019

Implementing Convex Optimization in R: Two Econometric Examples

Economists specify high-dimensional models to address heterogeneity in empirical studies with complex big data. Estimation of these models calls for optimization techniques to handle a large number of parameters. Convex problems can be effectively executed in modern statistical programming languages. We complement Koenker and Mizera (2014)'s work on numerical implementation of convex optimization, with focus on high-dimensional econometric estimators. Combining R and the convex solver MOSEK achieves faster speed and equivalent accuracy, demonstrated by examples from Su, Shi, and Phillips (2016) and Shi (2016). Robust performance of convex optimization is witnessed cross platforms. The convenience and reliability of convex optimization in R make it easy to turn new ideas into prototypes.

stat.CO↗

arXiv2019

Optimal WDM Power Allocation via Deep Learning for Radio on Free Space Optics Systems

Radio on Free Space Optics (RoFSO), as a universal platform for heterogeneous wireless services, is able to transmit multiple radio frequency signals at high rates in free space optical networks. This paper investigates the optimal design of power allocation for Wavelength Division Multiplexing (WDM) transmission in RoFSO systems. The proposed problem is a weighted total capacity maximization problem with two constraints of total power limitation and eye safety concern. The model-based Stochastic Dual Gradient algorithm is presented first, which solves the problem exactly by exploiting the null duality gap. The model-free Primal-Dual Deep Learning algorithm is then developed to learn and optimize the power allocation policy with Deep Neural Network (DNN) parametrization, which can be utilized without any knowledge of system models. Numerical simulations are performed to exhibit significant performance of our algorithms compared to the average equal power allocation.

eess.SP↗

arXiv2018

Overall momentum balance and redistribution of the lost energy in asymmetric dijet events in 2.76~ATeV Pb-Pb collisions with a multi-phase transport model

The overall transverse momentum balance and the redistribution of the lost energy from hard jets for asymmetric dijet events in PbPb collisions at 2.76~ATeV at the LHC is studied within A Multi-Phase Transport (AMPT) model. A detailed analysis is performed for the projected transverse momentum $\langle \slashed{p}_{T}^{||} \rangle$ contributed from the final charged hadrons carrying different transverse momenta and emitted from different angular directions. We find that the transverse momentum projection $\langle \slashed{p}_{T}^{||} \rangle $ in the leading jet direction is mainly contributed by hard hadrons ($p_T > 8.0$~GeV/$c$) in both peripheral and central PbPb collisions, while the opposite direction in central collisions is dominated by soft hadrons ($p_T = 0.5$-$2.0$~GeV/$c$). The study of in-cone and out-of-cone contributions to $\langle \slashed{p}_{T}^{||} \rangle$ shows that these soft hadrons are mostly emitted at large angles away from the dijet axis. Our AMPT calculation is in qualitative agreement with the CMS measurements and the primary mechanism for the energy transported to large angles in the AMPT model is the elastic scattering at the partonic stage. Future studies including also inelastic processes should be helpful in understanding the overestimation of the magnitudes of in-cone and out-of-cone imbalances from our AMPT calculations, and shed light on different roles played by radiative and collisional processes in the redistribution of the lost energy from hard jets.

hep-ph↗