arXiv subjects

Gang Wang

Publications and source records attributed to Gang Wang.

At least 433 records · Page 24Linked to original sources

arXiv2017

Randomized Block Frank-Wolfe for Convergent Large-Scale Learning

Owing to their low-complexity iterations, Frank-Wolfe (FW) solvers are well suited for various large-scale learning tasks. When block-separable constraints are present, randomized block FW (RB-FW) has been shown to further reduce complexity by updating only a fraction of coordinate blocks per iteration. To circumvent the limitations of existing methods, the present work develops step sizes for RB-FW that enable a flexible selection of the number of blocks to update per iteration while ensuring convergence and feasibility of the iterates. To this end, convergence rates of RB-FW are established through computational bounds on a primal sub-optimality measure and on the duality gap. The novel bounds extend the existing convergence analysis, which only applies to a step-size sequence that does not generally lead to feasible iterates. Furthermore, two classes of step-size sequences that guarantee feasibility of the iterates are also proposed to enhance flexibility in choosing decay rates. The novel convergence results are markedly broadened to encompass also nonconvex objectives, and further assert that RB-FW with exact line-search reaches a stationary point at rate $\mathcal{O}(1/\sqrt{t})$. Performance of RB-FW with different step sizes and number of blocks is demonstrated in two applications, namely charging of electrical vehicles and structural support vector machines. Extensive simulated tests demonstrate the performance improvement of RB-FW relative to existing randomized single-block FW methods.

math.OC↗

arXiv2017

DelugeNets: Deep Networks with Efficient and Flexible Cross-layer Information Inflows

Deluge Networks (DelugeNets) are deep neural networks which efficiently facilitate massive cross-layer information inflows from preceding layers to succeeding layers. The connections between layers in DelugeNets are established through cross-layer depthwise convolutional layers with learnable filters, acting as a flexible yet efficient selection mechanism. DelugeNets can propagate information across many layers with greater flexibility and utilize network parameters more effectively compared to ResNets, whilst being more efficient than DenseNets. Remarkably, a DelugeNet model with just model complexity of 4.31 GigaFLOPs and 20.2M network parameters, achieve classification errors of 3.76% and 19.02% on CIFAR-10 and CIFAR-100 dataset respectively. Moreover, DelugeNet-122 performs competitively to ResNet-200 on ImageNet dataset, despite costing merely half of the computations needed by the latter.

cs.CV↗

arXiv2017

Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow

This paper presents a new algorithm, termed \emph{truncated amplitude flow} (TAF), to recover an unknown vector $\bm{x}$ from a system of quadratic equations of the form $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, where $\bm{a}_i$'s are given random measurement vectors. This problem is known to be \emph{NP-hard} in general. We prove that as soon as the number of equations is on the order of the number of unknowns, TAF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with both the number of unknowns and the number of equations. Our TAF approach adopts the \emph{amplitude-based} empirical loss function, and proceeds in two stages. In the first stage, we introduce an \emph{orthogonality-promoting} initialization that can be obtained with a few power iterations. Stage two refines the initial estimate by successive updates of scalable \emph{truncated generalized gradient iterations}, which are able to handle the rather challenging nonconvex and nonsmooth amplitude-based objective function. In particular, when vectors $\bm{x}$ and $\bm{a}_i$'s are real-valued, our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version. Numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations. Furthermore, even under the same initialization, the proposed amplitude-based refinement outperforms existing Wirtinger flow variants, corroborating the superior performance of TAF over state-of-the-art algorithms.

stat.ML↗

arXiv2017

PSSE Redux: Convex Relaxation, Decentralized, Robust, and Dynamic Approaches

This chapter aspires to glean some of the recent advances in power system state estimation (PSSE), though our collection is not exhaustive by any means. The Cram{é}r-Rao bound, a lower bound on the (co)variance of any unbiased estimator, is first derived for the PSSE setup. After reviewing the classical Gauss-Newton iterations, contemporary PSSE solvers leveraging relaxations to convex programs and successive convex approximations are explored. A disciplined paradigm for distributed and decentralized schemes is subsequently exemplified under linear(ized) and exact grid models. Novel bad data processing models and fresh perspectives linking critical measurements to cyber-attacks on the state estimator are presented. Finally, spurred by advances in online convex optimization, model-free and model-based state trackers are reviewed.

eess.SY↗

arXiv2017

An Empirical Study of Language CNN for Image Captioning

Language Models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a Language CNN model which is suitable for statistical language modeling tasks and shows competitive performance in image captioning. In contrast to previous models which predict next word based on one previous word and hidden state, our language CNN is fed with all the previous words and can model the long-range dependencies of history words, which are critical for image captioning. The effectiveness of our approach is validated on two datasets MS COCO and Flickr30K. Our extensive experimental results show that our method outperforms the vanilla recurrent neural network based language models and is competitive with the state-of-the-art methods.

cs.CV↗

arXiv2017

A Novel Neural Network Model Specified for Representing Logical Relations

With computers to handle more and more complicated things in variable environments, it becomes an urgent requirement that the artificial intelligence has the ability of automatic judging and deciding according to numerous specific conditions so as to deal with the complicated and variable cases. ANNs inspired by brain is a good candidate. However, most of current numeric ANNs are not good at representing logical relations because these models still try to represent logical relations in the form of ratio based on functional approximation. On the other hand, researchers have been trying to design novel neural network models to make neural network model represent logical relations. In this work, a novel neural network model specified for representing logical relations is proposed and applied. New neurons and multiple kinds of links are defined. Inhibitory links are introduced besides exciting links. Different from current numeric ANNs, one end of an inhibitory link connects an exciting link rather than a neuron. Inhibitory links inhibit the connected exciting links conditionally to make this neural network model represent logical relations correctly. This model can simulate the operations of Boolean logic gates, and construct complex logical relations with the advantages of simpler neural network structures than recent works in this area. This work provides some ideas to make neural networks represent logical relations more directly and efficiently, and the model could be used as the complement to current numeric ANN to deal with logical issues and expand the application areas of ANN.

cs.NE↗

arXiv2017

Beyond Forward Shortcuts: Fully Convolutional Master-Slave Networks (MSNets) with Backward Skip Connections for Semantic Segmentation

Recent deep CNNs contain forward shortcut connections; i.e. skip connections from low to high layers. Reusing features from lower layers that have higher resolution (location information) benefit higher layers to recover lost details and mitigate information degradation. However, during inference the lower layers do not know about high layer features, although they contain contextual high semantics that benefit low layers to adaptively extract informative features for later layers. In this paper, we study the influence of backward skip connections which are in the opposite direction to forward shortcuts, i.e. paths from high layers to low layers. To achieve this -- which indeed runs counter to the nature of feed-forward networks -- we propose a new fully convolutional model that consists of a pair of networks. A `Slave' network is dedicated to provide the backward connections from its top layers to the `Master' network's bottom layers. The Master network is used to produce the final label predictions. In our experiments we validate the proposed FCN model on ADE20K (ImageNet scene parsing), PASCAL-Context, and PASCAL VOC 2011 datasets.

cs.CV↗

arXiv2017

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.

cs.CV↗

arXiv2017

Empirical Analysis of Password Reuse and Modification across Online Service

Leaked passwords from data breaches can pose a serious threat to users if the password is reused elsewhere. With more online services getting breached today, there is still a lack of large-scale quantitative understanding of the risks of password reuse across services. In this paper, we analyze a large collection of 28.8 million users and their 61.5 million passwords across 107 services. We find that 38% of the users have reused exactly the same password across different sites, while 20% have modified an existing password to create new ones. In addition, we find that the password modification patterns are highly consistent across different user demographics, indicating a high predictability. To quantify the risk, we build a new training-based guessing algorithm, and show that more than 16 million password pairs can be cracked within just 10 attempts (30% of the modified passwords and all the reused passwords).

cs.CR↗

arXiv2017

Reinforcement Learning for Learning Rate Control

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. We propose an algorithm to automatically learn learning rates using neural network based actor-critic methods from deep reinforcement learning (RL).In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made. The introduction of auxiliary actor and critic networks helps the main network achieve better performance. Experiments on different datasets and network architectures show that our approach leads to better convergence of SGD than human-designed competitors.

cs.LG↗

arXiv2017

Solving Almost all Systems of Random Quadratic Equations

This paper deals with finding an $n$-dimensional solution $x$ to a system of quadratic equations of the form $y_i=|\langle{a}_i,x\rangle|^2$ for $1\le i \le m$, which is also known as phase retrieval and is NP-hard in general. We put forth a novel procedure for minimizing the amplitude-based least-squares empirical loss, that starts with a weighted maximal correlation initialization obtainable with a few power or Lanczos iterations, followed by successive refinements based upon a sequence of iteratively reweighted (generalized) gradient iterations. The two (both the initialization and gradient flow) stages distinguish themselves from prior contributions by the inclusion of a fresh (re)weighting regularization technique. The overall algorithm is conceptually simple, numerically scalable, and easy-to-implement. For certain random measurement models, the novel procedure is shown capable of finding the true solution $x$ in time proportional to reading the data $\{(a_i;y_i)\}_{1\le i \le m}$. This holds with high probability and without extra assumption on the signal $x$ to be recovered, provided that the number $m$ of equations is some constant $c>0$ times the number $n$ of unknowns in the signal vector, namely, $m>cn$. Empirically, the upshots of this contribution are: i) (almost) $100\%$ perfect signal recovery in the high-dimensional (say e.g., $n\ge 2,000$) regime given only an information-theoretic limit number of noiseless equations, namely, $m=2n-1$ in the real-valued Gaussian case; and, ii) (nearly) optimal statistical accuracy in the presence of additive noise of bounded support. Finally, substantial numerical tests using both synthetic data and real images corroborate markedly improved signal recovery performance and computational efficiency of our novel procedure relative to state-of-the-art approaches.

math.OC↗

arXiv2017

Ultra-broadband photodetection of Weyl semimetal TaAs up to infrared 10 μm range at room temperature

Photodetectors with broadband optical response have promising applications in many advanced optoelectronic and photonic devices. Especially, those with the detection range up to mid-infrared at room temperature are very challenging and highly desired. Recently, Weyl semimetal has been discovered and proposed to be favorable for photodetection since in general it breaks Lorentz invariance to have tilted chiral Weyl cones around Fermi level, which leads to chirality dependent photocurrents at arbitrarily long wavelength. Furthermore, the linear dispersion bands in Weyl cones result in very high carrier mobility and much reduced thermal carrier compared with the parabolic ones in narrow-gap semiconductors. Here, we report that the Weyl semimetal TaAs based photodetector can operate at room temperature with spectral range from blue (438.5 nm) to mid-infrared (10.29 μm) light wavelengths and the responsibility and detectivity is more than 78 uA W-1 and 1.88*107 Jones, respectively. This is the first photodetector made by a Weyl semimetal and shows its promising in room-temperature mid-infrared photodetection.

cond-mat.mtrl-sci↗

arXiv2017

Synthesis of Highly Anisotropic Semiconducting GaTe Nanomaterials and Emerging Properties Enabled by Epitaxy

Pseudo-one dimensional (pseudo-1D) materials are a new-class of materials where atoms are arranged in chain like structures in two-dimensions (2D). Examples include recently discovered black phosphorus, ReS2 and ReSe2 from transition metal dichalcogenides, TiS3 and ZrS3 from transition metal trichalcogenides and most recently GaTe. The presence of structural anisotropy impacts their physical properties and leads to direction dependent light-matter interactions, dichroic optical responses, high mobility channels, and anisotropic thermal conduction. Despite the numerous reports on the vapor phase growth of isotropic TMDCs and post transition metal chalcogenides such as MoS2 and GaSe, the synthesis of pseudo-1D materials is particularly difficult due to the anisotropy in interfacial energy, which stabilizes dendritic growth rather than single crystalline growth with well-defined orientation. The growth of monoclinic GaTe has been demonstrated on flexible mica substrates with superior photodetecting performance. In this work, we demonstrate that pseudo-1D monoclinic GaTe layers can be synthesized on a variety of other substrates including GaAs (111), Si (111) and c-cut sapphire by physical vapor transport techniques. High resolution transmission electron microscopy (HRTEM) measurements, together with angle resolved micro-PL and micro-Raman techniques, provide for the very first time atomic scale resolution experiments on pseudo-1D structures in monoclinic GaTe and anisotropic properties. Interestingly, GaTe nanomaterials grown on sapphire exhibit highly efficient and narrow localized emission peaks below the band gap energy, which are found to be related to select types of line and point defects as evidenced by PL and Raman mapping scans. It makes the samples grown on sapphire more prominent than those grown on GaAs and Si, which demonstrate more regular properties.

cond-mat.mtrl-sci↗

arXiv2017

Power System State Estimation via Feasible Point Pursuit: Algorithms and Cramer-Rao Bound

Accurately monitoring the system's operating point is central to the reliable and economic operation of an electric power grid. Power system state estimation (PSSE) aims to obtain complete voltage magnitude and angle information at each bus given a number of system variables at selected buses and lines. Power flow analysis is a special case of PSSE, and amounts to solving a set of noise-free power flow equations. Physical laws dictate quadratic relationships between available quantities and unknown voltages, rendering general instances of power flow and PSSE nonconvex and NP-hard. Past approaches are largely based on gradient-type iterative procedures or semidefinite relaxation (SDR). Due to nonconvexity, the solution obtained via gradient-type schemes depends on initialization, while SDR methods do not perform as desired in challenging scenarios. This paper puts forth novel \emph{feasible point pursuit} (FPP)-based solvers for power flow and PSSE, which iteratively seek feasible solutions for a nonconvex quadratically constrained quadratic programming (QCQP) reformulation of the weighted least-squares (WLS) problem. Relative to the prior art, the developed solvers offer superior performance at the cost of higher complexity. Furthermore, they converge to a stationary point of the WLS problem. As a baseline for comparing different estimators, the Cram{\' e}r-Rao lower bound (CRLB) is derived for the fundamental PSSE problem in this paper. Judicious numerical tests on several IEEE benchmark systems showcase markedly improved performance of our FPP-based solvers for both power flow and PSSE tasks over popular WLS-based Gauss-Newton iterations and SDR approaches.

eess.SY↗

arXiv2017

Sky Subtraction for LAMOST

Sky subtraction is the key technique in data reduction of multi-fiber spectra. Knowledge of the related instrument character is necessary to determine the method adopted in sky subtraction. In this study, we described the sky subtraction method designed for LAMOST(Large sky Area Multi-Object fiber Spectroscopic Telescope) survey. The method has been intergrated into LAMOST 2D Pipeline v2.6 and applied to data of LAMOST DR3 and later. For LAMOST, sky emission line calibration is used to alleviate the position-dependent (thus time-dependent) ~4% fiber throughput uncertainty and the small wavelength instability (0.1Å) during observation. PCA (Principal Component Analysis) sky subtraction further reduces 25% of the sky line residual of the OH lines in the red part of the LAMOST spectra after the mater sky spectrum, which is derived from a B-spline fit of 20 sky fibers in each spectrograph, is adjusted by sky emission line and subtracted from each fiber. Further analysis shows that our wavelength calibration accuracy is about 4.5km/s, and the average sky subtraction residuals are about 3% for sky emission lines and 3% for continuum region. The relative sky subtraction residuals vary with the moon light background brightness, could reach as low as 1.5% for the sky emission line regions in the dark night. Tests on the F stars of both similar sky emission line strength and similar object continuum intensity show that the sky emission line residual of LAMOST is smaller than those of SDSS survey.

astro-ph.IM↗

arXiv2017

Towards Monetary Incentives in Social Q&A Services

Community-based question answering (CQA) services are facing key challenges to motivate domain experts to provide timely answers. Recently, CQA services are exploring new incentive models to engage experts and celebrities by allowing them to set a price on their answers. In this paper, we perform a data-driven analysis on two emerging payment-based CQA systems: Fenda (China) and Whale (US). By analyzing a large dataset of 220K questions (worth 1 million USD collectively), we examine how monetary incentives affect different players in the system. We find that, while monetary incentive enables quick answers from experts, it also drives certain users to aggressively game the system for profits. In addition, in this supplier-driven marketplace, users need to proactively adjust their price to make profits. Famous people are unwilling to lower their price, which in turn hurts their income and engagement over time. Finally, we discuss the key implications to future CQA design.

cs.AI↗

arXiv2017

Chiral magnetic effect in isobaric collisions

We give a numerical simulation of the generation of the magnetic field and the charge-separation signal due to the chiral magnetic effect (CME) --- the induction of an electric current by the magnetic field in a parity-odd matter --- in the collisions of isobaric nuclei, $^{96}_{44}$Ru + $^{96}_{44}$Ru and $^{96}_{40}$Zr + $^{96}_{40}$Zr, at $\sqrt{s_{\rm NN}}=200$ GeV. We show that such collisions provide an ideal tool to disentangle the CME signal from the possible elliptic-flow driven background effects. We also discuss some other effects that can be tested by using the isobaric collisions.

nucl-th↗

arXiv2017

Crowdsourcing Cybersecurity: Cyber Attack Detection using Social Media

Social media is often viewed as a sensor into various societal events such as disease outbreaks, protests, and elections. We describe the use of social media as a crowdsourced sensor to gain insight into ongoing cyber-attacks. Our approach detects a broad range of cyber-attacks (e.g., distributed denial of service (DDOS) attacks, data breaches, and account hijacking) in an unsupervised manner using just a limited fixed set of seed event triggers. A new query expansion strategy based on convolutional kernels and dependency parses helps model reporting structure and aids in identifying key event characteristics. Through a large-scale analysis over Twitter, we demonstrate that our approach consistently identifies and encodes events, outperforming existing methods.

cs.CR↗