Searcharxiv⌕ Search

arXiv subjects

Peng Xu

Publications and source records attributed to Peng Xu.

At least 379 records · Page 21Linked to original sources
arXiv2019

Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information

We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve $ ε$-approximate second-order optimality which have shown to be tight. Our Hessian approximation conditions constitute a major relaxation over the existing ones in the literature. Consequently, we are able to show that such mild conditions allow for the construction of the approximate Hessian through various random sampling methods. In this light, we consider the canonical problem of finite-sum minimization, provide appropriate uniform and non-uniform sub-sampling strategies to construct such Hessian approximations, and obtain optimal iteration complexity for the corresponding sub-sampled trust-region and cubic regularization methods.

math.OC↗
arXiv2019

Deep Zero-Shot Learning for Scene Sketch

We introduce a novel problem of scene sketch zero-shot learning (SSZSL), which is a challenging task, since (i) different from photo, the gap between common semantic domain (e.g., word vector) and sketch is too huge to exploit common semantic knowledge as the bridge for knowledge transfer, and (ii) compared with single-object sketch, more expressive feature representation for scene sketch is required to accommodate its high-level of abstraction and complexity. To overcome these challenges, we propose a deep embedding model for scene sketch zero-shot learning. In particular, we propose the augmented semantic vector to conduct domain alignment by fusing multi-modal semantic knowledge (e.g., cartoon image, natural image, text description), and adopt attention-based network for scene sketch feature learning. Moreover, we propose a novel distance metric to improve the similarity measure during testing. Extensive experiments and ablation studies demonstrate the benefit of our sketch-specific design.

cs.CV↗
arXiv2019

Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction

Knowledge Bases (KBs) require constant up-dating to reflect changes to the world they represent. For general purpose KBs, this is often done through Relation Extraction (RE), the task of predicting KB relations expressed in text mentioning entities known to the KB. One way to improve RE is to use KB Embeddings (KBE) for link prediction. However, despite clear connections between RE and KBE, little has been done toward properly unifying these models systematically. We help close the gap with a framework that unifies the learning of RE and KBE models leading to significant improvements over the state-of-the-art in RE. The code is available at https://github.com/billy-inn/HRERE.

cs.CL↗
arXiv2019

Concise Fuzzy System Modeling Integrating Soft Subspace Clustering and Sparse Learning

The superior interpretability and uncertainty modeling ability of Takagi-Sugeno-Kang fuzzy system (TSK FS) make it possible to describe complex nonlinear systems intuitively and efficiently. However, classical TSK FS usually adopts the whole feature space of the data for model construction, which can result in lengthy rules for high-dimensional data and lead to degeneration in interpretability. Furthermore, for highly nonlinear modeling task, it is usually necessary to use a large number of rules which further weakens the clarity and interpretability of TSK FS. To address these issues, a concise zero-order TSK FS construction method, called ESSC-SL-CTSK-FS, is proposed in this paper by integrating the techniques of enhanced soft subspace clustering (ESSC) and sparse learning (SL). In this method, ESSC is used to generate the antecedents and various sparse subspace for different fuzzy rules, whereas SL is used to optimize the consequent parameters of the fuzzy rules, based on which the number of fuzzy rules can be effectively reduced. Finally, the proposed ESSC-SL-CTSK-FS method is used to construct con-cise zero-order TSK FS that can explain the scenes in high-dimensional data modeling more clearly and easily. Experiments are conducted on various real-world datasets to confirm the advantages.

cs.LG↗
arXiv2019

Hierarchical feature fusion framework for frequency recognition in SSVEP-based BCIs

Effective frequency recognition algorithms are critical in steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). In this study, we present a hierarchical feature fusion framework which can be used to design high-performance frequency recognition methods. The proposed framework includes two primary technique for fusing features: spatial dimension fusion (SD) and frequency dimension fusion (FD). Both SD and FD fusions are obtained using a weighted strategy with a nonlinear function. To assess our novel methods, we used the correlated component analysis (CORRCA) method to investigate the efficiency and effectiveness of the proposed framework. Experimental results were obtained from a benchmark dataset of thirty-five subjects and indicate that the extended CORRCA method used within the framework significantly outperforms the original CORCCA method. Accordingly, the proposed framework holds promise to enhance the performance of frequency recognition methods in SSVEP-based BCIs.

q-bio.NC↗
arXiv2019

A novel repetition normalized adversarial reward for headline generation

While reinforcement learning can effectively improve language generation models, it often suffers from generating incoherent and repetitive phrases \cite{paulus2017deep}. In this paper, we propose a novel repetition normalized adversarial reward to mitigate these problems. Our repetition penalized reward can greatly reduce the repetition rate and adversarial training mitigates generating incoherent phrases. Our model significantly outperforms the baseline model on ROUGE-1\,(+3.24), ROUGE-L\,(+2.25), and a decreased repetition-rate (-4.98\%).

cs.CL↗
arXiv2019

Transfer Representation Learning with TSK Fuzzy System

Transfer learning can address the learning tasks of unlabeled data in the target domain by leveraging plenty of labeled data from a different but related source domain. A core issue in transfer learning is to learn a shared feature space in where the distributions of the data from two domains are matched. This learning process can be named as transfer representation learning (TRL). The feature transformation methods are crucial to ensure the success of TRL. The most commonly used feature transformation method in TRL is kernel-based nonlinear mapping to the high-dimensional space followed by linear dimensionality reduction. But the kernel functions are lack of interpretability and are difficult to be selected. To this end, the TSK fuzzy system (TSK-FS) is combined with transfer learning and a more intuitive and interpretable modeling method, called transfer representation learning with TSK-FS (TRL-TSK-FS) is proposed in this paper. Specifically, TRL-TSK-FS realizes TRL from two aspects. On one hand, the data in the source and target domains are transformed into the fuzzy feature space in which the distribution distance of the data between two domains is min-imized. On the other hand, discriminant information and geo-metric properties of the data are preserved by linear discriminant analysis and principal component analysis. In addition, another advantage arises with the proposed method, that is, the nonlinear transformation is realized by constructing fuzzy mapping with the antecedent part of the TSK-FS instead of kernel functions which are difficult to be selected. Extensive experiments are conducted on the text and image datasets. The results obviously show the superiority of the proposed method.

cs.LG↗
arXiv2019

Towards Constraining Parity-Violations in Gravity with Satellite Gradiometry

Parity violation in gravity, if existed, could have important implications, and it is meaningful to search and test the possible observational effects. Chern-Simons modified gravity serves as a natural model for gravitational parity-violations. Especially, considering extensions to Einstein-Hilbert action up to second order curvature terms, it is known that theories of gravitational parity-violation will reduce to the dynamical Chern-Simons gravity. In this letter, we outline the theoretical principles of testing the dynamical Chern-Simons gravity with orbiting gravity gradiometers, which could be naturally incorporated into future satellite gravity missions. The secular gravity gradient signals, due to the Mashhoon-Theiss (anomaly) effect, in dynamical Chern-Simons gravity are worked out, which can improve the constraint of the corresponding Chern-Simons length scale $ξ^{\frac{1}{4}}_{cs}$ obtained from such measurement scheme. For orbiting superconducting gradiometers or gradiometers with optical readout, a bound $ξ^{\frac{1}{4}}_{cs}\leq 10^6 \ km$ (or even better) could in principle be obtained, which will be at least 2 orders of magnitude stronger than the current one based on the observations from the GP-B mission and the LAGEOS I, II satellites.

gr-qc↗
arXiv2018

Trust Region Based Adversarial Attack on Neural Networks

Deep Neural Networks are quite vulnerable to adversarial perturbations. Current state-of-the-art adversarial attack methods typically require very time consuming hyper-parameter tuning, or require many iterations to solve an optimization based adversarial attack. To address this problem, we present a new family of trust region based adversarial attacks, with the goal of computing adversarial perturbations efficiently. We propose several attacks based on variants of the trust region optimization method. We test the proposed methods on Cifar-10 and ImageNet datasets using several different models including AlexNet, ResNet-50, VGG-16, and DenseNet-121 models. Our methods achieve comparable results with the Carlini-Wagner (CW) attack, but with significant speed up of up to $37\times$, for the VGG-16 model on a Titan Xp GPU. For the case of ResNet-50 on ImageNet, we can bring down its classification accuracy to less than 0.1\% with at most $1.5\%$ relative $L_\infty$ (or $L_2$) perturbation requiring only $1.02$ seconds as compared to $27.04$ seconds for the CW attack. We have open sourced our method which can be accessed at [1].

cs.LG↗
arXiv2018

De Broglie relations, Gravitational time dilation and weak equivalence principle

Interplays between quantum physics and gravity have long inspired exciting studies, which reveal subtle connections between quantum laws and the general notion of curved spacetime. One important example is the uniqueness of free-falling motions in both quantum and gravitational physics. In this work, we re-investigate the free-falling motions of quantum test wave packets that are distributed over weakly curved spacetime backgrounds. Except for the de Broglie relations, no assumption of priori given Hamiltonians or least actions satisfied by the quantum system is made. We find that the mean motions of quantum test wave packets can be deduced naturally from the de Broglie relations with a generalized treatment of gravitational time dilations in quantum waves. Such mean motions of quantum test masses in gravitational field are independent of their masses and compositions, and restore exactly the free-falling or geodesic motions of classical test masses in curved spacetime. This suggests a novel perspective that weak equivalence principle, which states the universality of free-fall, may be deeply rooted in quantum physics and be a phenomenon that has emerged from the quantum world.

quant-ph↗
arXiv2018

Hopping Transport in SrTiO3/Nd1-xTiO3/SrTiO3 Heterostructures

Electronic transport near the insulator-metal transition is investigated in the molecular beam epitaxy-grown SrTiO3/Nd1-xTiO3/SrTiO3 heterostructures using temperature dependent magnetotransport measurements. It was found that Nd-vacancies introduce localized electronic states resulting in the variable range hopping transport at low temperatures. At a fixed Nd-vacancies concentration, a crossover from Mott to Efros-Shklovskii (ES) variable range hopping transport was observed with decreasing temperature. With increasing disorder, a sign reversal of magnetoresistance from positive to negative was observed revealing interplay between intra-state interaction and the energy dependence of the localization length as a function of disorder. These findings highlight the important role of stoichiometry when exploring intrinsic effect using heterostructure and interfaces in addition to offering broad opportunity to tailor low temperature transport using non-stoichiometry defects.

cond-mat.mtrl-sci↗
arXiv2018

Reconfiguration of Brain Network between Resting-state and Oddball Paradigm

The oddball paradigm is widely applied to the investigation of multiple cognitive functions. Prior studies have explored the cortical oscillation and power spectral differing from the resting-state conduction to oddball paradigm, but whether brain networks existing the significant difference is still unclear. Our study addressed how the brain reconfigures its architecture from a resting-state condition (i.e., baseline) to P300 stimulus task in the visual oddball paradigm. In this study, electroencephalogram (EEG) datasets were collected from 24 postgraduate students, who were required to only mentally count the number of target stimulus; afterwards the functional EEG networks constructed in different frequency bands were compared between baseline and oddball task conditions to evaluate the reconfiguration of functional network in the brain. Compared to the baseline, our results showed the significantly (p < 0.05) enhanced delta/theta EEG connectivity and decreased alpha default mode network in the progress of brain reconfiguration to the P300 task. Furthermore, the reconfigured coupling strengths were demonstrated to relate to P300 amplitudes, which were then regarded as input features to train a classifier to differentiate the high and low P300 amplitudes groups with an accuracy of 77.78%. The findings of our study help us to understand the changes of functional brain connectivity from resting-state to oddball stimulus task, and the reconfigured network pattern has the potential for the selection of good subjects for P300-based brain- computer interface.

eess.SP↗
arXiv2018

Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training

In this paper, we propose Emo2Vec which encodes emotional semantics into vectors. We train Emo2Vec by multi-task learning six different emotion-related tasks, including emotion/sentiment analysis, sarcasm classification, stress detection, abusive language classification, insult detection, and personality recognition. Our evaluation of Emo2Vec shows that it outperforms existing affect-related representations, such as Sentiment-Specific Word Embedding and DeepMoji embeddings with much smaller training corpora. When concatenated with GloVe, Emo2Vec achieves competitive performances to state-of-the-art results on several tasks using a simple logistic regression classifier.

cs.CL↗
arXiv2018

GIANT: Globally Improved Approximate Newton Method for Distributed Optimization

For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At every iteration, each worker locally finds an Approximate NewTon (ANT) direction, which is sent to the main driver. The main driver, then, averages all the ANT directions received from workers to form a {\it Globally Improved ANT} (GIANT) direction. GIANT is highly communication efficient and naturally exploits the trade-offs between local computations and global communications in that more local computations result in fewer overall rounds of communications. Theoretically, we show that GIANT enjoys an improved convergence rate as compared with first-order methods and existing distributed Newton-type methods. Further, and in sharp contrast with many existing distributed Newton-type methods, as well as popular first-order methods, a highly advantageous practical feature of GIANT is that it only involves one tuning parameter. We conduct large-scale experiments on a computer cluster and, empirically, demonstrate the superior performance of GIANT.

cs.LG↗
arXiv2018

Anomaly Detection for Skin Disease Images Using Variational Autoencoder

In this paper, we demonstrate the potential of applying Variational Autoencoder (VAE) [10] for anomaly detection in skin disease images. VAE is a class of deep generative models which is trained by maximizing the evidence lower bound of data distribution [10]. When trained on only normal data, the resulting model is able to perform efficient inference and to determine if a test image is normal or not. We perform experiments on ISIC2018 Challenge Disease Classification dataset (Task 3) and compare different methods to use VAE to detect anomaly. The model is able to detect all diseases with 0.779 AUCROC. If we focus on specific diseases, the model is able to detect melanoma with 0.864 AUCROC and detect actinic keratosis with 0.872 AUCROC, even if it only sees the images of nevus. To the best of our knowledge, this is the first applied work of deep generative models for anomaly detection in dermatology.

cs.LG↗
arXiv2018

Freeness of spherical Hecke modules of unramified $U(2,1)$ in characteristic $p$

Let $F$ be a non-archimedean local field of odd residue characteristic $p$. Let $G$ be the unramified unitary group $U(2, 1)(E/F)$ in three variables, and $K$ be a maximal compact open subgroup of $G$. For an irreducible smooth representation $σ$ of $K$ over $\overline{\mathbf{F}}_p$, we prove that the compactly induced representation $\text{ind}^G _K σ$ is free of infinite rank over the spherical Hecke algebra $\mathcal{H}(K, σ)$.

math.RT↗
arXiv2018

Heterogeneity of Synaptic Input Connectivity Regulates Spike-based Neuronal Avalanches

Our mysterious brain is believed to operate near a non-equilibrium point and generate critical self-organized avalanches in neuronal activity. Recent experimental evidence has revealed significant heterogeneity in both synaptic input and output connectivity, but whether the structural heterogeneity participates in the regulation of neuronal avalanches remains poorly understood. By computational modelling, we predict that different types of structural heterogeneity contribute distinct effects on avalanche neurodynamics. In particular, neuronal avalanches can be triggered at an intermediate level of input heterogeneity, but heterogeneous output connectivity cannot evoke avalanche dynamics. In the criticality region, the co-emergence of multi-scale cortical activities is observed, and both the avalanche dynamics and neuronal oscillations are modulated by the input heterogeneity. Remarkably, we show similar results can be reproduced in networks with various types of in- and out-degree distributions. Overall, these findings not only provide details on the underlying circuitry mechanisms of nonrandom synaptic connectivity in the regulation of neuronal avalanches, but also inspire testable hypotheses for future experimental studies.

q-bio.NC↗