arXiv subjects

Xiao-Shan Gao

Publications and source records attributed to Xiao-Shan Gao.

At least 55 records · Page 3Linked to original sources

arXiv2022

Bit Complexity of Polynomial GCD on Sparse Representation

An input- and output-sensitive GCD algorithm for multi-variate polynomials over finite fields is proposed by combining the modular method with the Ben-Or/Tiwari sparse interpolation. The bit complexity of the algorithm is given and is sensitive to the sparse representation, while for previous sparse GCD algorithms, the complexities were given only in some special cases. It is shown that the new algorithm is superior both in theory and in practice comparing with existing GCD algorithms: the complexity in the degree is decreased from quadratic to linear and the running times are decreased by 1-3 orders of magnitude in various benchmarks.

cs.SC↗

arXiv2022

Achieve Optimal Adversarial Accuracy for Adversarial Deep Learning using Stackelberg Game

Adversarial deep learning is to train robust DNNs against adversarial attacks, which is one of the major research focuses of deep learning. Game theory has been used to answer some of the basic questions about adversarial deep learning such as the existence of a classifier with optimal robustness and the existence of optimal adversarial samples for a given class of classifiers. In most previous work, adversarial deep learning was formulated as a simultaneous game and the strategy spaces are assumed to be certain probability distributions in order for the Nash equilibrium to exist. But, this assumption is not applicable to the practical situation. In this paper, we give answers to these basic questions for the practical case where the classifiers are DNNs with a given structure, by formulating the adversarial deep learning as sequential games. The existence of Stackelberg equilibria for these games are proved. Furthermore, it is shown that the equilibrium DNN has the largest adversarial accuracy among all DNNs with the same structure, when Carlini-Wagner's margin loss is used. Trade-off between robustness and accuracy in adversarial deep learning is also studied from game theoretical aspect.

cs.LG↗

arXiv2022

A Robust Classification-autoencoder to Defend Outliers and Adversaries

In this paper, a robust classification-autoencoder (CAE) is proposed, which has strong ability to recognize outliers and defend adversaries. The main idea is to change the autoencoder from an unsupervised learning model into a classifier, where the encoder is used to compress samples with different labels into disjoint compression spaces and the decoder is used to recover samples from their compression spaces. The encoder is used both as a compressed feature learner and as a classifier, and the decoder is used to decide whether the classification given by the encoder is correct by comparing the input sample with the output. Since adversary samples are seemingly inevitable for the current DNN framework, the list classifier to defend adversaries is introduced based on CAE, which outputs several labels and the corresponding samples recovered by the CAE. Extensive experimental results are used to show that the CAE achieves state of the art to recognize outliers by finding almost all outliers; the list classifier gives near lossless classification in the sense that the output list contains the correct label for almost all adversaries and the size of the output list is reasonably small.

cs.LG↗

arXiv2022

Skew-sparse matrix multiplication

Based on the observation that $\mathbb{Q}^{(p-1) \times (p-1)}$ is isomorphic to a quotient skew polynomial ring, we propose a new method for $(p-1)\times (p-1)$ matrix multiplication over $\mathbb{Q}$, where $p$ is a prime number. The main feature of our method is the acceleration for matrix multiplication if the product is skew-sparse. Based on the new method, we design a deterministic algorithm with complexity $O(T^{ω-2} p^2)$, where $T\le p-1$ is a parameter determined by the skew-sparsity of input matrices and $ω$ is the asymptotic exponent of matrix multiplication. Moreover, by introducing randomness, we also propose a probabilistic algorithm with complexity $O^\thicksim(t^{ω-2}p^2+p^2\log\frac{1}ν)$, where $t\le p-1$ is the skew-sparsity of the product and $ν$ is the probability parameter.

cs.CC↗

arXiv2022

Adversarial Parameter Attack on Deep Neural Networks

In this paper, a new parameter perturbation attack on DNNs, called adversarial parameter attack, is proposed, in which small perturbations to the parameters of the DNN are made such that the accuracy of the attacked DNN does not decrease much, but its robustness becomes much lower. The adversarial parameter attack is stronger than previous parameter perturbation attacks in that the attack is more difficult to be recognized by users and the attacked DNN gives a wrong label for any modified sample input with high probability. The existence of adversarial parameters is proved. For a DNN $F_Θ$ with the parameter set $Θ$ satisfying certain conditions, it is shown that if the depth of the DNN is sufficiently large, then there exists an adversarial parameter set $Θ_a$ for $Θ$ such that the accuracy of $F_{Θ_a}$ is equal to that of $F_Θ$, but the robustness measure of $F_{Θ_a}$ is smaller than any given bound. An effective training algorithm is given to compute adversarial parameters and numerical experiments are used to demonstrate that the algorithms are effective to produce high quality adversarial parameters.

cs.LG↗

arXiv2022

Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks

In this paper, the bias classifier is introduced, that is, the bias part of a DNN with Relu as the activation function is used as a classifier. The work is motivated by the fact that the bias part is a piecewise constant function with zero gradient and hence cannot be directly attacked by gradient-based methods to generate adversaries, such as FGSM. The existence of the bias classifier is proved and an effective training method for the bias classifier is given. It is proved that by adding a proper random first-degree part to the bias classifier, an information-theoretically safe classifier against the original-model gradient attack is obtained in the sense that the attack will generate a totally random attacking direction. This seems to be the first time that the concept of information-theoretically safe classifier is proposed. Several attack methods for the bias classifier are proposed and numerical experiments are used to show that the bias classifier is more robust than DNNs with similar size against these attacks in most cases.

cs.LG↗

arXiv2022

Proving Information Inequalities and Identities with Symbolic Computation

Proving linear inequalities and identities of Shannon's information measures, possibly with linear constraints on the information measures, is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In particular, an identity $f = 0$ is verified by solving two LPs, one for $f \ge 0$ and one for $f \le 0$. In this paper, we develop a set of algorithms that can be implemented by symbolic computation. Based on these algorithms, procedures for verifying linear information inequalities and identities are devised. Compared with LP-based algorithms, our procedures can produce analytical proofs that are both human-verifiable and free of numerical errors. Our procedures are also more efficient computationally. For constrained inequalities, by taking advantage of the algebraic structure of the problem, the size of the LP that needs to be solved can be significantly reduced. For identities, instead of solving two LPs, the identity can be verified directly with very little computation.

cs.IT↗

arXiv2021

Analyzing the barren plateau phenomenon in training quantum neural networks with the ZX-calculus

In this paper, we propose a general scheme to analyze the gradient vanishing phenomenon, also known as the barren plateau phenomenon, in training quantum neural networks with the ZX-calculus. More precisely, we extend the barren plateaus theorem from unitary 2-design circuits to any parameterized quantum circuits under certain reasonable assumptions. The main technical contribution of this paper is representing certain integrations as ZX-diagrams and computing them with the ZX-calculus. The method is used to analyze four concrete quantum neural networks with different structures. It is shown that, for the hardware efficient ansatz and the MPS-inspired ansatz, there exist barren plateaus, while for the QCNN ansatz and the tree tensor network ansatz, there exists no barren plateau.

quant-ph↗

arXiv2021

A Generalization of the Concavity of Rényi Entropy Powe

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the $p$-nonlinear heat equation in $\mathbb{R}^n$ is always a concave function of time, which extends Costa's concavity inequality for Shannon's entropy power to Rényi entropies. In this paper, we give a generalization of Savaré-Toscani's result by giving a class of sufficient conditions of the parameters under which the concavity of the Rényi entropy power is still valid. These conditions are quite general and include the parameter range given by Savaré-Toscani as special cases. Also, the conditions are obtained with a systematical approach.

cs.IT↗

arXiv2020

QDNN: DNN with Quantum Neural Network Layers

In this paper, we introduce a quantum extension of classical DNN, QDNN. The QDNN consisting of quantum structured layers can uniformly approximate any continuous function and has more representation power than the classical DNN. It still keeps the advantages of the classical DNN such as the non-linear activation, the multi-layer structure, and the efficient backpropagation training algorithm. Moreover, the QDNN can be used on near-term noisy intermediate-scale quantum processors. A numerical experiment for image classification based on quantum DNN is given, where a high accuracy rate is achieved.

quant-ph↗

arXiv2020

Improve the Robustness and Accuracy of Deep Neural Network with $L_{2,\infty}$ Normalization

In this paper, the robustness and accuracy of the deep neural network (DNN) was enhanced by introducing the $L_{2,\infty}$ normalization of the weight matrices of the DNN with Relu as the activation function. It is proved that the $L_{2,\infty}$ normalization leads to large dihedral angles between two adjacent faces of the polyhedron graph of the DNN function and hence smoother DNN functions, which reduces over-fitting. A measure is proposed for the robustness of a classification DNN, which is the average radius of the maximal robust spheres with the sample data as centers. A lower bound for the robustness measure is given in terms of the $L_{2,\infty}$ norm. Finally, an upper bound for the Rademacher complexity of DNN with $L_{2,\infty}$ normalization is given. An algorithm is given to train a DNN with the $L_{2,\infty}$ normalization and experimental results are used to show that the $L_{2,\infty}$ normalization is effective to improve the robustness and accuracy.

stat.ML↗

arXiv2020

Lower Bound on Derivatives of Costa's Differential Entropy

Several conjectures concern the lower bound for the differential entropy $H(X_t)$ of an $n$-dimensional random vector $X_t$ introduced by Costa. Cheng and Geng conjectured that $H(X_t)$ is completely monotone, that is, $C_1(m,n): (-1)^{m+1}(d^m/d^m t)H(X_t)\ge0$. McKean conjectured that Gaussian $X_{Gt}$ achieves the minimum of $(-1)^{m+1}(d^m/d^m t)H(X_t)$ under certain conditions, that is, $C_2(m,n): (-1)^{m+1}(d^m/d^m t)H(X_t)\ge(-1)^{m+1}(d^m/d^m t)H(X_{Gt})$. McKean's conjecture was only considered in the univariate case before: $C_2(1,1)$ and $C_2(2,1)$ were proved by McKean and $C_2(i,1),i=3,4,5$ were proved by Zhang-Anantharam-Geng under the log-concave condition. In this paper, we prove $C_2(1,n)$, $C_2(2,n)$ and observe that McKean's conjecture might not be true for $n>1$ and $m>2$. We further propose a weaker version $C_3(m,n): (-1)^{m+1}(d^m/d^m t)H(X_t)\ge(-1)^{m+1}\frac{1}{n}(d^m/d^m t)H(X_{Gt})$ and prove $C_3(3,2)$, $C_3(3,3)$, $C_3(3,4)$, $C_3(4,2)$ under the log-concave condition. A systematical procedure to prove $C_l(m,n)$ is proposed based on semidefinite programming and the results mentioned above are proved using this procedure.

cs.IT↗

arXiv2020

Prove Costa's Entropy Power Inequality and High Order Inequality for Differential Entropy with Semidefinite Programming

Costa's entropy power inequality is an important generalization of Shannon's entropy power inequality. Related with Costa's entropy power inequality and a conjecture proposed by McKean in 1966, Cheng-Geng recently conjectured that $D(m,n): (-1)^{m+1}(\partial^m/\partial^m t)H(X_t)\ge0$, where $X_t$ is the $n$-dimensional random variable in Costa's entropy power inequality and $H(X_t)$ the differential entropy of $X_t$. $D(1,n)$ and $D(2,n)$ were proved by Costa as consequences of Costa's entropy power inequality. Cheng-Geng proved $D(3,1)$ and $D(4,1)$. In this paper, we propose a systematical procedure to prove $D(m,n)$ and Costa's entropy power inequality based on semidefinite programming. Using software packages based on this procedure, we prove $D(3,n)$ for $n=2,3,4$ and give a new proof for Costa's entropy power inequality. We also show that with the currently known constraints, $D(5,1)$ and $D(4,2)$ cannot be proved with the procedure.

math.PR↗

arXiv2019

Tropical Differential Groebner Basis

In this paper, the tropical differential Gröbner basis is studied, which is a natural generalization of the tropical Gröbner basis to the recently introduced tropical differential algebra. Like the differential Gröbner basis, the tropical differential Gröbner basis generally contains an infinite number of elements. We give a Buchberger style criterion for the tropical differential Gröbner basis. For ideals generated by homogeneous linear differential polynomials with constant coefficients, we give a complete algorithm to compute the tropical differential Gröbner basis.

cs.SC↗

arXiv2018

Quantum Algorithm for Optimization and Polynomial System Solving over Finite Field and Application to Cryptanalysis

In this paper, we give quantum algorithms for two fundamental computation problems: solving polynomial systems over finite fields and optimization where the arguments of the objective function and constraints take values from a finite field or a bounded interval of integers. The quantum algorithms can solve these problems with any given success probability and have polynomial runtime complexities in the size of the input, the degree of the inequality constraints, and the condition number of certain matrices derived from the problem. So, we achieved exponential speedup for these problems when their condition numbers are small. As applications, quantum algorithms are given to three basic computational problems in cryptography: the polynomial system with noise problem, the short integer solution problem, the shortest vector problem, as well as the cryptanalysis for the lattice based NTRU cryptosystem. It is shown that these problems and NTRU can against quantum computer attacks only if their condition numbers are large, so the condition number could be used as a new criterion for the lattice based post-quantum cryptosystems.

cs.SC↗

arXiv2018

Deterministic Interpolation of Sparse Black-box Multivariate Polynomials using Kronecker Type Substitutions

In this paper, we propose two new deterministic interpolation algorithms for a sparse multivariate polynomial given as a standard black-box by introducing new Kronecker type substitutions. Let $f\in \RB[x_1,\dots,x_n]$ be a sparse black-box polynomial with a degree bound $D$. When $\RB=\C$ or a finite field, our algorithms either have better bit complexity or better bit complexity in $D$ than existing deterministic algorithms. In particular, in the case of deterministic algorithms for standard black-box models, our second algorithm has the current best complexity in $D$ which is the dominant factor in the complexity.

cs.SC↗

arXiv2018

Quantum Algorithms for Boolean Equation Solving and Quantum Algebraic Attack on Cryptosystems

Decision of whether a Boolean equation system has a solution is an NPC problem and finding a solution is NP hard. In this paper, we present a quantum algorithm to decide whether a Boolean equation system FS has a solution and compute one if FS does have solutions with any given success probability. The runtime complexity of the algorithm is polynomial in the size of FS and the condition number of FS. As a consequence, we give a polynomial-time quantum algorithm for solving Boolean equation systems if their condition numbers are small, say polynomial in the size of FS. We apply our quantum algorithm for solving Boolean equations to the cryptanalysis of several important cryptosystems: the stream cipher Trivum, the block cipher AES, the hash function SHA-3/Keccak, and the multivariate public key cryptosystems, and show that they are secure under quantum algebraic attack only if the condition numbers of the corresponding equation systems are large. This leads to a new criterion for designing cryptosystems that can against the attack of quantum computers: their corresponding equation systems must have large condition numbers.

quant-ph↗

arXiv2018

Faster Interpolation Algorithms for Sparse Multivariate Polynomials Given by Straight-Line Programs\

In this paper, we propose new deterministic and Monte Carlo interpolation algorithms for sparse multivariate polynomials represented by straight-line programs. Let $f$ be an $n$-variate polynomial given by a straight-line program, which has a degree bound $D$ and a term bound $T$. Our deterministic algorithm is quadratic in $n,T$ and cubic in $\log D$ in the Soft-Oh sense, which has better complexities than existing deterministic interpolation algorithms in most cases. Our Monte Carlo interpolation algorithms have better complexities than existing Monte Carlo interpolation algorithms and are the first algorithms whose complexities are linear in $nT$ in the Soft-Oh sense. Since $nT$ is a factor of the size of $f$, our Monte Carlo algorithms are optimal in $n$ and $T$ in the Soft-Oh sense.

cs.SC↗