arXiv subjects

Ke Sun

Publications and source records attributed to Ke Sun.

At least 109 records · Page 6Linked to original sources

arXiv2023

Mathematical Challenges in Deep Learning

Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challenges to this field with applications in cell phones, personal computers, autonomous cars, and wireless base stations. Here we list a set of problems, ranging from training, inference, generalization bound, and optimization with some formalism to communicate these challenges with mathematicians, statisticians, and theoretical computer scientists. This is a subjective view of the research questions in deep learning that benefits the tech industry in long run.

cs.LG↗

arXiv2023

Realization of Scalable Cirac-Zoller Multi-Qubit Gates

The universality theorem in quantum computing states that any quantum computational task can be decomposed into a finite set of logic gates operating on one and two qubits. However, the process of such decomposition is generally inefficient, often leading to exponentially many gates to realize an arbitrary computational task. Practical processor designs benefit greatly from availability of multi-qubit gates that operate on more than two qubits to implement the desired circuit. In 1995, Cirac and Zoller proposed a method to realize native multi-qubit controlled-$Z$ gates in trapped ion systems, which has a stringent requirement on ground-state cooling of the motional modes utilized by the gate. An alternative approach, the Mølmer-Sørensen gate, is robust against residual motional excitation and has been a foundation for many high-fidelity gate demonstrations. This gate does not scale well beyond two qubits, incurring additional overhead when used to construct many target algorithms. Here, we take advantage of novel performance benefits of long ion chains to realize fully programmable and scalable high-fidelity Cirac-Zoller gates.

quant-ph↗

arXiv2023

Asymptotic Learning Requirements for Stealth Attacks on Linearized State Estimation

Information-theoretic stealth attacks are data injection attacks that minimize the amount of information acquired by the operator about the state variables, while simultaneously limiting the Kullback-Leibler divergence between the distribution of the measurements under attack and the distribution under normal operation with the aim of controling the probability of detection. For Gaussian distributed state variables, attack construction requires knowledge of the second order statistics of the state variables, which is estimated from a finite number of past realizations using a sample covariance matrix. Within this framework, the attack performance is studied for the attack construction with the sample covariance matrix. This results in an analysis of the amount of data required to learn the covariance matrix of the state variables used on the attack construction. The ergodic attack performance is characterized using asymptotic random matrix theory tools, and the variance of the attack performance is bounded. The ergodic performance and the variance bounds are assessed with simulations on IEEE test systems.

cs.IT↗

arXiv2022

Some Properties of the Nash Equilibrium in $2 \times 2$ Zero-Sum Games

In this report, some properties of the set of Nash equilibria (NEs) of $2 \times 2$ zero-sum games are reviewed. In particular, the cardinality of the set of NEs is given in terms of the entries of the payoff matrix. Moreover, closed-form expressions for the NE strategies and the payoff at the NE (the value of the game) are provided in terms of the entries of the payoff matrix. The results presented in this report are not necessarily new knowledge, as they follow from the definition of the NE after some tedious calculations. Nevertheless this synthetic presentation is original in the literature.

cs.GT↗

arXiv2022

Fair Wrapping for Black-box Predictions

We introduce a new family of techniques to post-process ("wrap") a black-box classifier in order to reduce its bias. Our technique builds on the recent analysis of improper loss functions whose optimization can correct any twist in prediction, unfairness being treated as a twist. In the post-processing, we learn a wrapper function which we define as an $α$-tree, which modifies the prediction. We provide two generic boosting algorithms to learn $α$-trees. We show that our modification has appealing properties in terms of composition of $α$-trees, generalization, interpretability, and KL divergence between modified and original predictions. We exemplify the use of our technique in three fairness notions: conditional value-at-risk, equality of opportunity, and statistical parity; and provide experiments on several readily available datasets.

stat.ML↗

arXiv2022

Identification, Amplification and Measurement: A bridge to Gaussian Differential Privacy

Gaussian differential privacy (GDP) is a single-parameter family of privacy notions that provides coherent guarantees to avoid the exposure of sensitive individual information. Despite the extra interpretability and tighter bounds under composition GDP provides, many widely used mechanisms (e.g., the Laplace mechanism) inherently provide GDP guarantees but often fail to take advantage of this new framework because their privacy guarantees were derived under a different background. In this paper, we study the asymptotic properties of privacy profiles and develop a simple criterion to identify algorithms with GDP properties. We propose an efficient method for GDP algorithms to narrow down possible values of an optimal privacy measurement, $μ$ with an arbitrarily small and quantifiable margin of error. For non GDP algorithms, we provide a post-processing procedure that can amplify existing privacy guarantees to meet the GDP condition. As applications, we compare two single-parameter families of privacy notions, $ε$-DP, and $μ$-GDP, and show that all $ε$-DP algorithms are intrinsically also GDP. Lastly, we show that the combination of our measurement process and the composition theorem of GDP is a powerful and convenient tool to handle compositions compared to the traditional standard and advanced composition theorems.

cs.CR↗

arXiv2022

Angle-robust Two-Qubit Gates in a Linear Ion Crystal

In trapped-ion quantum computers, two-qubit entangling gates are generated by applying spin-dependent force which uses phonons to mediate interaction between the internal states of the ions. To maintain high-fidelity two-qubit gates under fluctuating experimental parameters, robust pulse-design methods are applied to remove the residual spin-motion entanglement in the presence of motional mode frequency drifts. Here we propose an improved pulse-design method that also guarantees the robustness of the two-qubit rotation angle against uniform mode frequency drifts by combining pulses with opposite sensitivity of the angle to mode frequency drifts. We experimentally measure the performance of the designed gates and see an improvement on both gate fidelity and gate performance under uniform mode frequency offsets.

quant-ph↗

arXiv2022

Intrinsic Universal Measurements of Non-linear Embeddings

A basic problem in machine learning is to find a mapping $f$ from a low dimensional latent space $\mathcal{Y}$ to a high dimensional observation space $\mathcal{X}$. Modern tools such as deep neural networks are capable to represent general non-linear mappings. A learner can easily find a mapping which perfectly fits all the observations. However, such a mapping is often not considered as good, because it is not simple enough and can overfit. How to define simplicity? We try to make a formal definition on the amount of information imposed by a non-linear mapping $f$. Intuitively, we measure the local discrepancy between the pullback geometry and the intrinsic geometry of the latent space. Our definition is based on information geometry and is independent of the empirical observations, nor specific parameterizations. We prove its basic properties and discuss relationships with related machine learning methods.

cs.LG↗

arXiv2022

Secure Quantized Training for Deep Learning

We implement training of neural networks in secure multi-party computation (MPC) using quantization commonly used in said setting. We are the first to present an MNIST classifier purely trained in MPC that comes within 0.2 percent of the accuracy of the same convolutional neural network trained via plaintext computation. More concretely, we have trained a network with two convolutional and two dense layers to 99.2% accuracy in 3.5 hours (under one hour for 99% accuracy). We have also implemented AlexNet for CIFAR-10, which converges in a few hours. We develop novel protocols for exponentiation and inverse square root. Finally, we present experiments in a range of MPC security models for up to ten parties, both with honest and dishonest majority as well as semi-honest and malicious security.

cs.LG↗

arXiv2022

An adaptive model checking test for functional linear model

Numerous studies have been devoted to the estimation and inference problems for functional linear models (FLM). However, few works focus on model checking problem that ensures the reliability of results. Limited tests in this area do not have tractable null distributions or asymptotic analysis under alternatives. Also, the functional predictor is usually assumed to be fully observed, which is impractical. To address these problems, we propose an adaptive model checking test for FLM. It combines regular moment-based and conditional moment-based tests, and achieves model adaptivity via the dimension of a residual-based subspace. The advantages of our test are manifold. First, it has a tractable chi-squared null distribution and higher powers under the alternatives than its components. Second, asymptotic properties under different underlying models are developed, including the unvisited local alternatives. Third, the test statistic is constructed upon finite grid points, which incorporates the discrete nature of collected data. We develop the desirable relationship between sample size and number of grid points to maintain the asymptotic properties. Besides, we provide a data-driven approach to estimate the dimension leading to model adaptivity, which is promising in sufficient dimension reduction. We conduct comprehensive numerical experiments to demonstrate the advantages the test inherits from its two simple components.

stat.ME↗

arXiv2022

You Cannot Always Win the Race: Analyzing the LFENCE/JMP Mitigation for Branch Target Injection

LFENCE/JMP is an existing software mitigation option for Branch Target Injection (BTI) and similar transient execution attacks stemming from indirect branch predictions, which is commonly used on AMD processors. However, the effectiveness of this mitigation can be compromised by the inherent race condition between the speculative execution of the predicted target and the architectural resolution of the intended target, since this can create a window in which code can still be transiently executed. This work investigates the potential sources of latency that may contribute to such a speculation window. We show that an attacker can "win the race", and thus that this window can still be sufficient to allow exploitation of BTI-style attacks on a variety of different x86 CPUs, despite the presence of the LFENCE/JMP mitigation.

cs.CR↗

arXiv2022

RTGNN: A Novel Approach to Model Stochastic Traffic Dynamics

Modeling stochastic traffic dynamics is critical to developing self-driving cars. Because it is difficult to develop first principle models of cars driven by humans, there is great potential for using data driven approaches in developing traffic dynamical models. While there is extensive literature on this subject, previous works mainly address the prediction accuracy of data-driven models. Moreover, it is often difficult to apply these models to common planning frameworks since they fail to meet the assumptions therein. In this work, we propose a new stochastic traffic model, Recurrent Traffic Graph Neural Network (RTGNN), by enforcing additional structures on the model so that the proposed model can be seamlessly integrated with existing motion planning algorithms. RTGNN is a Markovian model and is able to infer future traffic states conditioned on the motion of the ego vehicle. Specifically, RTGNN uses a definition of the traffic state that includes the state of all players in a local region and is therefore able to make joint predictions for all agents of interest. Meanwhile, we explicitly model the hidden states of agents, "intentions," as part of the traffic state to reflect the inherent partial observability of traffic dynamics. The above mentioned properties are critical for integrating RTGNN with motion planning algorithms coupling prediction and decision making. Despite the additional structures, we show that RTGNN is able to achieve state-of-the-art accuracy through comparisons with other similar works.

cs.LG↗

arXiv2022

Contrastive Laplacian Eigenmaps

Graph contrastive learning attracts/disperses node representations for similar/dissimilar node pairs under some notion of similarity. It may be combined with a low-dimensional embedding of nodes to preserve intrinsic and structural properties of a graph. In this paper, we extend the celebrated Laplacian Eigenmaps with contrastive learning, and call them COntrastive Laplacian EigenmapS (COLES). Starting from a GAN-inspired contrastive formulation, we show that the Jensen-Shannon divergence underlying many contrastive graph embedding models fails under disjoint positive and negative distributions, which may naturally emerge during sampling in the contrastive setting. In contrast, we demonstrate analytically that COLES essentially minimizes a surrogate of Wasserstein distance, which is known to cope well under disjoint distributions. Moreover, we show that the loss of COLES belongs to the family of so-called block-contrastive losses, previously shown to be superior compared to pair-wise losses typically used by contrastive methods. We show on popular benchmarks/backbones that COLES offers favourable accuracy/scalability compared to DeepWalk, GCN, Graph2Gauss, DGI and GRACE baselines.

cs.LG↗

arXiv2021

Dual Contrastive Learning for General Face Forgery Detection

With various facial manipulation techniques arising, face forgery detection has drawn growing attention due to security concerns. Previous works always formulate face forgery detection as a classification problem based on cross-entropy loss, which emphasizes category-level differences rather than the essential discrepancies between real and fake faces, limiting model generalization in unseen domains. To address this issue, we propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which specially constructs positive and negative paired data and performs designed contrastive learning at different granularities to learn generalized feature representation. Concretely, combined with the hard sample selection strategy, Inter-Instance Contrastive Learning (Inter-ICL) is first proposed to promote task-related discriminative features learning by especially constructing instance pairs. Moreover, to further explore the essential discrepancies, Intra-Instance Contrastive Learning (Intra-ICL) is introduced to focus on the local content inconsistencies prevalent in the forged faces by constructing local-region pairs inside instances. Extensive experiments and visualizations on several datasets demonstrate the generalization of our method against the state-of-the-art competitors.

cs.CV↗

arXiv2021

Clustering in Hilbert simplex geometry

Clustering categorical distributions in the finite-dimensional probability simplex is a fundamental task met in many applications dealing with normalized histograms. Traditionally, the differential-geometric structures of the probability simplex have been used either by (i) setting the Riemannian metric tensor to the Fisher information matrix of the categorical distributions, or (ii) defining the dualistic information-geometric structure induced by a smooth dissimilarity measure, the Kullback-Leibler divergence. In this work, we introduce for clustering tasks a novel computationally-friendly framework for modeling geometrically the probability simplex: The {\em Hilbert simplex geometry}. In the Hilbert simplex geometry, the distance is the non-separable Hilbert's metric distance which satisfies the property of information monotonicity with distance level set functions described by polytope boundaries. We show that both the Aitchison and Hilbert simplex distances are norm distances on normalized logarithmic representations with respect to the $\ell_2$ and variation norms, respectively. We discuss the pros and cons of those different statistical modelings, and benchmark experimentally these different kind of geometries for center-based $k$-means and $k$-center clustering. Furthermore, since a canonical Hilbert distance can be defined on any bounded convex subset of the Euclidean space, we also consider Hilbert's geometry of the elliptope of correlation matrices and study its clustering performances compared to Fröbenius and log-det divergences.

cs.LG↗

arXiv2021

On the Variance of the Fisher Information for Deep Learning

In the realm of deep learning, the Fisher information matrix (FIM) gives novel insights and useful tools to characterize the loss landscape, perform second-order optimization, and build geometric learning theories. The exact FIM is either unavailable in closed form or too expensive to compute. In practice, it is almost always estimated based on empirical samples. We investigate two such estimators based on two equivalent representations of the FIM -- both unbiased and consistent. Their estimation quality is naturally gauged by their variance given in closed form. We analyze how the parametric structure of a deep neural network can affect the variance. The meaning of this variance measure and its upper bounds are then discussed in the context of deep learning.

cs.LG↗

arXiv2021

Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a rigorous mathematical justification for the benefits of Anderson mixing in RL has not yet been put forward. In this paper, we provide deeper insights into a class of acceleration schemes built on Anderson mixing that improve the convergence of deep RL algorithms. Our main results establish a connection between Anderson mixing and quasi-Newton methods and prove that Anderson mixing increases the convergence radius of policy iteration schemes by an extra contraction factor. The key focus of the analysis roots in the fixed-point iteration nature of RL. We further propose a stabilization strategy by introducing a stable regularization term in Anderson mixing and a differentiable, non-expansive MellowMax operator that can allow both faster convergence and more stable behavior. Extensive experiments demonstrate that our proposed method enhances the convergence, stability, and performance of RL algorithms.

cs.LG↗

arXiv2021

High stability cryogenic system for quantum computing with compact packaged ion traps

Cryogenic environments benefit ion trapping experiments by offering lower motional heating rates, collision energies, and an ultra-high vacuum (UHV) environment for maintaining long ion chains for extended periods of time. Mechanical vibrations caused by compressors in closed-cycle cryostats can introduce relative motion between the ion and the wavefronts of lasers used to manipulate the ions. Here, we present a novel ion trapping system where a commercial low-vibration closed-cycle cryostat is used in a custom monolithic enclosure. We measure mechanical vibrations of the sample stage using an optical interferometer, and observe a root-mean-square relative displacement of 2.4 nm and a peak-to-peak displacement of 17 nm between free-space beams and the trapping location. We packaged a surface ion trap in a cryo-package assembly that enables easy handling, while creating a UHV environment for the ions. The trap cryo-package contains activated carbon getter material for enhanced sorption pumping near the trapping location, and source material for ablation loading. Using $^{171}$Yb$^{+}$ as our ion we estimate the operating pressure of the trap as a function of package temperature using phase transitions of zig-zag ion chains as a probe. We measured the radial mode heating rate of a single ion to be 13 quanta/s on average. The Ramsey coherence measurements yield 330 ms coherence time for counter-propagating Raman carrier transitions using a 355 nm mode-locked pulse laser, demonstrating the high optical stability.

physics.atom-ph↗