arXiv subjects

John Wright

Publications and source records attributed to John Wright.

At least 73 records · Page 4Linked to original sources

arXiv2019

The New Nitrides: Layered, Ferroelectric, Magnetic, Metallic and Superconducting Nitrides to Boost the GaN Photonics and Electronics Eco-System

The nitride semiconductor materials GaN, AlN, and InN, and their alloys and heterostructures have been investigated extensively in the last 3 decades, leading to several technologically successful photonic and electronic devices. Just over the past few years, a number of new nitride materials have emerged with exciting photonic, electronic, and magnetic properties. Some examples are 2D and layered hBN and the III-V diamond analog cBN, the transition metal nitrides ScN, YN, and their alloys (e.g. ferroelectric ScAlN), piezomagnetic GaMnN, ferrimagnetic Mn4N, and epitaxial superconductor/semiconductor NbN/GaN heterojunctions. This article reviews the fascinating and emerging physics and science of these new nitride materials. It also discusses their potential applications in future generations of devices that take advantage of the photonic and electronic devices eco-system based on transistors, light-emitting diodes, and lasers that have already been created by the nitride semiconductors.

cond-mat.mtrl-sci↗

arXiv2019

Geometry and Symmetry in Short-and-Sparse Deconvolution

We study the $\textit{Short-and-Sparse (SaS) deconvolution}$ problem of recovering a short signal $\mathbf a_0$ and a sparse signal $\mathbf x_0$ from their convolution. We propose a method based on nonconvex optimization, which under certain conditions recovers the target short and sparse signals, up to a signed shift symmetry which is intrinsic to this model. This symmetry plays a central role in shaping the optimization landscape for deconvolution. We give a $\textit{regional analysis}$, which characterizes this landscape geometrically, on a union of subspaces. Our geometric characterization holds when the length-$p_0$ short signal $\mathbf a_0$ has shift coherence $μ$, and $\mathbf x_0$ follows a random sparsity model with sparsity rate $θ\in \Bigl[\frac{c_1}{p_0}, \frac{c_2}{p_0\sqrtμ+ \sqrt{p_0}}\Bigr]\cdot\frac{1}{\log^2p_0}$. Based on this geometry, we give a provable method that successfully solves SaS deconvolution with high probability.

eess.SP↗

arXiv2019

On the Global Geometry of Sphere-Constrained Sparse Blind Deconvolution

Blind deconvolution is the problem of recovering a convolutional kernel $\boldsymbol a_0$ and an activation signal $\boldsymbol x_0$ from their convolution $\boldsymbol y = \boldsymbol a_0 \circledast \boldsymbol x_0$. This problem is ill-posed without further constraints or priors. This paper studies the situation where the nonzero entries in the activation signal are sparsely and randomly populated. We normalize the convolution kernel to have unit Frobenius norm and cast the sparse blind deconvolution problem as a nonconvex optimization problem over the sphere. With this spherical constraint, every spurious local minimum turns out to be close to some signed shift truncation of the ground truth, under certain hypotheses. This benign property motivates an effective two stage algorithm that recovers the ground truth from the partial information offered by a suboptimal local minimum. This geometry-inspired algorithm recovers the ground truth for certain microscopy problems, also exhibits promising performance in the more challenging image deblurring problem. Our insights into the global geometry and the two stage algorithm extend to the convolutional dictionary learning problem, where a superposition of multiple convolution signals is observed.

cs.CV↗

arXiv2018

Efficient Dictionary Learning with Gradient Descent

Randomly initialized first-order optimization algorithms are the method of choice for solving many high-dimensional nonconvex problems in machine learning, yet general theoretical guarantees cannot rule out convergence to critical points of poor objective value. For some highly structured nonconvex problems however, the success of gradient descent can be understood by studying the geometry of the objective. We study one such problem -- complete orthogonal dictionary learning, and provide converge guarantees for randomly initialized gradient descent to the neighborhood of a global optimum. The resulting rates scale as low order polynomials in the dimension even though the objective possesses an exponential number of saddle points. This efficient convergence can be viewed as a consequence of negative curvature normal to the stable manifolds associated with saddle points, and we provide evidence that this feature is shared by other nonconvex problems of importance as well.

math.OC↗

arXiv2017

A positive-definite form of bounce-averaged quasilinear velocity diffusion for the parallel inhomogeneity in a tokamak

In this paper, the analytical form of the quasilinear diffusion coefficients is modified from the Kennel-Engelmann diffusion coefficients to guarantee the positive definiteness of its bounce average in a toroidal geometry. By evaluating the parallel inhomogeneity of plasmas and magnetic fields in the trajectory integral, we can ensure the positive definiteness and help illuminate some non-resonant toroidal effects in the quasilinear diffusion. When the correlation length of the plasma-wave interaction is comparable to the magnetic field variation length, the variation becomes important and the parabolic variation at the outer-midplane, the inner-midplane, and trapping tips can be evaluated by Airy functions. The new form allows the coefficients to include both resonant and non-resonant contributions, and the correlations between the consecutive resonances and in many poloidal periods. The positive-definite form is implemented in a wave code TORIC and we present an example for ITER using this form.

physics.plasm-ph↗

arXiv2017

Quantum state certification

We consider the problem of quantum state certification, where one is given $n$ copies of an unknown $d$-dimensional quantum mixed state $ρ$, and one wants to test whether $ρ$ is equal to some known mixed state $σ$ or else is $ε$-far from $σ$. The goal is to use notably fewer copies than the $Ω(d^2)$ needed for full tomography on $ρ$ (i.e., density estimation). We give two robust state certification algorithms: one with respect to fidelity using $n = O(d/ε)$ copies, and one with respect to trace distance using $n = O(d/ε^2)$ copies. The latter algorithm also applies when $σ$ is unknown as well. These copy complexities are optimal up to constant factors.

quant-ph↗

arXiv2017

Which Distribution Distances are Sublinearly Testable?

Given samples from an unknown distribution $p$ and a description of a distribution $q$, are $p$ and $q$ close or far? This question of "identity testing" has received significant attention in the case of testing whether $p$ and $q$ are equal or far in total variation distance. However, in recent work, the following questions have been been critical to solving problems at the frontiers of distribution testing: -Alternative Distances: Can we test whether $p$ and $q$ are far in other distances, say Hellinger? -Tolerance: Can we test when $p$ and $q$ are close, rather than equal? And if so, close in which distances? Motivated by these questions, we characterize the complexity of distribution testing under a variety of distances, including total variation, $\ell_2$, Hellinger, Kullback-Leibler, and $χ^2$. For each pair of distances $d_1$ and $d_2$, we study the complexity of testing if $p$ and $q$ are close in $d_1$ versus far in $d_2$, with a focus on identifying which problems allow strongly sublinear testers (i.e., those with complexity $O(n^{1 - γ})$ for some $γ> 0$ where $n$ is the size of the support of the distributions $p$ and $q$). We provide matching upper and lower bounds for each case. We also study these questions in the case where we only have samples from $q$ (equivalence testing), showing qualitative differences from identity testing in terms of when tolerance can be achieved. Our algorithms fall into the classical paradigm of $χ^2$-statistics, but require crucial changes to handle the challenges introduced by each distance we consider. Finally, we survey other recent results in an attempt to serve as a reference for the complexity of various distribution testing problems.

cs.DS↗

arXiv2017

Using Negative Curvature in Solving Nonlinear Programs

Minimization methods that search along a curvilinear path composed of a non-ascent nega- tive curvature direction in addition to the direction of steepest descent, dating back to the late 1970s, have been an effective approach to finding a stationary point of a function at which its Hessian is positive semidefinite. For constrained nonlinear programs arising from recent appli- cations, the primary goal is to find a stationary point that satisfies the second-order necessary optimality conditions. Motivated by this, we generalize the approach of using negative curvature directions from unconstrained optimization to nonlinear ones. We focus on equality constrained problems and prove that our proposed negative curvature method is guaranteed to converge to a stationary point satisfying second-order necessary conditions. A possible way to extend our proposed negative curvature method to general nonlinear programs is also briefly discussed.

math.OC↗

arXiv2017

Scalable Robust Matrix Recovery: Frank-Wolfe Meets Proximal Methods

Recovering matrices from compressive and grossly corrupted observations is a fundamental problem in robust statistics, with rich applications in computer vision and machine learning. In theory, under certain conditions, this problem can be solved in polynomial time via a natural convex relaxation, known as Compressive Principal Component Pursuit (CPCP). However, all existing provable algorithms for CPCP suffer from superlinear per-iteration cost, which severely limits their applicability to large scale problems. In this paper, we propose provable, scalable and efficient methods to solve CPCP with (essentially) linear per-iteration cost. Our method combines classical ideas from Frank-Wolfe and proximal methods. In each iteration, we mainly exploit Frank-Wolfe to update the low-rank component with rank-one SVD and exploit the proximal step for the sparse term. Convergence results and implementation details are also discussed. We demonstrate the scalability of the proposed approach with promising numerical experiments on visual data.

math.OC↗

arXiv2017

Quasilinear diffusion coefficients in a finite Larmor radius expansion for ion cyclotron heated plasmas

In this paper, a reduced model of quasilinear diffusion by a small Larmor radius approximation is derived to couple the Maxwell's equations and the Fokker-Planck equation self-consistently for ion cyclotron range of frequency waves in a tokamak. The reduced model ensures the important properties of the full model by Kennel-Engelmann diffusion, such as diffusion directions, wave polarizations, and H-theorem. The kinetic energy change (W-dot) is used to derive the reduced model diffusion coefficients for the fundamental damping and the second harmonic damping to the lowest order of the finite Larmor radius expansion. The quasilinear diffusion coefficients are implemented in a coupled code (TORIC-CQL3D) with the equivalent reduced model of dielectric tensor. We also present the simulations of the ITER minority heating scenario, in which the reduced model is verified within the allowable errors from the full model results.

physics.plasm-ph↗

arXiv2017

A Geometric Analysis of Phase Retrieval

Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of $m$ measurements, $y_k = |\mathbf a_k^* \mathbf x|$ for $k = 1, \dots, m$, is it possible to recover $\mathbf x \in \mathbb{C}^n$ (i.e., length-$n$ complex vector)? This **generalized phase retrieval** (GPR) problem is a fundamental task in various disciplines, and has been the subject of much recent investigation. Natural nonconvex heuristics often work remarkably well for GPR in practice, but lack clear theoretical explanations. In this paper, we take a step towards bridging this gap. We prove that when the measurement vectors $\mathbf a_k$'s are generic (i.i.d. complex Gaussian) and the number of measurements is large enough ($m \ge C n \log^3 n$), with high probability, a natural least-squares formulation for GPR has the following benign geometric structure: (1) there are no spurious local minimizers, and all global minimizers are equal to the target signal $\mathbf x$, up to a global phase; and (2) the objective function has a negative curvature around each saddle point. This structure allows a number of iterative optimization methods to efficiently find a global minimizer, without special initialization. To corroborate the claim, we describe and analyze a second-order trust-region algorithm.

cs.IT↗

arXiv2016

Efficient quantum tomography II

Following [OW16], we continue our analysis of: (1) "Quantum tomography", i.e., learning a quantum state, i.e., the quantum generalization of learning a discrete probability distribution; (2) The distribution of Young diagrams output by the RSK algorithm on random words. Regarding (2), we introduce two powerful new tools: (i) A precise upper bound on the expected length of the longest union of $k$ disjoint increasing subsequences in a random length-$n$ word with letter distribution $α_1 \geq α_2 \geq \cdots \geq α_d$; (ii) A new majorization property of the RSK algorithm that allows one to analyze the Young diagram formed by the lower rows $λ_k, λ_{k+1}, \dots$ of its output. These tools allow us to prove several new theorems concerning the distribution of random Young diagrams in the nonasymptotic regime, giving concrete error bounds that are optimal, or nearly so, in all parameters. As one example, we give a fundamentally new proof of the fact that the expected length of the longest increasing sequence in a random length-$n$ permutation is bounded by $2\sqrt{n}$. This is the $k = 1$, $α_i \equiv \frac1d$, $d \to \infty$ special case of a much more general result we prove: the expected length of the $k$th Young diagram row produced by an $α$-random word is $α_k n \pm 2\sqrt{α_kd n}$. From our new analyses of random Young diagrams we derive several new results in quantum tomography, including: (i) Learning the eigenvalues of an unknown state to $ε$-accuracy in Hellinger-squared, chi-squared, or KL distance, using $n = O(d^2/ε)$ copies; (ii) Learning the optimal rank-$k$ approximation of an unknown state to $ε$-fidelity (Hellinger-squared distance) using $n = \widetilde{O}(kd/ε)$ copies.

quant-ph↗

arXiv2016

Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method

We consider the problem of recovering a complete (i.e., square and invertible) matrix $\mathbf A_0$, from $\mathbf Y \in \mathbb{R}^{n \times p}$ with $\mathbf Y = \mathbf A_0 \mathbf X_0$, provided $\mathbf X_0$ is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers $\mathbf A_0$ when $\mathbf X_0$ has $O(n)$ nonzeros per column, under suitable probability model for $\mathbf X_0$. Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint, and hence is naturally phrased in the language of manifold optimization. In a companion paper (arXiv:1511.03607), we have showed that with high probability our nonconvex formulation has no "spurious" local minimizers and around any saddle point the objective function has a negative directional curvature. In this paper, we take advantage of the particular geometric structure, and describe a Riemannian trust region algorithm that provably converges to a local minimizer with from arbitrary initializations. Such minimizers give excellent approximations to rows of $\mathbf X_0$. The rows are then recovered by linear programming rounding and deflation.

cs.IT↗

arXiv2016

Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture

We consider the problem of recovering a complete (i.e., square and invertible) matrix $\mathbf A_0$, from $\mathbf Y \in \mathbb{R}^{n \times p}$ with $\mathbf Y = \mathbf A_0 \mathbf X_0$, provided $\mathbf X_0$ is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers $\mathbf A_0$ when $\mathbf X_0$ has $O(n)$ nonzeros per column, under suitable probability model for $\mathbf X_0$. In contrast, prior results based on efficient algorithms either only guarantee recovery when $\mathbf X_0$ has $O(\sqrt{n})$ zeros per column, or require multiple rounds of SDP relaxation to work when $\mathbf X_0$ has $O(n^{1-δ})$ nonzeros per column (for any constant $δ\in (0, 1)$). } Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint. In this paper, we provide a geometric characterization of the objective landscape. In particular, we show that the problem is highly structured: with high probability, (1) there are no "spurious" local minimizers; and (2) around all saddle points the objective has a negative directional curvature. This distinctive structure makes the problem amenable to efficient optimization algorithms. In a companion paper (arXiv:1511.04777), we design a second-order trust-region algorithm over the sphere that provably converges to a local minimizer from arbitrary initializations, despite the presence of saddle points.

cs.IT↗

arXiv2016

Finding a sparse vector in a subspace: Linear sparsity using alternating directions

Is it possible to find the sparsest vector (direction) in a generic subspace $\mathcal{S} \subseteq \mathbb{R}^p$ with $\mathrm{dim}(\mathcal{S})= n < p$? This problem can be considered a homogeneous variant of the sparse recovery problem, and finds connections to sparse dictionary learning, sparse PCA, and many other problems in signal processing and machine learning. In this paper, we focus on a **planted sparse model** for the subspace: the target sparse vector is embedded in an otherwise random subspace. Simple convex heuristics for this planted recovery problem provably break down when the fraction of nonzero entries in the target sparse vector substantially exceeds $O(1/\sqrt{n})$. In contrast, we exhibit a relatively simple nonconvex approach based on alternating directions, which provably succeeds even when the fraction of nonzero entries is $Ω(1)$. To the best of our knowledge, this is the first practical algorithm to achieve linear scaling under the planted sparse model. Empirically, our proposed algorithm also succeeds in more challenging data models, e.g., sparse dictionary learning.

cs.IT↗

arXiv2016

When Are Nonconvex Problems Not Scary?

In this note, we focus on smooth nonconvex optimization problems that obey: (1) all local minimizers are also global; and (2) around any saddle point or local maximizer, the objective has a negative directional curvature. Concrete applications such as dictionary learning, generalized phase retrieval, and orthogonal tensor decomposition are known to induce such structures. We describe a second-order trust-region algorithm that provably converges to a global minimizer efficiently, without special initializations. Finally we highlight alternatives, and open problems in this direction.

math.OC↗

arXiv2015

Complete Dictionary Recovery over the Sphere

We consider the problem of recovering a complete (i.e., square and invertible) matrix $\mathbf A_0$, from $\mathbf Y \in \mathbb R^{n \times p}$ with $\mathbf Y = \mathbf A_0 \mathbf X_0$, provided $\mathbf X_0$ is sufficiently sparse. This recovery problem is central to the theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals, and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers $\mathbf A_0$ when $\mathbf X_0$ has $O(n)$ nonzeros per column, under suitable probability model for $\mathbf X_0$. In contrast, prior results based on efficient algorithms provide recovery guarantees when $\mathbf X_0$ has only $O(n^{1-δ})$ nonzeros per column for any constant $δ\in (0, 1)$. Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint, and hence is naturally phrased in the language of manifold optimization. To show this apparently hard problem is tractable, we first provide a geometric characterization of the high-dimensional objective landscape, which shows that with high probability there are no "spurious" local minima. This particular geometric structure allows us to design a Riemannian trust region algorithm over the sphere that provably converges to one local minimizer with an arbitrary initialization, despite the presence of saddle points. The geometric approach we develop here may also shed light on other problems arising from nonconvex recovery of structured signals.

cs.IT↗

arXiv2015

Efficient quantum tomography

In the quantum state tomography problem, one wishes to estimate an unknown $d$-dimensional mixed quantum state $ρ$, given few copies. We show that $O(d/ε)$ copies suffice to obtain an estimate $\hatρ$ that satisfies $\|\hatρ - ρ\|_F^2 \leq ε$ (with high probability). An immediate consequence is that $O(\mathrm{rank}(ρ) \cdot d/ε^2) \leq O(d^2/ε^2)$ copies suffice to obtain an $ε$-accurate estimate in the standard trace distance. This improves on the best known prior result of $O(d^3/ε^2)$ copies for full tomography, and even on the best known prior result of $O(d^2\log(d/ε)/ε^2)$ copies for spectrum estimation. Our result is the first to show that nontrivial tomography can be obtained using a number of copies that is just linear in the dimension. Next, we generalize these results to show that one can perform efficient principal component analysis on $ρ$. Our main result is that $O(k d/ε^2)$ copies suffice to output a rank-$k$ approximation $\hatρ$ whose trace distance error is at most $ε$ more than that of the best rank-$k$ approximator to $ρ$. This subsumes our above trace distance tomography result and generalizes it to the case when $ρ$ is not guaranteed to be of low rank. A key part of the proof is the analogous generalization of our spectrum-learning results: we show that the largest $k$ eigenvalues of $ρ$ can be estimated to trace-distance error $ε$ using $O(k^2/ε^2)$ copies. In turn, this result relies on a new coupling theorem concerning the Robinson-Schensted-Knuth algorithm that should be of independent combinatorial interest.

quant-ph↗