arXiv subjects

Yuehaw Khoo

Publications and source records attributed to Yuehaw Khoo.

At least 55 records · Page 3Linked to original sources

arXiv2022

Generative Modeling via Tree Tensor Network States

In this paper, we present a density estimation framework based on tree tensor-network states. The proposed method consists of determining the tree topology with Chow-Liu algorithm, and obtaining a linear system of equations that defines the tensor-network components via sketching techniques. Novel choices of sketch functions are developed in order to consider graphical models that contain loops. Sample complexity guarantees are provided and further corroborated by numerical experiments.

stat.ML↗

arXiv2022

Quantitatively visualizing bipartite datasets

As experiments continue to increase in size and scope, a fundamental challenge of subsequent analyses is to recast the wealth of information into an intuitive and readily-interpretable form. Often, each measurement only conveys the relationship between a pair of entries, and it is difficult to integrate these local interactions across a dataset to form a cohesive global picture. The classic localization problem tackles this question, transforming local measurements into a global map that reveals the underlying structure of a system. Here, we examine the more challenging bipartite localization problem, where pairwise distances are only available for bipartite data comprising two classes of entries (such as antibody-virus interactions, drug-cell potency, or user-rating profiles). We modify previous algorithms to solve bipartite localization and examine how each method behaves in the presence of noise, outliers, and partially-observed data. As a proof of concept, we apply these algorithms to antibody-virus neutralization measurements to create a basis set of antibody behaviors, formalize how potently inhibiting some viruses necessitates weakly inhibiting other viruses, and quantify how often combinations of antibodies exhibit degenerate behavior.

physics.bio-ph↗

arXiv2021

NMR Assignment through Linear Programming

Nuclear Magnetic Resonance (NMR) Spectroscopy is the second most used technique (after X-ray crystallography) for structural determination of proteins. A computational challenge in this technique involves solving a discrete optimization problem that assigns the resonance frequency to each atom in the protein. This paper introduces LIAN (LInear programming Assignment for NMR), a novel linear programming formulation of the problem which yields state-of-the-art results in simulated and experimental datasets.

cs.AI↗

arXiv2021

Committor functions via tensor networks

We propose a novel approach for computing committor functions, which describe transitions of a stochastic process between metastable states. The committor function satisfies a backward Kolmogorov equation, and in typical high-dimensional settings of interest, it is intractable to compute and store the solution with traditional numerical methods. By parametrizing the committor function in a matrix product state/tensor train format and using a similar representation for the equilibrium probability density, we solve the variational formulation of the backward Kolmogorov equation with linear time and memory complexity in the number of dimensions. This approach bypasses the need for sampling the equilibrium distribution, which can be difficult when the distribution has multiple modes. Numerical results demonstrate the effectiveness of the proposed method for high-dimensional problems.

math.NA↗

arXiv2021

Scalable semidefinite programming approach to variational embedding for quantum many-body problems

In quantum embedding theories, a quantum many-body system is divided into localized clusters of sites which are treated with an accurate `high-level' theory and glued together self-consistently by a less accurate `low-level' theory at the global scale. The recently introduced variational embedding approach for quantum many-body problems combines the insights of semidefinite relaxation and quantum embedding theory to provide a lower bound on the ground-state energy that improves as the cluster size is increased. The variational embedding method is formulated as a semidefinite program (SDP), which can suffer from poor computational scaling when treated with black-box solvers. We exploit the interpretation of this SDP as an embedding method to develop an algorithm which alternates parallelizable local updates of the high-level quantities with updates that enforce the low-level global constraints. Moreover, we show how translation invariance in lattice systems can be exploited to reduce the complexity of projecting a key matrix to the positive semidefinite cone.

math.OC↗

arXiv2021

A semigroup method for high dimensional committor functions based on neural network

This paper proposes a new method based on neural networks for computing the high-dimensional committor functions that satisfy Fokker-Planck equations. Instead of working with partial differential equations, the new method works with an integral formulation based on the semigroup of the differential operator. The variational form of the new formulation is then solved by parameterizing the committor function as a neural network. There are two major benefits of this new approach. First, stochastic gradient descent type algorithms can be applied in the training of the committor function without the need of computing any mixed second-order derivatives. Moreover, unlike the previous methods that enforce the boundary conditions through penalty terms, the new method takes into account the boundary conditions automatically. Numerical results are provided to demonstrate the performance of the proposed method.

math.NA↗

arXiv2020

Multiscale semidefinite programming approach to positioning problems with pairwise structure

We consider the optimization of pairwise objective functions, i.e., objective functions of the form $H(\mathbf{x}) = H(x_1,\ldots,x_N) = \sum_{1\leq i<j \leq N} H_{ij}(x_i,x_j)$ for $x_i$ in some continuous state spaces $\mathcal{X}_i$. Global optimization in this setting is generally confounded by the possible existence of spurious local minima and the impossibility of global search due to the curse of dimensionality. In this paper, we approach such problems via convex relaxation of the marginal polytope considered in graphical modeling, proceeding in a multiscale fashion which exploits the smoothness of the cost function. We show theoretically that, compared with existing methods, such an approach is advantageous even in simple settings for sensor network localization (SNL). We successfully apply our method to SNL problems, particularly difficult instances with high noise. We also validate performance on the optimization of the Lennard-Jones potential, which is plagued by the existence of many near-optimal configurations. We demonstrate that in MMR allows us to effectively explore these configurations.

math.NA↗

arXiv2020

Maximizing robustness of point-set registration by leveraging non-convexity

Point-set registration is a classical image processing problem that looks for the optimal transformation between two sets of points. In this work, we analyze the impact of outliers when finding the optimal rotation between two point clouds. The presence of outliers motivates the use of least unsquared deviation, which is a non-smooth minimization problem over non-convex domain. We compare approaches based on non-convex optimization over special orthogonal group and convex relaxations. We show that if the fraction of outliers is larger than a certain threshold, any naive convex relaxation fails to recover the ground truth rotation regardless of the sample size and dimension. In contrast, minimizing the least unsquared deviation directly over the special orthogonal group exactly recovers the ground truth rotation for any level of corruption as long as the sample size is large enough. These theoretical findings are supported by numerical simulations.

math.OC↗

arXiv2020

Semidefinite relaxation of multi-marginal optimal transport for strictly correlated electrons in second quantization

We consider the strictly correlated electron (SCE) limit of the fermionic quantum many-body problem in the second-quantized formalism. This limit gives rise to a multi-marginal optimal transport (MMOT) problem. Here the marginal state space for our MMOT problem is the binary set $\{0,1\}$, and the number of marginals is the number $L$ of sites in the model. The costs of storing and computing the exact solution of the MMOT problem both scale exponentially with respect to $L$. We propose an efficient convex relaxation which can be solved by semidefinite programming (SDP). In particular, the semidefinite constraint is only of size $2L\times 2L$. Moreover, the SDP-based method yields an approximation of the dual potential needed to the perform self-consistent field iteration in the so-called Kohn-Sham SCE framework, which, once converged, yields a lower bound for the total energy of the system. We demonstrate the effectiveness of our methods on spinless and spinful Hubbard-type models. Numerical results indicate that our relaxation methods yield tight lower bounds for the optimal cost, in the sense that the error due to the semidefinite relaxation is much smaller than the intrinsic modeling error of the Kohn-Sham SCE method. We also describe how our relaxation methods generalize to arbitrary MMOT problems with pairwise cost functions.

math.OC↗

arXiv2020

Drop-Activation: Implicit Parameter Reduction and Harmonic Regularization

Overfitting frequently occurs in deep learning. In this paper, we propose a novel regularization method called Drop-Activation to reduce overfitting and improve generalization. The key idea is to drop nonlinear activation functions by setting them to be identity functions randomly during training time. During testing, we use a deterministic network with a new activation function to encode the average effect of dropping activations randomly. Our theoretical analyses support the regularization effect of Drop-Activation as implicit parameter reduction and verify its capability to be used together with Batch Normalization (Ioffe and Szegedy 2015). The experimental results on CIFAR-10, CIFAR-100, SVHN, EMNIST, and ImageNet show that Drop-Activation generally improves the performance of popular neural network architectures for the image classification task. Furthermore, as a regularizer Drop-Activation can be used in harmony with standard training and regularization techniques such as Batch Normalization and Auto Augment (Cubuk et al. 2019). The code is available at \url{https://github.com/LeungSamWai/Drop-Activation}.

cs.LG↗

arXiv2019

Method of moments for 3-D single particle ab initio modeling with non-uniform distribution of viewing angles

Single-particle reconstruction in cryo-electron microscopy (cryo-EM) is an increasingly popular technique for determining the 3-D structure of a molecule from several noisy 2-D projections images taken at unknown viewing angles. Most reconstruction algorithms require a low-resolution initialization for the 3-D structure, which is the goal of ab initio modeling. Suggested by Zvi Kam in 1980, the method of moments (MoM) offers one approach, wherein low-order statistics of the 2-D images are computed and a 3-D structure is estimated by solving a system of polynomial equations. Unfortunately, Kam's method suffers from restrictive assumptions, most notably that viewing angles should be distributed uniformly. Often unrealistic, uniformity entails the computation of higher-order correlations, as in this case first and second moments fail to determine the 3-D structure. In the present paper, we remove this hypothesis, by permitting an unknown, non-uniform distribution of viewing angles in MoM. Perhaps surprisingly, we show that this case is statistically easier than the uniform case, as now first and second moments generically suffice to determine low-resolution expansions of the molecule. In the idealized setting of a known, non-uniform distribution, we find an efficient provable algorithm inverting first and second moments. For unknown, non-uniform distributions, we use non-convex optimization methods to solve for both the molecule and distribution.

math.NA↗

arXiv2019

Efficient construction of tensor ring representations from sampling

In this paper we propose an efficient method to compress a high dimensional function into a tensor ring format, based on alternating least-squares (ALS). Since the function has size exponential in $d$ where $d$ is the number of dimensions, we propose efficient sampling scheme to obtain $O(d)$ important samples in order to learn the tensor ring. Furthermore, we devise an initialization method for ALS that allows fast convergence in practice. Numerical examples show that to approximate a function with similar accuracy, the tensor ring format provided by the proposed method has less parameters than tensor-train format and also better respects the structure of the original function.

math.NA↗

arXiv2018

SwitchNet: a neural network model for forward and inverse scattering problems

We propose a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered field (and vice versa). The main difficulty of using a neural network for this problem is that a scatterer has a global impact on the scattered wave field, rendering typical convolutional neural network with local connections inapplicable. While it is possible to deal with such a problem using a fully connected network, the number of parameters grows quadratically with the size of the input and output data. By leveraging the inherent low-rank structure of the scattering problems and introducing a novel switching layer with sparse connections, the SwitchNet architecture uses much fewer parameters and facilitates the training process. Numerical experiments show promising accuracy in learning the forward and inverse maps between the scatterers and the scattered wave field.

math.NA↗

arXiv2018

Convex relaxation approaches for strictly correlated density functional theory

In this paper, we introduce methods from convex optimization to solve the multimarginal transport type problems arise in the context of density functional theory. Convex relaxations are used to provide outer approximation to the set of $N$-representable 2-marginals and 3-marginals, which in turn provide lower bounds to the energy. We further propose rounding schemes to obtain upper bound to the energy. Numerical experiments demonstrate a gap of the order of $10^{-3}$ to $10^{-2}$ between the upper and lower bounds. The Kantorovich potential of the multi-marginal transport problem is also approximated with a similar accuracy.

math.OC↗

arXiv2018

Solving parametric PDE problems with artificial neural networks

The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modeled into the equations as random coefficients. However, very often the variability of physical quantities derived from a PDE can be captured by a few features on the space of the coefficient fields. Based on such an observation, we propose using a neural-network (NN) based method to parameterize the physical quantity of interest as a function of input coefficients. The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE. We further demonstrate the simplicity and accuracy of the approach through notable examples of PDEs in engineering and physics.

math.NA↗

arXiv2018

Solving for high dimensional committor functions using artificial neural networks

In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes. In particular, we aim for numerical schemes for the committor function, the central object of transition path theory, which satisfies a high-dimensional Fokker-Planck equation. By working with the variational formulation of such partial differential equation and parameterizing the committor function in terms of a neural network, approximations can be obtained via optimizing the neural network weights using stochastic algorithms. The numerical examples show that moderate accuracy can be achieved for high-dimensional problems.

cs.LG↗

arXiv2017

Semidefinite Programming Approach for the Quadratic Assignment Problem with a Sparse Graph

The matching problem between two adjacency matrices can be formulated as the NP-hard quadratic assignment problem (QAP). Previous work on semidefinite programming (SDP) relaxations to the QAP have produced solutions that are often tight in practice, but such SDPs typically scale badly, involving matrix variables of dimension $n^2$ where n is the number of nodes. To achieve a speed up, we propose a further relaxation of the SDP involving a number of positive semidefinite matrices of dimension $\mathcal{O}(n)$ no greater than the number of edges in one of the graphs. The relaxation can be further strengthened by considering cliques in the graph, instead of edges. The dual problem of this novel relaxation has a natural three-block structure that can be solved via a convergent Augmented Direction Method of Multipliers (ADMM) in a distributed manner, where the most expensive step per iteration is computing the eigendecomposition of matrices of dimension $\mathcal{O}(n)$. The new SDP relaxation produces strong bounds on quadratic assignment problems where one of the graphs is sparse with reduced computational complexity and running times, and can be used in the context of nuclear magnetic resonance spectroscopy (NMR) to tackle the assignment problem.

math.OC↗

arXiv2017

Integrating NOE and RDC using sum-of-squares relaxation for protein structure determination

We revisit the problem of protein structure determination from geometrical restraints from NMR, using convex optimization. It is well-known that the NP-hard distance geometry problem of determining atomic positions from pairwise distance restraints can be relaxed into a convex semidefinite program. Often the NOE distance restraints are too imprecise and sparse for accurate structure determination. Residual dipolar coupling (RDC) measurements provide additional geometric information on the angles between atom-pair directions and axes of the principal-axis-frame. The optimization problem involving RDC is highly non-convex and requires a good initialization even within the simulated annealing framework. In this paper, we model the protein backbone as an articulated structure composed of rigid units. Determining the rotation of each rigid unit gives the full protein structure. We propose solving the non-convex optimization problems using the sum-of-squares (SOS) hierarchy. The two algorithms - RDC-SOS and RDC-NOE-SOS, have polynomial time complexity in the number of amino-acid residues and run efficiently on a standard desktop. In many instances, the proposed methods exactly recover the solution to the original non-convex optimization problem. We introduce a statistical tool, the Cramer-Rao bound (CRB), to provide an information theoretic bound on the highest resolution one can hope to achieve when determining protein structure from noisy measurements using any methodology. Our simulation results show that when the RDC measurements are corrupted by Gaussian noise of realistic variance, both SOS based algorithms attain the CRB. We successfully apply our method in a divide-and-conquer fashion to determine the structure of ubiquitin from experimental NOE and RDC measurements, achieving more accurate and faster reconstructions compared to the current state of the art.

cs.CE↗