arXiv subjects

Dhagash Mehta

Publications and source records attributed to Dhagash Mehta.

At least 55 records · Page 3Linked to original sources

arXiv2018

The loss surface of deep linear networks viewed through the algebraic geometry lens

By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of the deep linear neural network models. After clarifying on the various definitions of "flat" minima, we show that the geometrically flat minima, which are merely artifacts of residual continuous symmetries of the deep linear networks, can be straightforwardly removed by a generalized $L_2$ regularization. Then, we establish upper bounds on the number of isolated stationary points of these networks with the help of algebraic geometry. Using these upper bounds and utilizing a numerical algebraic geometry method, we find all stationary points of modest depth and matrix size. We show that in the presence of the non-zero regularization, deep linear networks indeed possess local minima which are not the global minima. Our computational results clarify certain aspects of the loss surfaces of deep linear networks and provide novel insights.

stat.ML↗

arXiv2018

The Loss Surface of XOR Artificial Neural Networks

Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition states (saddle points of index one), as well as the ratio of transition states to minima, grow rapidly with the number of nodes in the network. There is also a strong dependence on the regularisation parameter, with the landscape becoming more convex (fewer minima) as the regularisation term increases. We demonstrate that in our formulation, stationary points for networks with $N_h$ hidden nodes, including the minimal network required to fit the XOR data, are also stationary points for networks with $N_{h} +1$ hidden nodes when all the weights involving the additional nodes are zero. Hence, smaller networks optimized to train the XOR data are embedded in the landscapes of larger networks. Our results clarify certain aspects of the classification and sensitivity (to perturbations in the input data) of minima and saddle points for this system, and may provide insight into dropout and network compression.

stat.ML↗

arXiv2018

Optimal Configurations in Coverage Control with Polynomial Costs

We revisit the static coverage control problem for placement of vehicles with simple motion on the real line, under the assumption that the cost is a polynomial function of the locations of the vehicles. The main contribution of this paper is to demonstrate the use of tools from numerical algebraic geometry, in particular, a numerical polynomial homotopy continuation method that guarantees to find all solutions of polynomial equations, in order to characterize the \emph{global minima} for the coverage control problem. The results are then compared against a classic distributed approach involving the use of Lloyd descent, which is known to converge only to a local minimum under certain technical conditions.

eess.SY↗

arXiv2017

Properties of Kinetic Transition Networks for Atomic Clusters and Glassy Solids

A database of minima and transition states corresponds to a network where the minima represent nodes and the transition states correspond to edges between the pairs of minima they connect via steepest-descent paths. Here we construct networks for small clusters bound by the Morse potential for a selection of physically relevant parameters, in two and three dimensions. The properties of these unweighted and undirected networks are analysed to examine two features: whether they are small-world, where the shortest path between nodes involves only a small number or edges; and whether they are scale-free, having a degree distribution that follows a power law. Small-world character is present, but statistical tests show that a power law is not a good fit, so the networks are not scale-free. These results for clusters are compared with the corresponding properties for the molecular and atomic structural glass formers ortho-terphenyl and binary Lennard-Jones. These glassy systems do not show small-world properties, suggesting that such behaviour is linked to the structure-seeking landscapes of the Morse clusters.

cond-mat.dis-nn↗

arXiv2017

Counting equilibria of the Kuramoto model using birationally invariant intersection index

Synchronization in networks of interconnected oscillators is a fascinating phenomenon that appear naturally in many independent fields of science and engineering. A substantial amount of work has been devoted to understanding all possible synchronization configurations on a given network. In this setting, a key problem is to determine the total number of such configurations. Through an algebraic formulation, for tree and cycle graphs, we provide an upper bound on this number using the birationally invariant intersection index of a system of rational functions on a toric variety.

math.AG↗

arXiv2017

Fixed Points of Belief Propagation -- An Analysis via Polynomial Homotopy Continuation

Belief propagation (BP) is an iterative method to perform approximate inference on arbitrary graphical models. Whether BP converges and if the solution is a unique fixed point depends on both the structure and the parametrization of the model. To understand this dependence it is interesting to find \emph{all} fixed points. In this work, we formulate a set of polynomial equations, the solutions of which correspond to BP fixed points. To solve such a nonlinear system we present the numerical polynomial-homotopy-continuation (NPHC) method. Experiments on binary Ising models and on error-correcting codes show how our method is capable of obtaining all BP fixed points. On Ising models with fixed parameters we show how the structure influences both the number of fixed points and the convergence properties. We further asses the accuracy of the marginals and weighted combinations thereof. Weighting marginals with their respective partition function increases the accuracy in all experiments. Contrary to the conjecture that uniqueness of BP fixed points implies convergence, we find graphs for which BP fails to converge, even though a unique fixed point exists. Moreover, we show that this fixed point gives a good approximation, and the NPHC method is able to obtain this fixed point.

stat.ML↗

arXiv2017

Locating Power Flow Solution Space Boundaries: A Numerical Polynomial Homotopy Approach

The solution space of any set of power flow equations may contain different number of real-valued solutions. The boundaries that separate these regions are referred to as power flow solution space boundaries. Knowledge of these boundaries is important as they provide a measure for voltage stability. Traditionally, continuation based methods have been employed to compute these boundaries on the basis of initial guesses for the solution. However, with rapid growth of renewable energy sources these boundaries will be increasingly affected by variable parameters such as penetration levels, locations of the renewable sources, and voltage set-points, making it difficult to generate an initial guess that can guarantee all feasible solutions for the power flow problem. In this paper we solve this problem by applying a numerical polynomial homotopy based continuation method. The proposed method guarantees to find all solution boundaries within a given parameter space up to a chosen level of discretization, independent of any initial guess. Power system operators can use this computational tool conveniently to plan the penetration levels of renewable sources at different buses. We illustrate the proposed method through simulations on 3-bus and 10-bus power system examples with renewable generation.

eess.SY↗

arXiv2017

Perspective: Energy Landscapes for Machine Learning

Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular potential energy landscapes can be applied to these machine learning landscapes to gain new insight into the solution space involved in training and the nature of the corresponding predictions. In particular, we can define quantities analogous to molecular structure, thermodynamics, and kinetics, and relate these emergent properties to the structure of the underlying landscape. This Perspective aims to describe these analogies with examples from recent applications, and suggest avenues for new interdisciplinary research.

stat.ML↗

arXiv2016

Kinetic Transition Networks for the Thomson Problem and Smale's 7th Problem

The Thomson Problem, arrangement of identical charges on the surface of a sphere, has found many applications in physics, chemistry and biology. Here we show that the energy landscape of the Thomson Problem for $N$ particles with $N=132, 135, 138, 141, 144, 147$ and $150$ is single funnelled, characteristic of a structure-seeking organisation where the global minimum is easily accessible. Algorithmically constructing starting points close to the global minimum of such a potential with spherical constraints is one of Smale's 18 unsolved problems in mathematics for the 21st century because it is important in the solution of univariate and bivariate random polynomial equations. By analysing the kinetic transition networks, we show that a randomly chosen minimum is in fact always `close' to the global minimum in terms of the number of transition states that separate them, a characteristic of small world networks.

cond-mat.soft↗

arXiv2016

Decomposing the parameter space of biological networks via a numerical discriminant approach

Many systems in biology, physics and engineering can be described by systems of ordinary differential equation containing many parameters. When studying the dynamic behavior of these large, nonlinear systems, it is useful to identify and characterize the steady-state solutions as the model parameters vary, a technically challenging problem in a high-dimensional parameter landscape. Rather than simply determining the number and stability of steady-states at distinct points in parameter space, we decompose the parameter space into finitely many regions, the steady-state solutions being consistent within each distinct region. From a computational algebraic viewpoint, the boundary of these regions is contained in the discriminant locus. We develop global and local numerical algorithms for constructing the discriminant locus and classifying the parameter landscape. We showcase our numerical approaches by applying them to molecular and cell-network models.

q-bio.MN↗

arXiv2016

Investigating the Maximum Number of Real Solutions to the Power Flow Equations: Analysis of Lossless Four-Bus Systems

The power flow equations model the steady-state relationship between the power injections and voltage phasors in an electric power system. By separating the real and imaginary components of the voltage phasors, the power flow equations can be formulated as a system of quadratic polynomials. Only the real solutions to these polynomial equations are physically meaningful. This paper focuses on the maximum number of real solutions to the power flow equations. An upper bound on the number of real power flow solutions commonly used in the literature is the maximum number of complex solutions. There exist two- and three-bus systems for which all complex solutions are real. It is an open question whether this is also the case for larger systems. This paper investigates four-bus systems using techniques from numerical algebraic geometry and conjectures a negative answer to this question. In particular, this paper studies lossless, four-bus systems composed of PV buses connected by lines with arbitrary susceptances. Computing the Galois group, which is degenerate, enables conversion of the problem of counting the number of real solutions to the power flow equations into counting the number of positive roots of a univariate sextic polynomial. From this analysis, it is conjectured that the system has at most 16 real solutions, which is strictly less than the maximum number of complex solutions, namely 20. We also provide explicit parameter values where this system has 16 real solutions so that the conjectured upper bound is achievable.

math.OC↗

arXiv2015

On the Network Topology Dependent Solution Count of the Algebraic Load Flow Equations

A large amount of research activity in power systems areas has focused on developing computational methods to solve load flow equations where a key question is the maximum number of isolated solutions.Though several concrete upper bounds exist, recent studies have hinted that much sharper upper bounds that depend the topology of underlying power networks may exist. This paper establishes such a topology dependent solution bound which is actually the best possible bound in the sense that it is always attainable. We also develop a geometric construction called adjacency polytope which accurately captures the topology of the underlying power network and is immensely useful in the computation of the solution bound. Finally we highlight the significant implications of the development of such solution bound in solving load flow equations.

math.OC↗

arXiv2015

Recent Advances in Computational Methods for the Power Flow Equations

The power flow equations are at the core of most of the computations for designing and operating electric power systems. The power flow equations are a system of multivariate nonlinear equations which relate the power injections and voltages in a power system. A plethora of methods have been devised to solve these equations, starting from Newton-based methods to homotopy continuation and other optimization-based methods. While many of these methods often efficiently find a high-voltage, stable solution due to its large basin of attraction, most of the methods struggle to find low-voltage solutions which play significant role in certain stability-related computations. While we do not claim to have exhausted the existing literature on all related methods, this tutorial paper introduces some of the recent advances in methods for solving power flow equations to the wider power systems community as well as bringing attention from the computational mathematics and optimization communities to the power systems problems. After briefly reviewing some of the traditional computational methods used to solve the power flow equations, we focus on three emerging methods: the numerical polynomial homotopy continuation method, Groebner basis techniques, and moment/sum-of-squares relaxations using semidefinite programming. In passing, we also emphasize the importance of an upper bound on the number of solutions of the power flow equations and review the current status of research in this direction.

math.OC↗

arXiv2015

Toward Topologically Based Upper Bounds on the Number of Power Flow Solutions

The power flow equations, which relate power injections and voltage phasors, are at the heart of many electric power system computations. While Newton-based methods typically find the "high-voltage" solution to the power flow equations, which is of primary interest, there are potentially many "low-voltage" solutions that are useful for certain analyses. This paper addresses the number of solutions to the power flow equations. There exist upper bounds on the number of power flow solutions; however, there is only limited work regarding bounds that are functions of network topology. This paper empirically explores the relationship between the network topology, as characterized by the maximal cliques, and the number of power flow solutions. To facilitate this analysis, we use a numerical polynomial homotopy continuation approach that is guaranteed to find all complex solutions to the power flow equations. The number of solutions obtained from this approach upper bounds the number of real solutions. Testing with many small networks informs the development of upper bounds that are functions of the network topology. Initial results include empirically derived expressions for the maximum number of solutions for certain classes of network topologies.

math.OC↗

arXiv2015

An index-resolved fixed-point homotopy and potential energy landscapes

Stationary points (SPs) of the potential energy landscapes can be classified by their Morse index, i.e., the number of negative eigenvalues of the Hessian evaluated at the SPs. In understanding chemical clusters through their potential energy landscapes, only SPs of a particular Morse index are needed. We propose a modification of the "fixed-point homotopy" method which can be used to directly target stationary points of a specified Morse index. We demonstrate the effectiveness of our approach by applying it to the Lennard-Jones clusters.

cond-mat.soft↗

arXiv2015

Statistics of Stationary Points of Random Finite Polynomial Potentials

The stationary points (SPs) of the potential energy landscapes (PELs) of multivariate random potentials (RPs) have found many applications in many areas of Physics, Chemistry and Mathematical Biology. However, there are few reliable methods available which can find all the SPs accurately. Hence, one has to rely on indirect methods such as Random Matrix theory. With a combination of the numerical polynomial homotopy continuation method and a certification method, we obtain all the certified SPs of the most general polynomial RP for each sample chosen from the Gaussian distribution with mean 0 and variance 1. While obtaining many novel results for the finite size case of the RP, we also discuss the implications of our results on mathematics of random systems and string theory landscapes.

cond-mat.stat-mech↗

arXiv2015

A Collection of Challenging Optimization Problems in Science, Engineering and Economics

Function optimization and finding simultaneous solutions of a system of nonlinear equations (SNE) are two closely related and important optimization problems. However, unlike in the case of function optimization in which one is required to find the global minimum and sometimes local minima, a database of challenging SNEs where one is required to find stationary points (extrama and saddle points) is not readily available. In this article, we initiate building such a database of important SNE (which also includes related function optimization problems), arising from Science, Engineering and Economics. After providing a short review of the most commonly used mathematical and computational approaches to find solutions of such systems, we provide a preliminary list of challenging problems by writing the Mathematical formulation down, briefly explaning the origin and importance of the problem and giving a short account on the currently known results, for each of the problems. We anticipate that this database will not only help benchmarking novel numerical methods for solving SNEs and function optimization problems but also will help advancing the corresponding research areas.

math.NA↗

arXiv2015

Global Structure of Curves from Generalized Unitarity Cut of Three-loop Diagrams

This paper studies the global structure of algebraic curves defined by generalized unitarity cut of four-dimensional three-loop diagrams with eleven propagators. The global structure is a topological invariant that is characterized by the geometric genus of the algebraic curve. We use the Riemann-Hurwitz formula to compute the geometric genus of algebraic curves with the help of techniques involving convex hull polytopes and numerical algebraic geometry. Some interesting properties of genus for arbitrary loop orders are also explored where computing the genus serves as an initial step for integral or integrand reduction of three-loop amplitudes via an algebraic geometric approach.

hep-th↗