arXiv subjects

David J. Wales

Publications and source records attributed to David J. Wales.

At least 37 records · Page 2Linked to original sources

arXiv2018

The Loss Surface of XOR Artificial Neural Networks

Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition states (saddle points of index one), as well as the ratio of transition states to minima, grow rapidly with the number of nodes in the network. There is also a strong dependence on the regularisation parameter, with the landscape becoming more convex (fewer minima) as the regularisation term increases. We demonstrate that in our formulation, stationary points for networks with $N_h$ hidden nodes, including the minimal network required to fit the XOR data, are also stationary points for networks with $N_{h} +1$ hidden nodes when all the weights involving the additional nodes are zero. Hence, smaller networks optimized to train the XOR data are embedded in the landscapes of larger networks. Our results clarify certain aspects of the classification and sensitivity (to perturbations in the input data) of minima and saddle points for this system, and may provide insight into dropout and network compression.

stat.ML↗

arXiv2018

From Sticky-Hard-Sphere to Lennard-Jones-Type Clusters

A relation $\mathcal{M}_{\mathrm{SHS}\to\mathrm{LJ}}$ between the set of non-isomorphic sticky hard sphere clusters $\mathcal{M}_\mathrm{SHS}$ and the sets of local energy minima $\mathcal{M}_{LJ}$ of the $(m,n)$-Lennard-Jones potential $V^\mathrm{LJ}_{mn}(r) = \frac{\varepsilon}{n-m} [ m r^{-n} - n r^{-m} ]$ is established. The number of nonisomorphic stable clusters depends strongly and nontrivially on both $m$ and $n$, and increases exponentially with increasing cluster size $N$ for $N \gtrsim 10$. While the map from $\mathcal{M}_\mathrm{SHS}\to \mathcal{M}_{\mathrm{SHS}\to\mathrm{LJ}}$ is non-injective and non-surjective, the number of Lennard-Jones structures missing from the map is relatively small for cluster sizes up to $N=13$, and most of the missing structures correspond to energetically unfavourable minima even for fairly low $(m,n)$. Furthermore, even the softest Lennard-Jones potential predicts that the coordination of 13 spheres around a central sphere is problematic (the Gregory-Newton problem). A more realistic extended Lennard-Jones potential chosen from coupled-cluster calculations for a rare gas dimer leads to a substantial increase in the number of nonisomorphic clusters, even though the potential curve is very similar to a (6,12)-Lennard-Jones potential.

physics.atm-clus↗

arXiv2017

Perspective: Energy Landscapes for Machine Learning

Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular potential energy landscapes can be applied to these machine learning landscapes to gain new insight into the solution space involved in training and the nature of the corresponding predictions. In particular, we can define quantities analogous to molecular structure, thermodynamics, and kinetics, and relate these emergent properties to the structure of the underlying landscape. This Perspective aims to describe these analogies with examples from recent applications, and suggest avenues for new interdisciplinary research.

stat.ML↗

arXiv2017

Energetically favoured defects in dense packings of particles on spherical surfaces

The dense packing of interacting particles on spheres has proved to be a useful model for virus capsids and colloidosomes. Indeed, icosahedral symmetry observed in virus capsids corresponds to potential energy minima that occur for magic numbers of, e.g., 12, 32 and 72 identical Lennard-Jones particles, for which the packing has exactly the minimum number of twelve five-fold defects. It is unclear, however, how stable these structures are against thermal agitation. We investigate this property by means of basin-hopping global optimisation and Langevin dynamics for particle numbers between ten and one hundred. An important measure is the number and type of point defects, that is, particles that do not have six nearest neighbours. We find that small icosahedral structures are the most robust against thermal fluctuations, exhibiting fewer excess defects and rearrangements for a wide temperature range. Furthermore, we provide evidence that excess defects appearing at low non-zero temperatures lower the potential energy at the expense of entropy. At higher temperatures defects are, as expected, thermally excited and thus entropically stabilised. If we replace the Lennard-Jones potential by a very short-ranged (Morse) potential, which is arguably more appropriate for colloids and virus capsid proteins, we find that the same particle numbers give a minimum in the potential energy, although for larger particle numbers these minima correspond to different packings. Furthermore, defects are more difficult to excite thermally for the short-ranged potential, suggesting that the short-ranged interaction further stabilises equilibrium structures.

cond-mat.soft↗

arXiv2016

Structural analysis of high-dimensional basins of attraction

We propose an efficient Monte Carlo method for the computation of the volumes of high-dimensional bodies with arbitrary shape. We start with a region of known volume within the interior of the manifold and then use the multistate Bennett acceptance-ratio method to compute the dimensionless free-energy difference between a series of equilibrium simulations performed within this object. The method produces results that are in excellent agreement with thermodynamic integration, as well as a direct estimate of the associated statistical uncertainties. The histogram method also allows us to directly obtain an estimate of the interior radial probability density profile, thus yielding useful insight into the structural properties of such a high-dimensional body. We illustrate the method by analyzing the effect of structural disorder on the basins of attraction of mechanically stable packings of soft repulsive spheres.

cond-mat.dis-nn↗

arXiv2016

Kinetic Transition Networks for the Thomson Problem and Smale's 7th Problem

The Thomson Problem, arrangement of identical charges on the surface of a sphere, has found many applications in physics, chemistry and biology. Here we show that the energy landscape of the Thomson Problem for $N$ particles with $N=132, 135, 138, 141, 144, 147$ and $150$ is single funnelled, characteristic of a structure-seeking organisation where the global minimum is easily accessible. Algorithmically constructing starting points close to the global minimum of such a potential with spherical constraints is one of Smale's 18 unsolved problems in mathematics for the 21st century because it is important in the solution of univariate and bivariate random polynomial equations. By analysing the kinetic transition networks, we show that a randomly chosen minimum is in fact always `close' to the global minimum in terms of the number of transition states that separate them, a characteristic of small world networks.

cond-mat.soft↗

arXiv2016

Turning intractable counting into sampling: computing the configurational entropy of three-dimensional jammed packings

We report a numerical calculation of the total number of disordered jammed configurations $Ω$ of $N$ repulsive, three-dimensional spheres in a fixed volume $V$. To make these calculations tractable, we increase the computational efficiency of the approach of Xu et al. (Phys. Rev. Lett. 106, 245502 (2011)) and Asenjo et al. (Phys. Rev. Lett. 112, 098002 (2014)) and we extend the method to allow computation of the configurational entropy as a function of pressure. The approach that we use computes the configurational entropy by sampling the absolute volume of basins of attraction of the stable packings in the potential energy landscape. We find a surprisingly strong correlation between the pressure of a configuration and the volume of its basin of attraction in the potential energy landscape. This relation is well described by a power law. Our methodology to compute the number of minima in the potential energy landscape should be applicable to a wide range of other enumeration problems in statistical physics, string theory, cosmology and machine learning, that aim to find the distribution of the extrema of a scalar cost function that depends on many degrees of freedom.

cond-mat.stat-mech↗

arXiv2014

Certification and the Potential Energy Landscape

Typically, there is no guarantee that a numerical approximation obtained using standard nonlinear equation solvers is indeed an actual solution, meaning that it lies in the quadratic convergence basin. Instead, it may lie only in the linear convergence basin, or even in a chaotic region, and hence not converge to the corresponding stationary point when further optimization is attempted. In some cases, these non-solutions could be misleading. Proving that a numerical approximation will quadratically converge to a stationary point is termed \textit{certification}. In this report, we provide details of how Smale's $α$-theory can be used to certify numerically obtained stationary points of a potential energy landscape, providing a \textit{mathematical proof} that the numerical approximation does indeed correspond to an actual stationary point, independent of the precision employed.

physics.chem-ph↗

arXiv2014

Analysing kinetic transition networks for rare events

The graph transformation approach is a recently proposed method for computing mean first passage times, rates, and committor probabilities for kinetic transition networks. Here we compare the performance to existing linear algebra methods, focusing on large, sparse networks. We show that graph transformation provides a much more robust framework, succeeding when numerical precision issues cause the other methods to fail completely. These are precisely the situations that correspond to rare event dynamics for which the graph transformation was introduced.

cond-mat.stat-mech↗

arXiv2014

Design principles for Bernal spirals and helices with tunable pitch

Using the framework of potential energy landscape theory, we describe two in silico designs for self-assembling helical colloidal superstructures based upon dipolar dumbbells and Janus-type building blocks, respectively. Helical superstructures with controllable pitch length are obtained using external magnetic field driven assembly of asymmetric dumbbells involving screened electrostatic as well as magnetic dipolar interactions. The pitch of the helix is tuned by modulating the Debye screening length over an experimentally accessible range. The second design is based on building blocks composed of rigidly linked spheres with short-range anisotropic interactions, which are predicted to self-assemble into Bernal spirals. These spirals are quite flexible, and longer helices undergo rearrangements via cooperative, hinge-like moves, in agreement with experiment.

cond-mat.soft↗

arXiv2014

Superposition Enhanced Nested Sampling

The theoretical analysis of many problems in physics, astronomy and applied mathematics requires an efficient numerical exploration of multimodal parameter spaces that exhibit broken ergodicity. Monte Carlo methods are widely used to deal with these classes of problems, but such simulations suffer from a ubiquitous sampling problem: the probability of sampling a particular state is proportional to its entropic weight. Devising an algorithm capable of sampling efficiently the full phase space is a long-standing problem. Here we report a new hybrid method for the exploration of multimodal parameter spaces exhibiting broken ergodicity. Superposition enhanced nested sampling (SENS) combines the strengths of global optimization with the unbiased/athermal sampling of nested sampling, greatly enhancing its efficiency with no additional parameters. We report extensive tests of this new approach for atomic clusters that are known to have energy landscapes for which conventional sampling schemes suffer from broken ergodicity. We also introduce a novel parallelization algorithm for nested sampling.

cond-mat.stat-mech↗

arXiv2013

Potential Energy Landscapes for the 2D XY Model: Minima, Transition States and Pathways

We describe a numerical study of the potential energy landscape for the two-dimensional XY model (with no disorder), considering up to 100 spins and CPU and GPU implementations of local optimization, focusing on minima and saddles of index one (transition states). We examine both periodic and anti-periodic boundary conditions, and show that the number of stationary points located increases exponentially with increasing lattice size. The corresponding disconnectivity graphs exhibit funneled landscapes; the global minima are readily located because they exhibit relatively large basins of attraction compared to the higher energy minima as the lattice size increases.

cond-mat.stat-mech↗

arXiv2013

Visualizing Basins of Attraction for Different Minimization Algorithms

We report a study of the basins of attraction for potential energy minima defined by different minimisation algorithms for an atomic system. We find that whereas some minimisation algorithms produce compact basins, others produce basins with complex boundaries or basins consisting of disconnected parts. Such basins deviate from the "correct" basin of attraction defined by steepest-descent pathways, and the differences can be controlled to some extent by adjustment of the maximum step size. The choice of the most convenient minimisation algorithm depends on the problem in hand. We show that while L-BFGS is the fastest minimiser, the FIRE algorithm is also quite fast, and can lead to less fragmented basins of attraction.

physics.comp-ph↗

arXiv2013

Energy landscapes, structural topologies and rearrangement mechanisms in clusters of dipolar particles

Clusters of spherical particles with isotropic attraction favour compact structures that maximise the number of energetically optimal nearest-neighbour interactions. In contrast, dipolar interactions lead to the formation of chains with a low coordination number. When both isotropic and dipolar interactions are present, the competition between them can lead to intricate knot, link and coil structures. Here, we investigate how these structures may self-organise and interconvert in clusters bound by the Stockmayer potential (Lennard-Jones plus point dipole). We map out the low-lying region of the energy landscape using disconnectivity graphs to follow how it evolves as the strength of the dipolar interactions increases. From comprehensive surveys of isomerisation pathways, we identify a number of rearrangement mechanisms that allow the topology of chain-like structures to interconvert.

cond-mat.soft↗

arXiv2013

Defect Motifs for Constant Mean Curvature Surfaces

The energy landscapes of electrostatically charged particles embedded on constant mean curvature surfaces are analysed for a wide range of system size, curvature, and interaction potentials. The surfaces are taken to be rigid, and the basin-hopping method is used to locate the putative global minimum structures. The defect motifs favoured by potential energy agree with experimental observations for colloidal systems: extended defects (scars and pleats) for weakly positive and negative Gaussian curvatures, and isolated defects for strongly negative Gaussian curvatures. Near the phase boundary between these regimes the two motifs are in strong competition, as evidenced from the appearance of distinct funnels in the potential energy landscape. We also report a novel defect motif consisting of pentagon pairs.

cond-mat.soft↗

arXiv2013

Certifying the Potential Energy Landscape

It is highly desirable for a numerical approximation of a stationary point for a potential energy landscape to lie in the quadratic convergence basin of that stationary point. However, it is possible that an approximation may lie only in the linear convergence basin, or even in a chaotic region, and hence not converge to the actual stationary point when further optimization is attempted. Proving that a numerical approximation will quadratically converge to the associated stationary point is termed certifying the numerical approximation. We employ Smale's α-theory to stationary points, providing a certification that serves as a mathematical proof that the numerical approximation does indeed correspond to an actual stationary point, independent of the precision employed. As a practical example, employing recently developed certification algorithms, we show how the α-theory can be used to certify all the known minima and transition states of Lennard-Jones LJ$_{N}$ atomic clusters for N = 7, ...,14.

cond-mat.soft↗

arXiv2011

Hydrogen on graphene under stress: Molecular dissociation and gap opening

Density functional calculations are employed to study the molecular dissociation of hydrogen on graphene, the diffusion of chemisorbed atomic species, and the electronic properties of the resulting hydrogen on graphene system. Our results show that applying stress to the graphene substrate can lower the barrier to dissociation of molecular hydrogen by a factor of six, and change the process from endothermic to exothermic. These values for the barrier and the heat of reaction, unlike the zero stress values, are compatible with the time scales observed in experiments. Diffusion, on the other hand, is not greatly modified by stress. We analyse the electronic structure for configurations relevant to molecular dissociation and adsorption of atomic hydrogen on a graphene single layer. An absolute band gap of 0.5 eV is found for the equilibrium optimum configuration for a narrow range of coverages ($θ\approx 0.25$). This value is in good agreement with experiment [Elias et al., Science {\bf 323}, 610 (2009)].

cond-mat.mtrl-sci↗

arXiv2010

The Energy Landscape, Folding Pathways and the Kinetics of a Knotted Protein

The folding pathway and rate coefficients of the folding of a knotted protein are calculated for a potential energy function with minimal energetic frustration. A kinetic transition network is constructed using the discrete path sampling approach, and the resulting potential energy surface is visualized by constructing disconnectivity graphs. Owing to topological constraints, the low-lying portion of the landscape consists of three distinct regions, corresponding to the native knotted state and to configurations where either the N- or C-terminus is not yet folded into the knot. The fastest folding pathways from denatured states exhibit early formation of the N-terminus portion of the knot and a rate-determining step where the C-terminus is incorporated. The low-lying minima with the N-terminus knotted and the C-terminus free therefore constitute an off-pathway intermediate for this model. The insertion of both the N- and C-termini into the knot occur late in the folding process, creating large energy barriers that are the rate limiting steps in the folding process. When compared to other protein folding proteins of a similar length, this system folds over six orders of magnitude more slowly.

q-bio.BM↗