arXiv subjects

Travis Desell

Publications and source records attributed to Travis Desell.

45 records · Page 3Linked to original sources

arXiv2018

Accelerating the Evolution of Convolutional Neural Networks with Node-Level Mutations and Epigenetic Weight Initialization

This paper examines three generic strategies for improving the performance of neuro-evolution techniques aimed at evolving convolutional neural networks (CNNs). These were implemented as part of the Evolutionary eXploration of Augmenting Convolutional Topologies (EXACT) algorithm. EXACT evolves arbitrary convolutional neural networks (CNNs) with goals of better discovering and understanding new effective architectures of CNNs for machine learning tasks and to potentially automate the process of network design and selection. The strategies examined are node-level mutation operations, epigenetic weight initialization and pooling connections. Results were gathered over the period of a month using a volunteer computing project, where over 225,000 CNNs were trained and evaluated across 16 different EXACT searches. The node mutation operations where shown to dramatically improve evolution rates over traditional edge mutation operations (as used by the NEAT algorithm), and epigenetic weight initialization was shown to further increase the accuracy and generalizability of the trained CNNs. As a negative but interesting result, allowing for pooling connections was shown to degrade the evolution progress. The best trained CNNs reached 99.46% accuracy on the MNIST test data in under 13,500 CNN evaluations -- accuracy comparable with some of the best human designed CNNs.

cs.NE↗

arXiv2018

A tangle of stellar streams in the north Galactic cap

Stellar halo substructures were identified using statistical photometric parallax of blue main sequence turnoff stars from fourteen Sloan Digital Sky Survey stripes in the north Galactic cap. Four structures are consistent with previous measurements of the Sagittarius dwarf tidal stream: the leading tail, the "bifurcated" stream, the trailing tail, and Stream C. The stellar overdensity in Virgo, about 15 kpc from the Sun, could arise from the crossing of the Parallel Stream and a new, candidate stream dubbed the Perpendicular Stream. The data suggests the presence of a wide stream near NGC 5466, with a distance of 5 to 15 kpc. Measurements of the flattening of the smooth stellar halo from the fourteen stripes average q=0.58, with a dispersion of 0.04.

astro-ph.GA↗

arXiv2018

Fitting the density substructure of the stellar halo with MilkyWay@home

We propose and test a method for applying statistical photometric parallax to main sequence turn off (MSTO) stars in the Sloan Digital Sky Survey (SDSS). Using simulated data, we show that if our density model is similar to the actual density distribution of our data, we can reliably determine the density model parameters of three major substructures in the Milky Way halo using the computational resources available on MilkyWay@home (a twenty parameter fit). We fit the stellar density in SDSS stripe 19 with a smooth stellar spheroid component and three major streams. One of these streams is consistent with the Sagittarius tidal stream at $21.1$ kpc away, one is consistent with the trailing tail of the Sagittarius tidal stream in the north Galactic cap at $48$ kpc away, and one is possibly part of the Virgo Overdensity at $6$ kpc away. We find the one sigma widths of these three streams to be $1.0$ kpc, $17.6$ kpc, and $6.1$ kpc, respectively. The width of the trailing tail is extremely wide ($41$ kpc full width at half maximum). This large width could have implications for the shape of the Milky Way dark matter halo. The width of the Virgo Overdensity-like structure is consistent with what we might expect for a "cloud"-like structure; analysis of additional stripes of data are needed to outline the full extent of this structure and confirm its association with the Virgo Overdensity.

astro-ph.GA↗

arXiv2017

Optimizing Long Short-Term Memory Recurrent Neural Networks Using Ant Colony Optimization to Predict Turbine Engine Vibration

This article expands on research that has been done to develop a recurrent neural network (RNN) capable of predicting aircraft engine vibrations using long short-term memory (LSTM) neurons. LSTM RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, making this approach ungeneralizable across multiple engines. In initial work, multiple LSTM RNN architectures were proposed, evaluated and compared. This research improves the performance of the most effective LSTM network design proposed in the previous work by using a promising neuroevolution method based on ant colony optimization (ACO) to develop and enhance the LSTM cell structure of the network. A parallelized version of the ACO neuroevolution algorithm has been developed and the evolved LSTM RNNs were compared to the previously used fixed topology. The evolved networks were trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. Results were obtained using MPI (Message Passing Interface) on a high performance computing (HPC) cluster, evolving 1000 different LSTM cell structures using 168 cores over 4 days. The new evolved LSTM cells showed an improvement of 1.35%, reducing prediction error from 5.51% to 4.17% when predicting excessive engine vibrations 10 seconds in the future, while at the same time dramatically reducing the number of weights from 21,170 to 11,810.

cs.NE↗

arXiv2017

Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing

This work presents a new algorithm called evolutionary exploration of augmenting convolutional topologies (EXACT), which is capable of evolving the structure of convolutional neural networks (CNNs). EXACT is in part modeled after the neuroevolution of augmenting topologies (NEAT) algorithm, with notable exceptions to allow it to scale to large scale distributed computing environments and evolve networks with convolutional filters. In addition to multithreaded and MPI versions, EXACT has been implemented as part of a BOINC volunteer computing project, allowing large scale evolution. During a period of two months, over 4,500 volunteered computers on the Citizen Science Grid trained over 120,000 CNNs and evolved networks reaching 98.32% test data accuracy on the MNIST handwritten digits dataset. These results are even stronger as the backpropagation strategy used to train the CNNs was fairly rudimentary (ReLU units, L2 regularization and Nesterov momentum) and these were initial test runs done without refinement of the backpropagation hyperparameters. Further, the EXACT evolutionary strategy is independent of the method used to train the CNNs, so they could be further improved by advanced techniques like elastic distortions, pretraining and dropout. The evolved networks are also quite interesting, showing "organic" structures and significant differences from standard human designed architectures.

cs.NE↗

arXiv2016

A Robust Asynchronous Newton Method for Massive Scale Computing Systems

Volunteer computing grids offer super-computing levels of computing power at the relatively low cost of operating a server. In previous work, the authors have shown that it is possible to take traditionally iterative evolutionary algorithms and execute them on volunteer computing grids by performing them asynchronously. The asynchronous implementations dramatically increase scalability and decrease the time taken to converge to a solution. Iterative and asynchronous optimization algorithms implemented using MPI on clusters and supercomputers, and BOINC on volunteer computing grids have been packaged together in a framework for generic distributed optimization (FGDO). This paper presents a new extension to FGDO for an asynchronous Newton method (ANM) for local optimization. ANM is resilient to heterogeneous, faulty and unreliable computing nodes and is extremely scalable. Preliminary results show that it can converge to a local optimum significantly faster than conjugate gradient descent does.

cs.DC↗

arXiv2014

MilkyWay@home: Harnessing volunteer computers to constrain dark matter in the Milky Way

MilkyWay@home is a volunteer computing project that allows people from every country in the world to volunteer their otherwise idle processors to Milky Way research. Currently, more than 25,000 people (150,000 since November 9, 2007) contribute about half a PetaFLOPS of computing power to our project. We currently run two types of applications: one application fits the spatial density profile of tidal streams using statistical photometric parallax, and the other application finds the N-body simulation parameters that produce tidal streams that best match the measured density profile of known tidal streams. The stream fitting application is well developed and is producing published results. The Sagittarius dwarf leading tidal tail has been fit, and the algorithm is currently running on the trailing tidal tail and bifurcated pieces. We will soon have a self-consistent model for the density of the smooth component of the stellar halo and the largest tidal streams. The $N$-body application has been implemented for fitting dwarf galaxy progenitor properties only, and is in the testing stages. We use an Earth-Mover Distance method to measure goodness-of-fit for density of stars along the tidal stream. We will add additional spatial dimensions as well as kinematic measures in a piecemeal fashion, with the eventual goal of fitting the orbit and parameters of the Milky Way potential (and thus the density distribution of dark matter) using multiple tidal streams.

astro-ph.GA↗

arXiv2013

A Spatial Characterization of the Sagittarius Dwarf Galaxy Tidal Tails

We measure the spatial density of F turnoff stars in the Sagittarius dwarf tidal stream, from Sloan Digital Sky Survey (SDSS) data, using statistical photometric parallax. We find a set of continuous, consistent parameters that describe the leading Sgr stream's position, direction, and width for 15 stripes in the North Galactic Cap, and 3 stripes in the South Galactic Cap. We produce a catalog of stars that has the density characteristics of the dominant leading Sgr tidal stream that can be compared with simulations. We find that the width of the leading (North) tidal tail is consistent with recent triaxial and axisymmetric halo model simulations. The density along the stream is roughly consistent common disruption models in the North, but possibly not in the South. We explore the possibility that one or more of the dominant Sgr streams has been mis-identified, and that one or more of the `bifurcated' pieces is the real Sgr tidal tail, but we do not reach definite conclusions. If two dwarf progenitors are assumed, fits to the planes of the dominant and `bifurcated' tidal tails favor an association of the Sgr dwarf spheroidal galaxy with the dominant Southern stream and the `bifurcated' stream in the North. In the North Galactic Cap, the best fit Hernquist density profile for the smooth component of the stellar halo is oblate, with a flattening parameter q = 0.53, and a scale length of r_0 = 6.73. The Southern data for both the tidal debris and the smooth component of the stellar halo do not match the model fits to the North, although the stellar halo is still overwhelmingly oblate. Finally, we verify that we can reproduce the parameter fits on the asynchronous Milkyway@home volunteer computing platform.

astro-ph.GA↗

arXiv2008

Maximum Likelihood Fitting of Tidal Streams With Application to the Sagittarius Dwarf Tidal Tails

We present a maximum likelihood method for determining the spatial properties of tidal debris and of the Galactic spheroid. With this method we characterize Sagittarius debris using stars with the colors of blue F turnoff stars in SDSS stripe 82. The debris is located at (alpha, delta, R) = (31.37 deg +/- 0.26 deg, 0.0 deg, 29.22 +/- 0.20 kpc), with a (spatial) direction given by the unit vector < -0.991 +/- 0.007 kpc, 0.042 +/- 0.033 kpc, 0.127 +/- 0.046 kpc >, in Galactocentric Cartesian coordinates, and with FWHM = 6.74 +/- 0.06 kpc. This 2.5 degee-wide stripe contains 0.9% as many F turnoff stars as the current Sagittarius dwarf galaxy. Over small spatial extent, the debris is modeled as a cylinder with a density that falls off as a Gaussian with distance from the axis, while the smooth component of the spheroid is modeled with a Hernquist profile. We assume that the absolute magnitude of F turnoff stars is distributed as a Gaussian, which is an improvement over previous methods which fixed the absolute magnitude at Mg0 = 4.2. The effectiveness and correctness of the algorithm is demonstrated on a simulated set of F turnoff stars created to mimic SDSS stripe 82 data, which shows that we have a much greater accuracy than previous studies. Our algorithm can be applied to divide the stellar data into two catalogs: one which fits the stream density profile and one with the characteristics of the spheroid. This allows us to effectively separate tidal debris from the spheroid population, both facilitating the study of the tidal stream dynamics and providing a test of whether a smooth spheroidal population exists.

astro-ph↗