arXiv subjects

Matthew J. Graham

Publications and source records attributed to Matthew J. Graham.

At least 217 records · Page 12Linked to original sources

arXiv2014

DAMEWARE: A web cyberinfrastructure for astrophysical data mining

Astronomy is undergoing through a methodological revolution triggered by an unprecedented wealth of complex and accurate data. The new panchromatic, synoptic sky surveys require advanced tools for discovering patterns and trends hidden behind data which are both complex and of high dimensionality. We present DAMEWARE (DAta Mining & Exploration Web Application REsource): a general purpose, web-based, distributed data mining environment developed for the exploration of large datasets, and finely tuned for astronomical applications. By means of graphical user interfaces, it allows the user to perform classification, regression or clustering tasks with machine learning methods. Salient features of DAMEWARE include its capability to work on large datasets with minimal human intervention, and to deal with a wide variety of real problems such as the classification of globular clusters in the galaxy NGC1399, the evaluation of photometric redshifts and, finally, the identification of candidate Active Galactic Nuclei in multiband photometric surveys. In all these applications, DAMEWARE allowed to achieve better results than those attained with more traditional methods. With the aim of providing potential users with all needed information, in this paper we briefly describe the technological background of DAMEWARE, give a short introduction to some relevant aspects of data mining, followed by a summary of some science cases and, finally, we provide a detailed description of a template use case.

astro-ph.IM↗

arXiv2013

A novel variability-based method for quasar selection: evidence for a rest frame ~54 day characteristic timescale

We compare quasar selection techniques based on their optical variability using data from the Catalina Real-time Transient Survey (CRTS). We introduce a new technique based on Slepian wavelet variance (SWV) that shows comparable or better performance to structure functions and damped random walk models but with fewer assumptions. Combining these methods with WISE mid-IR colors produces a highly efficient quasar selection technique which we have validated spectroscopically. The SWV technique also identifies characteristic timescales in a time series and we find a characteristic rest frame timescale of ~54 days, confirmed in the light curves of ~18000 quasars from CRTS, SDSS and MACHO data, and anticorrelated with absolute magnitude. This indicates a transition between a damped random walk and $P(f) \propto f^{-1/3}$ behaviours and is the first strong indication that a damped random walk model may be too simplistic to describe optical quasar variability.

astro-ph.CO↗

arXiv2013

Feature Selection Strategies for Classifying High Dimensional Astronomical Data Sets

The amount of collected data in many scientific fields is increasing, all of them requiring a common task: extract knowledge from massive, multi parametric data sets, as rapidly and efficiently possible. This is especially true in astronomy where synoptic sky surveys are enabling new research frontiers in the time domain astronomy and posing several new object classification challenges in multi dimensional spaces; given the high number of parameters available for each object, feature selection is quickly becoming a crucial task in analyzing astronomical data sets. Using data sets extracted from the ongoing Catalina Real-Time Transient Surveys (CRTS) and the Kepler Mission we illustrate a variety of feature selection strategies used to identify the subsets that give the most information and the results achieved applying these techniques to three major astronomical problems.

astro-ph.IM↗

arXiv2013

A comparison of period finding algorithms

This paper presents a comparison of popular period finding algorithms applied to the light curves of variable stars from the Catalina Real-time Transient Survey (CRTS), MACHO and ASAS data sets. We analyze the accuracy of the methods against magnitude, sampling rates, quoted period, quality measures (signal-to-noise and number of observations), variability, and object classes. We find that measure of dispersion-based techniques - analysis-of-variance with harmonics and conditional entropy - consistently give the best results but there are clear dependencies on object class and light curve quality. Period aliasing and identifying a period harmonic also remain significant issues. We consider the performance of the algorithms and show that a new conditional entropy-based algorithm is the most optimal in terms of completeness and speed. We also consider a simple ensemble approach and find that it performs no better than individual algorithms.

astro-ph.IM↗

arXiv2013

Using conditional entropy to identify periodicity

This paper presents a new period finding method based on conditional entropy that is both efficient and accurate. We demonstrate its applicability on simulated and real data. We find that it has comparable performance to other information-based techniques with simulated data but is superior with real data, both for finding periods and just identifying periodic behaviour. In particular, it is robust against common aliasing issues found with other period-finding algorithms.

astro-ph.IM↗

arXiv2013

Environments of Strong / Ultrastrong, Ultraviolet Fe II Emitting Quasars

We have investigated the strength of ultraviolet Fe II emission from quasars within the environments of Large Quasar Groups (LQGs) in comparison with quasars elsewhere, for 1.1 <= <= 1.7, using the DR7QSO catalogue of the Sloan Digital Sky Survey. We use the Weymann et al. W2400 equivalent width, defined between the rest-frame continuum-windows 2240-2255 and 2665-2695 Ang., as the measure of the UV Fe II emission. We find a significant shift of the W2400 distribution to higher values for quasars within LQGs, predominantly for those LQGs with 1.1 <= <= 1.5. There is a tentative indication that the shift to higher values increases with the quasar i magnitude. We find evidence that within LQGs the ultrastrong emitters with W2400 >= 45 Ang. (more precisely, ultrastrong-plus with W2400 >= 44 Ang.) have preferred nearest-neighbour separations of ~ 30-50 Mpc to the adjacent quasar of any W2400 strength. No such effect is seen for the ultrastrong emitters that are not in LQGs. The possibilities for increasing the strength of the Fe II emission appear to be iron abundance, Ly-alpha fluorescence, and microturbulence, and probably all of these operate. The dense environment of the LQGs may have led to an increased rate of star formation and an enhanced abundance of iron in the nuclei of galaxies. Similarly the dense environment may have led to more active blackholes and increased Ly-alpha fluorescence. The preferred nearest-neighbour separation for the stronger emitters would appear to suggest a dynamical component, such as microturbulence. In one particular LQG, the Huge-LQG (the largest structure known in the early universe), six of the seven strongest emitters very obviously form three pairings within the total of 73 members.

astro-ph.CO↗

arXiv2013

Machine-assisted discovery of relationships in astronomy

High-volume feature-rich data sets are becoming the bread-and-butter of 21st century astronomy but present significant challenges to scientific discovery. In particular, identifying scientifically significant relationships between sets of parameters is non-trivial. Similar problems in biological and geosciences have led to the development of systems which can explore large parameter spaces and identify potentially interesting sets of associations. In this paper, we describe the application of automated discovery systems of relationships to astronomical data sets, focussing on an evolutionary programming technique and an information-theory technique. We demonstrate their use with classical astronomical relationships - the Hertzsprung-Russell diagram and the fundamental plane of elliptical galaxies. We also show how they work with the issue of binary classification which is relevant to the next generation of large synoptic sky surveys, such as LSST. We find that comparable results to more familiar techniques, such as decision trees, are achievable. Finally, we consider the reality of the relationships discovered and how this can be used for feature selection and extraction.

astro-ph.IM↗

arXiv2012

A structure in the early universe at z ~ 1.3 that exceeds the homogeneity scale of the R-W concordance cosmology

A Large Quasar Group (LQG) of particularly large size and high membership has been identified in the DR7QSO catalogue of the Sloan Digital Sky Survey. It has characteristic size (volume^1/3) ~ 500 Mpc (proper size, present epoch), longest dimension ~ 1240 Mpc, membership of 73 quasars, and mean redshift = 1.27. In terms of both size and membership it is the most extreme LQG found in the DR7QSO catalogue for the redshift range 1.0 <= z <= 1.8 of our current investigation. Its location on the sky is ~ 8.8 deg north (~ 615 Mpc projected) of the Clowes & Campusano LQG at the same redshift, = 1.28, which is itself one of the more extreme examples. Their boundaries approach to within ~ 2 deg (~ 140 Mpc projected). This new, huge LQG appears to be the largest structure currently known in the early universe. Its size suggests incompatibility with the Yadav et al. scale of homogeneity for the concordance cosmology, and thus challenges the assumption of the cosmological principle.

astro-ph.CO↗

arXiv2012

Data challenges of time domain astronomy

Astronomy has been at the forefront of the development of the techniques and methodologies of data intensive science for over a decade with large sky surveys and distributed efforts such as the Virtual Observatory. However, it faces a new data deluge with the next generation of synoptic sky surveys which are opening up the time domain for discovery and exploration. This brings both new scientific opportunities and fresh challenges, in terms of data rates from robotic telescopes and exponential complexity in linked data, but also for data mining algorithms used in classification and decision making. In this paper, we describe how an informatics-based approach-part of the so-called "fourth paradigm" of scientific discovery-is emerging to deal with these. We review our experiences with the Palomar-Quest and Catalina Real-Time Transient Sky Surveys; in particular, addressing the issue of the heterogeneity of data associated with transient astronomical events (and other sensor networks) and how to manage and analyze it.

astro-ph.IM↗

arXiv2012

Connecting the time domain community with the Virtual Astronomical Observatory

The time domain has been identified as one of the most important areas of astronomical research for the next decade. The Virtual Observatory is in the vanguard with dedicated tools and services that enable and facilitate the discovery, dissemination and analysis of time domain data. These range in scope from rapid notifications of time-critical astronomical transients to annotating long-term variables with the latest modeling results. In this paper, we will review the prior art in these areas and focus on the capabilities that the VAO is bringing to bear in support of time domain science. In particular, we will focus on the issues involved with the heterogeneous collections of (ancillary) data associated with astronomical transients, and the time series characterization and classification tools required by the next generation of sky surveys, such as LSST and SKA.

astro-ph.IM↗

arXiv2012

Responding to the Event Deluge

We present the VOEventNet infrastructure for large-scale rapid follow-up of astronomical events, including selection, annotation, machine intelligence, and coordination of observations. The VOEvent standard is central to this vision, with distributed and replicated services rather than centralized facilities. We also describe some of the event brokers, services, and software that are connected to the network. These technologies will become more important in the coming years, with new event streams from Gaia, LOFAR, LIGO, LSST, and many others.

astro-ph.IM↗

arXiv2011

The VAO Transient Facility

The time domain community wants robust and reliable tools to enable production of and subscription to community-endorsed event notification packets (VOEvent). The VAO Transient Facility (VTF) is being designed to be the premier brokering service for the community, both collecting and disseminating observations about time-critical astronomical transients but also supporting annotations and the application of intelligent machine-learning to those observations. This distinguishes two types of activity associated with the facility: core infrastructure and user services. In this paper, we will review the prior art in both areas and describe the planned capabilities of the VTF. In particular, we will focus on scalability and quality-of-service issues required by the next generation of sky surveys, such as LSST and SKA.

astro-ph.IM↗

arXiv2011

Two close Large Quasar Groups of size ~ 350 Mpc at z ~ 1.2

The Clowes & Campusano (1991) Large Quasar Group (LQG) at = 1.28 has been re-examined using the quasar data from the DR7QSO catalogue of the Sloan Digital Sky Survey. In the 1991 discovery, the LQG impinged on the northern, southern and eastern limits of the survey. In the DR7QSO data, the western, northern and southern boundaries of the LQG remain essentially the same, but an extension eastwards of \sim 2 deg is indicated. In the DR7QSO data, the LQG has 34 members, with = 1.28. A new group of 38 members is indicated at = 1.11 and within \sim 2.0 deg of the Clowes & Campusano LQG. The characteristic sizes of these two LQGs, \sim 350-400 Mpc, appear to be only marginally consistent with the scale of homogeneity in the concordance cosmology. In addition to their intrinsic interest, these two LQGs provide locations in which to investigate early large-scale structure in galaxies and to identify high-z clusters. A method is presented for assessing the statistical significance and overdensity of groups found by linkage of points.

astro-ph.CO↗

arXiv2011

The Art of Data Science

To flourish in the new data-intensive environment of 21st century science, we need to evolve new skills. These can be expressed in terms of the systemized framework that formed the basis of mediaeval education - the trivium (logic, grammar, and rhetoric) and quadrivium (arithmetic, geometry, music, and astronomy). However, rather than focusing on number, data is the new keystone. We need to understand what rules it obeys, how it is symbolized and communicated and what its relationship to physical space and time is. In this paper, we will review this understanding in terms of the technologies and processes that it requires. We contend that, at least, an appreciation of all these aspects is crucial to enable us to extract scientific information and knowledge from the data sets which threaten to engulf and overwhelm us.

astro-ph.IM↗

arXiv2004

Grist: Grid-based Data Mining for Astronomy

The Grist project (http://grist.caltech.edu/) is developing a grid-technology based system as a research environment for astronomy with massive and complex datasets. This knowledge extraction system will consist of a library of distributed grid services controlled by a workflow system, compliant with standards emerging from the grid computing, web services, and virtual observatory communities. This new technology is being used to find high redshift quasars, study peculiar variable objects, search for transients in real time, and fit SDSS QSO spectra to measure black hole masses. Grist services are also a component of the ``hyperatlas'' project to serve high-resolution multi-wavelength imagery over the Internet. In support of these science and outreach objectives, the Grist framework will provide the enabling fabric to tie together distributed grid services in the areas of data access, federation, mining, subsetting, source extraction, image mosaicking, statistics, and visualization.

astro-ph↗

arXiv2004

Palomar-QUEST: A case study in designing sky surveys in the VO era

The advent of wide-area multicolour synoptic sky surveys is leading to data sets unprecedented in size, complexity and data throughput. VO technology offers a way to exploit these to the full but requires changes in design philosophy. The Palomar-QUEST survey is a major new survey being undertaken by Caltech, Yale, JPL and Indiana University to repeatedly observe 1/3 of the sky (~15000 sq. deg. between -27 < Dec <27 in seven passbands. Utilising the 48-inch Oschin Schmidt Telescope at the Palomar Observatory with the 112-CCD QUEST camera covering the full 4 x 4 sq. deg. field of view, it will generate \~1TB of data per month. In this paper, we review the design of QUEST as a VO resource, a federated data set and an exemplar of VO standards.

astro-ph↗

arXiv2003

A Sample of Field Ellipticals

Using well-defined selection criteria applied to the LEDA galaxy catalogue we have constructed a sample of elliptical galaxies that can be taken to lie in the field. Such criteria can easily be applied to theoretical simulations for direct comparison with observations. The variation of the number of `isolated' ellipticals with selection criteria is also investigated. A preliminary study of the environment of the field ellipticals shows that, in the mean, they are surrounded by a population of dwarf galaxies, out to projected radii of at least 500 kpc, with a radial density profile of $r^{-0.6\pm 0.2}$ and a luminosity function slope of $α\sim -1.8$. The results are compared and contrasted to the satellite population around isolated spiral galaxies.

astro-ph↗

arXiv1998

A new assessment of the completeness of quasar surveys: implications for the luminosity function

We apply a simple statistical method (Derenzo & Hildebrand 1969) to estimating the completeness of quasar surveys. It requires that an area has been covered by two or more, preferably different, selection techniques. We use three suitable data sets with separate selections from: variability and UV-excess (170 quasars); objective prism and UV-excess (141 quasars); multicolour and X-ray ({\it ROSAT,} 19 quasars). We find that, for selection by UV-excess, the common limit of $U-B \le -0.35 \pm -0.05$ leads to losses of $\sim 35%$, typically missing low-luminosity $(M_{B} \gtrsim -24.5)$ quasars, independently of redshift. Systematic incompleteness will therefore affect the new generation of large quasar surveys that select by $U-B \le -0.35$. By correcting for this incompleteness, we find, from the first data set ($B < 21.0$ and $z < 2.2$), that the evolution of the quasar luminosity function (LF) is best described by joint luminosity and density evolution. When extrapolated to $z = 0$, the LF matches that of local Seyfert galaxies better than any previous determination. The LF shows an increase in the number of low-luminosity quasars at low redshifts and of brighter quasars at intermediate redshifts, relative to the LF of Boyle et al. (1990). This result is consistent with models in which quasars fade from an initial bright phase.

astro-ph↗