arXiv subjects

Christopher J. Shallue

Publications and source records attributed to Christopher J. Shallue.

16 recordsLinked to original sources

arXiv2025

What Hawking Radiation Looks Like as You Fall into a Black Hole

We study the measurements of a freely falling Unruh-DeWitt particle detector near the horizon of a semiclassical Schwarzschild black hole. Our results show that the detector's response increases smoothly as it approaches and crosses the horizon in both the Hartle-Hawking and Unruh vacua. However, these measurements are dominated by the effects of switching the detector on and off, rather than by the detection of Hawking radiation particles. We demonstrate that a freely falling Unruh-DeWitt detector cannot directly measure Hawking radiation near the horizon because the time required for thermalization is longer than the time spent near the horizon. We propose an operational definition of the effective temperature along an infalling trajectory based on measurements by a particle detector. Using this method, we find that the effective temperature measured by a freely falling observer in the Hartle-Hawking vacuum increases smoothly from the Hawking temperature far from the horizon to twice the Hawking temperature at the horizon, and continues to rise into the interior of the black hole. This effective temperature closely matches an analytical prediction derived by embedding Schwarzschild spacetime into a higher-dimensional Minkowski space, suggesting that further exploration of higher-dimensional embeddings could provide new insights into the near-horizon behavior of black holes.

gr-qc↗

arXiv2025

Warm Hawking Relics From Primordial Black Hole Domination

We study the cosmological impact of warm, dark-sector relic particles produced as Hawking radiation in a primordial-black-hole-dominated universe before big bang nucleosynthesis. If these dark-sector particles are stable, they would survive to the present day as "Hawking relics" and modify the growth of cosmological structure. We show that such relics are produced with much larger momenta, but in smaller quantities than the familiar thermal relics considered in standard cosmology. Consequently, Hawking relics with keV-MeV masses affect the growth of large-scale structure in a similar way to eV-scale thermal relics like massive neutrinos. We model their production and evolution, and show that their momentum distributions are broader than comparable relics with thermal distributions. Warm Hawking relics affect the growth of cosmological perturbations and we constrain their abundance to be less than $2\%$ of the dark matter over a broad range of their viable parameter space. Finally, we examine how future measurements of the matter power spectrum can distinguish Hawking relics from thermal particles.

astro-ph.CO↗

arXiv2024

Characterization of K2-167 b and CALM, a new stellar activity mitigation method

We report precise radial velocity (RV) observations of HD 212657 (= K2-167), a star shown by K2 to host a transiting sub-Neptune-sized planet in a 10 day orbit. Using Transiting Exoplanet Survey Satellite (TESS) photometry, we refined the planet parameters, especially the orbital period. We collected 74 precise RVs with the HARPS-N spectrograph between August 2015 and October 2016. Although this planet was first found to transit in 2015 and validated in 2018, excess RV scatter originally limited mass measurements. Here, we measure a mass by taking advantage of reductions in scatter from updates to the HARPS-N Data Reduction System (2.3.5) and our new activity mitigation method called CCF Activity Linear Model (CALM), which uses activity-induced line shape changes in the spectra without requiring timing information. Using the CALM framework, we performed a joint fit with RVs and transits using EXOFASTv2 and find $M_p = 6.3_{-1.4}^{+1.4}$ $M_{\oplus}$ and $R_p = 2.33^{+0.17}_{-0.15}$ $R_{\oplus}$, which places K2-167 b at the upper edge of the radius valley. We also find hints of a secondary companion at a $\sim$ 22 day period, but confirmation requires additional RVs. Although characterizing lower-mass planets like K2-167 b is often impeded by stellar variability, these systems especially help probe the formation physics (i.e. photoevaporation, core-powered mass loss) of the radius valley. In the future, CALM or similar techniques could be widely applied to FGK-type stars, help characterize a population of exoplanets surrounding the radius valley, and further our understanding of their formation.

astro-ph.EP↗

arXiv2023

Reconstructing Cosmological Initial Conditions from Late-Time Structure with Convolutional Neural Networks

We present a method to reconstruct the initial linear-regime matter density field from the late-time non-linearly evolved density field in which we channel the output of standard first-order reconstruction to a convolutional neural network (CNN). Our method shows dramatic improvement over the reconstruction of either component alone. We show why CNNs are not well-suited for reconstructing the initial density directly from the late-time density: CNNs are local models, but the relationship between initial and late-time density is not local. Our method leverages standard reconstruction as a preprocessing step, which inverts bulk gravitational flows sourced over very large scales, transforming the residual reconstruction problem from long-range to local and making it ideally suited for a CNN. We develop additional techniques to account for redshift distortions, which warp the density fields measured by galaxy surveys. Our method improves the range of scales of high-fidelity reconstruction by a factor of 2 in wavenumber above standard reconstruction, corresponding to a factor of 8 increase in the number of well-reconstructed modes. In addition, our method almost completely eliminates the anisotropy caused by redshift distortions. As galaxy surveys continue to map the Universe in increasingly greater detail, our results demonstrate the opportunity offered by CNNs to untangle the non-linear clustering at intermediate scales more accurately than ever before.

astro-ph.CO↗

arXiv2022

Identifying Exoplanets with Deep Learning. IV. Removing Stellar Activity Signals from Radial Velocity Measurements Using Neural Networks

Exoplanet detection with precise radial velocity (RV) observations is currently limited by spurious RV signals introduced by stellar activity. We show that machine learning techniques such as linear regression and neural networks can effectively remove the activity signals (due to starspots/faculae) from RV observations. Previous efforts focused on carefully filtering out activity signals in time using modeling techniques like Gaussian Process regression (e.g. Haywood et al. 2014). Instead, we systematically remove activity signals using only changes to the average shape of spectral lines, and no information about when the observations were collected. We trained our machine learning models on both simulated data (generated with the SOAP 2.0 software; Dumusque et al. 2014) and observations of the Sun from the HARPS-N Solar Telescope (Dumusque et al. 2015; Phillips et al. 2016; Collier Cameron et al. 2019). We find that these techniques can predict and remove stellar activity from both simulated data (improving RV scatter from 82 cm/s to 3 cm/s) and from more than 600 real observations taken nearly daily over three years with the HARPS-N Solar Telescope (improving the RV scatter from 1.753 m/s to 1.039 m/s, a factor of ~ 1.7 improvement). In the future, these or similar techniques could remove activity signals from observations of stars outside our solar system and eventually help detect habitable-zone Earth-mass exoplanets around Sun-like stars.

astro-ph.EP↗

arXiv2022

The EXPRES Stellar Signals Project II. State of the Field in Disentangling Photospheric Velocities

Measured spectral shifts due to intrinsic stellar variability (e.g., pulsations, granulation) and activity (e.g., spots, plages) are the largest source of error for extreme precision radial velocity (EPRV) exoplanet detection. Several methods are designed to disentangle stellar signals from true center-of-mass shifts due to planets. The EXPRES Stellar Signals Project (ESSP) presents a self-consistent comparison of 22 different methods tested on the same extreme-precision spectroscopic data from EXPRES. Methods derived new activity indicators, constructed models for mapping an indicator to the needed RV correction, or separated out shape- and shift-driven RV components. Since no ground truth is known when using real data, relative method performance is assessed using the total and nightly scatter of returned RVs and agreement between the results of different methods. Nearly all submitted methods return a lower RV RMS than classic linear decorrelation, but no method is yet consistently reducing the RV RMS to sub-meter-per-second levels. There is a concerning lack of agreement between the RVs returned by different methods. These results suggest that continued progress in this field necessitates increased interpretability of methods, high-cadence data to capture stellar signals at all timescales, and continued tests like the ESSP using consistent data sets with more advanced metrics for method performance. Future comparisons should make use of various well-characterized data sets -- such as solar data or data with known injected planetary and/or stellar signals -- to better understand method performance and whether planetary signals are preserved.

astro-ph.EP↗

arXiv2021

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes

Recently the LARS and LAMB optimizers have been proposed for training neural networks faster using large batch sizes. LARS and LAMB add layer-wise normalization to the update rules of Heavy-ball momentum and Adam, respectively, and have become popular in prominent benchmarks and deep learning libraries. However, without fair comparisons to standard optimizers, it remains an open question whether LARS and LAMB have any benefit over traditional, generic algorithms. In this work we demonstrate that standard optimization algorithms such as Nesterov momentum and Adam can match or exceed the results of LARS and LAMB at large batch sizes. Our results establish new, stronger baselines for future comparisons at these batch sizes and shed light on the difficulties of comparing optimizers for neural network training more generally.

cs.LG↗

arXiv2020

On Empirical Comparisons of Optimizers for Deep Learning

Selecting an optimizer is a central step in the contemporary deep learning pipeline. In this paper, we demonstrate the sensitivity of optimizer comparisons to the hyperparameter tuning protocol. Our findings suggest that the hyperparameter search space may be the single most important factor explaining the rankings obtained by recent empirical comparisons in the literature. In fact, we show that these results can be contradicted when hyperparameter search spaces are changed. As tuning effort grows without bound, more general optimizers should never underperform the ones they can approximate (i.e., Adam should never perform worse than momentum), but recent attempts to compare optimizers either assume these inclusion relationships are not practically relevant or restrict the hyperparameters in ways that break the inclusions. In our experiments, we find that inclusion relationships between optimizers matter in practice and always predict optimizer comparisons. In particular, we find that the popular adaptive gradient methods never underperform momentum or gradient descent. We also report practical tips around tuning often ignored hyperparameters of adaptive gradient methods and raise concerns about fairly benchmarking optimizers for neural network training.

cs.LG↗

arXiv2020

Faster Neural Network Training with Data Echoing

In the twilight of Moore's law, GPUs and other specialized hardware accelerators have dramatically sped up neural network training. However, earlier stages of the training pipeline, such as disk I/O and data preprocessing, do not run on accelerators. As accelerators continue to improve, these earlier stages will increasingly become the bottleneck. In this paper, we introduce "data echoing," which reduces the total computation used by earlier pipeline stages and speeds up training whenever computation upstream from accelerators dominates the training time. Data echoing reuses (or "echoes") intermediate outputs from earlier pipeline stages in order to reclaim idle capacity. We investigate the behavior of different data echoing algorithms on various workloads, for various amounts of echoing, and for various batch sizes. We find that in all settings, at least one data echoing algorithm can match the baseline's predictive performance using less upstream computation. We measured a factor of 3.25 decrease in wall-clock time for ResNet-50 on ImageNet when reading training data over a network.

cs.LG↗

arXiv2019

Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

Increasing the batch size is a popular way to speed up neural network training, but beyond some critical batch size, larger batch sizes yield diminishing returns. In this work, we study how the critical batch size changes based on properties of the optimization algorithm, including acceleration and preconditioning, through two different lenses: large scale experiments, and analysis of a simple noisy quadratic model (NQM). We experimentally demonstrate that optimization algorithms that employ preconditioning, specifically Adam and K-FAC, result in much larger critical batch sizes than stochastic gradient descent with momentum. We also demonstrate that the NQM captures many of the essential features of real neural network training, despite being drastically simpler to work with. The NQM predicts our results with preconditioned optimizers, previous results with accelerated gradient descent, and other results around optimal learning rates and large batch training, making it a useful tool to generate testable predictions about neural network optimization.

cs.LG↗

arXiv2019

Measuring the Effects of Data Parallelism on Neural Network Training

Recent hardware developments have dramatically increased the scale of data parallelism available for neural network training. Among the simplest ways to harness next-generation hardware is to increase the batch size in standard mini-batch neural network training algorithms. In this work, we aim to experimentally characterize the effects of increasing the batch size on training time, as measured by the number of steps necessary to reach a goal out-of-sample error. We study how this relationship varies with the training algorithm, model, and data set, and find extremely large variation between workloads. Along the way, we show that disagreements in the literature on how batch size affects model quality can largely be explained by differences in metaparameter tuning and compute budgets at different batch sizes. We find no evidence that larger batch sizes degrade out-of-sample performance. Finally, we discuss the implications of our results on efforts to train neural networks much faster in the future. Our experimental data is publicly available as a database of 71,638,836 loss measurements taken over the course of training for 168,160 individual models across 35 workloads.

cs.LG↗

arXiv2019

Identifying Exoplanets with Deep Learning III: Automated Triage and Vetting of TESS Candidates

NASA's Transiting Exoplanet Survey Satellite (TESS) presents us with an unprecedented volume of space-based photometric observations that must be analyzed in an efficient and unbiased manner. With at least $\sim1,000,000$ new light curves generated every month from full frame images alone, automated planet candidate identification has become an attractive alternative to human vetting. Here we present a deep learning model capable of performing triage and vetting on TESS candidates. Our model is modified from an existing neural network designed to automatically classify Kepler candidates, and is the first neural network to be trained and tested on real TESS data. In triage mode, our model can distinguish transit-like signals (planet candidates and eclipsing binaries) from stellar variability and instrumental noise with an average precision (the weighted mean of precisions over all classification thresholds) of 97.0% and an accuracy of 97.4%. In vetting mode, the model is trained to identify only planet candidates with the help of newly added scientific domain knowledge, and achieves an average precision of 69.3% and an accuracy of 97.8%. We apply our model on new data from Sector 6, and present 288 new signals that received the highest scores in triage and vetting and were also identified as planet candidates by human vetters. We also provide a homogeneously classified set of TESS candidates suitable for future training.

astro-ph.EP↗

arXiv2019

Identifying Exoplanets with Deep Learning II: Two New Super-Earths Uncovered by a Neural Network in K2 Data

For years, scientists have used data from NASA's Kepler Space Telescope to look for and discover thousands of transiting exoplanets. In its extended K2 mission, Kepler observed stars in various regions of sky all across the ecliptic plane, and therefore in different galactic environments. Astronomers want to learn how the population of exoplanets are different in these different environments. However, this requires an automatic and unbiased way to identify the exoplanets in these regions and rule out false positive signals that mimic transiting planet signals. We present a method for classifying these exoplanet signals using deep learning, a class of machine learning algorithms that have become popular in fields ranging from medical science to linguistics. We modified a neural network previously used to identify exoplanets in the Kepler field to be able to identify exoplanets in different K2 campaigns, which range in galactic environments. We train a convolutional neural network, called AstroNet-K2, to predict whether a given possible exoplanet signal is really caused by an exoplanet or a false positive. AstroNet-K2 is highly successful at classifying exoplanets and false positives, with accuracy of 98% on our test set. It is especially efficient at identifying and culling false positives, but for now, still needs human supervision to create a complete and reliable planet candidate sample. We use AstroNet-K2 to identify and validate two previously unknown exoplanets. Our method is a step towards automatically identifying new exoplanets in K2 data and learning how exoplanet populations depend on their galactic birthplace.

astro-ph.EP↗

arXiv2018

Embedding Text in Hyperbolic Spaces

Natural language text exhibits hierarchical structure in a variety of respects. Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised learning algorithms that work on text data. Recent work by Nickel & Kiela (2017) proposed using hyperbolic instead of Euclidean embedding spaces to represent hierarchical data and demonstrated encouraging results when embedding graphs. In this work, we extend their method with a re-parameterization technique that allows us to learn hyperbolic embeddings of arbitrarily parameterized objects. We apply this framework to learn word and sentence embeddings in hyperbolic space in an unsupervised manner from text corpora. The resulting embeddings seem to encode certain intuitive notions of hierarchy, such as word-context frequency and phrase constituency. However, the implicit continuous hierarchy in the learned hyperbolic space makes interrogating the model's learned hierarchies more difficult than for models that learn explicit edges between items. The learned hyperbolic embeddings show improvements over Euclidean embeddings in some -- but not all -- downstream tasks, suggesting that hierarchical organization is more useful for some tasks than others.

cs.CL↗

arXiv2017

Identifying Exoplanets with Deep Learning: A Five Planet Resonant Chain around Kepler-80 and an Eighth Planet around Kepler-90

NASA's Kepler Space Telescope was designed to determine the frequency of Earth-sized planets orbiting Sun-like stars, but these planets are on the very edge of the mission's detection sensitivity. Accurately determining the occurrence rate of these planets will require automatically and accurately assessing the likelihood that individual candidates are indeed planets, even at low signal-to-noise ratios. We present a method for classifying potential planet signals using deep learning, a class of machine learning algorithms that have recently become state-of-the-art in a wide variety of tasks. We train a deep convolutional neural network to predict whether a given signal is a transiting exoplanet or a false positive caused by astrophysical or instrumental phenomena. Our model is highly effective at ranking individual candidates by the likelihood that they are indeed planets: 98.8% of the time it ranks plausible planet signals higher than false positive signals in our test set. We apply our model to a new set of candidate signals that we identified in a search of known Kepler multi-planet systems. We statistically validate two new planets that are identified with high confidence by our model. One of these planets is part of a five-planet resonant chain around Kepler-80, with an orbital period closely matching the prediction by three-body Laplace relations. The other planet orbits Kepler-90, a star which was previously known to host seven transiting planets. Our discovery of an eighth planet brings Kepler-90 into a tie with our Sun as the star known to host the most planets.

astro-ph.EP↗

arXiv2012

Permutation polynomials of finite fields

Let $\mathbb{F}_q$ be the finite field of $q$ elements. Then a \emph{permutation polynomial} (PP) of $\mathbb{F}_q$ is a polynomial $f \in \mathbb{F}_q[x]$ such that the associated function $c \mapsto f(c)$ is a permutation of the elements of $\mathbb{F}_q$. In 1897 Dickson gave what he claimed to be a complete list of PPs of degree at most 6, however there have been suggestions recently that this classification might be incomplete. Unfortunately, Dickson's claim of a full characterisation is not easily verified because his published proof is difficult to follow. This is mainly due to antiquated terminology. In this project we present a full reconstruction of the classification of degree 6 PPs, which combined with a recent paper by Li \emph{et al.} finally puts to rest the characterisation problem of PPs of degree up to 6. In addition, we give a survey of the major results on PPs since Dickson's 1897 paper. Particular emphasis is placed on the proof of the so-called \emph{Carlitz Conjecture}, which states that if $q$ is odd and `large' and $n$ is even then there are no PPs of degree $n$. This important result was resolved in the affirmative by research spanning three decades. A generalisation of Carlitz's conjecture due to Mullen proposes that if $q$ is odd and `large' and $n$ is even then no polynomial of degree $n$ is `close' to being a PP. This has remained an unresolved problem in published literature. We provide a counterexample to Mullen's conjecture, and also point out how recent results imply a more general version of this statement (provided one increases what is meant by $q$ being `large').

math.NT↗