arXiv subjects

David Brooks

Publications and source records attributed to David Brooks.

At least 163 records · Page 9Linked to original sources

arXiv2023

GreenScale: Carbon-Aware Systems for Edge Computing

To improve the environmental implications of the growing demand of computing, future applications need to improve the carbon-efficiency of computing infrastructures. State-of-the-art approaches, however, do not consider the intermittent nature of renewable energy. The time and location-based carbon intensity of energy fueling computing has been ignored when determining how computation is carried out. This poses a new challenge -- deciding when and where to run applications across consumer devices at the edge and servers in the cloud. Such scheduling decisions become more complicated with the stochastic runtime variance and the amortization of the rising embodied emissions. This work proposes GreenScale, a framework to understand the design and optimization space of carbon-aware scheduling for green applications across the edge-cloud infrastructure. Based on the quantified carbon output of the infrastructure components, we demonstrate that optimizing for carbon, compared to performance and energy efficiency, yields unique scheduling solutions. Our evaluation with three representative categories of applications (i.e., AI, Game, and AR/VR) demonstrate that the carbon emissions of the applications can be reduced by up to 29.1% with the GreenScale. The analysis in this work further provides a detailed road map for edge-cloud application developers to build green applications.

cs.DC↗

arXiv2023

Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenters

Technology companies have been leading the way to a renewable energy transformation, by investing in renewable energy sources to reduce the carbon footprint of their datacenters. In addition to helping build new solar and wind farms, companies make power purchase agreements or purchase carbon offsets, rather than relying on renewable energy every hour of the day, every day of the week (24/7). Relying on renewable energy 24/7 is challenging due to the intermittent nature of wind and solar energy. Inherent variations in solar and wind energy production causes excess or lack of supply at different times. To cope with the fluctuations of renewable energy generation, multiple solutions must be applied. These include: capacity sizing with a mix of solar and wind power, energy storage options, and carbon aware workload scheduling. However, depending on the region and datacenter workload characteristics, the carbon-optimal solution varies. Existing work in this space does not give a holistic view of the trade-offs of each solution and often ignore the embodied carbon cost of the solutions. In this work, we provide a framework, Carbon Explorer, to analyze the multi-dimensional solution space by taking into account operational and embodided footprint of the solutions to help make datacenters operate on renewable energy 24/7. The solutions we analyze include capacity sizing with a mix of solar and wind power, battery storage, and carbon aware workload scheduling, which entails shifting the workloads from times when there is lack of renewable supply to times with abundant supply.

cs.DC↗

arXiv2023

MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation

Deep learning recommendation systems serve personalized content under diverse tail-latency targets and input-query loads. In order to do so, state-of-the-art recommendation models rely on terabyte-scale embedding tables to learn user preferences over large bodies of contents. The reliance on a fixed embedding representation of embedding tables not only imposes significant memory capacity and bandwidth requirements but also limits the scope of compatible system solutions. This paper challenges the assumption of fixed embedding representations by showing how synergies between embedding representations and hardware platforms can lead to improvements in both algorithmic- and system performance. Based on our characterization of various embedding representations, we propose a hybrid embedding representation that achieves higher quality embeddings at the cost of increased memory and compute requirements. To address the system performance challenges of the hybrid representation, we propose MP-Rec -- a co-design technique that exploits heterogeneity and dynamic selection of embedding representations and underlying hardware platforms. On real system hardware, we demonstrate how matching custom accelerators, i.e., GPUs, TPUs, and IPUs, with compatible embedding representations can lead to 16.65x performance speedup. Additionally, in query-serving scenarios, MP-Rec achieves 2.49x and 3.76x higher correct prediction throughput and 0.19% and 0.22% better model quality on a CPU-GPU system for the Kaggle and Terabyte datasets, respectively.

cs.AR↗

arXiv2023

Overview of the DESI Milky Way Survey

We describe the Milky Way Survey (MWS) that will be undertaken with the Dark Energy Spectroscopic Instrument (DESI) on the Mayall 4m telescope at the Kitt Peak National Observatory. Over the next 5 yr DESI MWS will observe approximately seven million stars at Galactic latitudes |b|>20 degrees, with an inclusive target selection scheme focused on the thick disk and stellar halo. MWS will also include several high-completeness samples of rare stellar types, including white dwarfs, low-mass stars within 100pc of the Sun, and horizontal branch stars. We summarize the potential of DESI to advance understanding of Galactic structure and stellar evolution. We introduce the final definitions of the main MWS target classes and estimate the number of stars in each class that will be observed. We describe our pipelines for deriving radial velocities, atmospheric parameters, and chemical abundances. We use ~500,000 spectra of unique stellar targets from the DESI Survey Validation program (SV) to demonstrate that our pipelines can measure radial velocities to ~1 km/s and [Fe/H] accurate to ~0.2 dex for typical stars in our main sample. We find the stellar parameter distributions from ~100 sq. deg of SV observations with >90% completeness on our main sample are in good agreement with expectations from mock catalogs and previous surveys.

astro-ph.GA↗

arXiv2023

DESI and DECaLS (D&D): galaxy-galaxy lensing measurements with 1% survey and its forecast

The shear measurement from DECaLS (Dark Energy Camera Legacy Survey) provides an excellent opportunity for galaxy-galaxy lensing study with DESI (Dark Energy Spectroscopic Instrument) galaxies, given the large ($\sim 9000$ deg$^2$) sky overlap. We explore this potential by combining the DESI 1\% survey and DECaLS DR8. With $\sim 106$ deg$^2$ sky overlap, we achieve significant detection of galaxy-galaxy lensing for BGS and LRG as lenses. Scaled to the full BGS sample, we expect the statistical errors to improve from $18(12)\%$ to a promising level of $2(1.3)\%$ at $θ>8^{'}(<8^{'})$. This brings stronger requirements for future systematics control. To fully realize such potential, we need to control the residual multiplicative shear bias $|m|<0.01$ and the bias in the mean redshift $|Δz|<0.015$. We also expect significant detection of galaxy-galaxy lensing with DESI LRG/ELG full samples as lenses, and cosmic magnification of ELG through cross-correlation with low-redshift DECaLS shear. {If such systematical error control can be achieved,} we find the advantages of DECaLS, comparing with KiDS (Kilo Degree Survey) and HSC (Hyper-Suprime Cam), are at low redshift, large-scale, and in measuring the shear-ratio (to $σ_R\sim 0.04$) and cosmic magnification.

astro-ph.CO↗

arXiv2023

MAVFI: An End-to-End Fault Analysis Framework with Anomaly Detection and Recovery for Micro Aerial Vehicles

Safety and resilience are critical for autonomous unmanned aerial vehicles (UAVs). We introduce MAVFI, the micro aerial vehicles (MAVs) resilience analysis methodology to assess the effect of silent data corruption (SDC) on UAVs' mission metrics, such as flight time and success rate, for accurately measuring system resilience. To enhance the safety and resilience of robot systems bound by size, weight, and power (SWaP), we offer two low-overhead anomaly-based SDC detection and recovery algorithms based on Gaussian statistical models and autoencoder neural networks. Our anomaly error protection techniques are validated in numerous simulated environments. We demonstrate that the autoencoder-based technique can recover up to all failure cases in our studied scenarios with a computational overhead of no more than 0.0062%. Our application-aware resilience analysis framework, MAVFI, can be utilized to comprehensively test the resilience of other Robot Operating System (ROS)-based applications and is publicly available at https://github.com/harvard-edge/MAVBench/tree/mavfi.

cs.RO↗

arXiv2023

PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices

The ability to accurately predict deep neural network (DNN) inference performance metrics, such as latency, power, and memory footprint, for an arbitrary DNN on a target hardware platform is essential to the design of DNN based models. This ability is critical for the (manual or automatic) design, optimization, and deployment of practical DNNs for a specific hardware deployment platform. Unfortunately, these metrics are slow to evaluate using simulators (where available) and typically require measurement on the target hardware. This work describes PerfSAGE, a novel graph neural network (GNN) that predicts inference latency, energy, and memory footprint on an arbitrary DNN TFlite graph (TFL, 2017). In contrast, previously published performance predictors can only predict latency and are restricted to pre-defined construction rules or search spaces. This paper also describes the EdgeDLPerf dataset of 134,912 DNNs randomly sampled from four task search spaces and annotated with inference performance metrics from three edge hardware platforms. Using this dataset, we train PerfSAGE and provide experimental results that demonstrate state-of-the-art prediction accuracy with a Mean Absolute Percentage Error of <5% across all targets and model search spaces. These results: (1) Outperform previous state-of-art GNN-based predictors (Dudziak et al., 2020), (2) Accurately predict performance on accelerators (a shortfall of non-GNN-based predictors (Zhang et al., 2021)), and (3) Demonstrate predictions on arbitrary input graphs without modifications to the feature extractor.

cs.LG↗

arXiv2023

The Target-selection Pipeline for the Dark Energy Spectroscopic Instrument

In 2021 May, the Dark Energy Spectroscopic Instrument (DESI) began a 5 yr survey of approximately 50 million total extragalactic and Galactic targets. The primary DESI dark-time targets are emission line galaxies (ELGs), luminous red galaxies (LRGs) and quasars (QSOs). In bright time, DESI will focus on two surveys known as the Bright Galaxy Survey (BGS) and the Milky Way Survey (MWS). DESI also observes a selection of "secondary" targets for bespoke science goals. This paper gives an overview of the publicly available pipeline (desitarget) used to process targets for DESI observations. Highlights include details of the different DESI survey targeting phases, the targeting ID (TARGETID) used to define unique targets, the bitmasks used to indicate a particular type of target, the data model and structure of DESI targeting files, and examples of how to access and use the desitarget code base. This paper will also describe "supporting" DESI target classes, such as standard stars, sky locations, and random catalogs that mimic the angular selection function of DESI targets. The DESI target selection pipeline is complex and sizable; this paper attempts to summarize the most salient information required to understand and work with DESI targeting data.

astro-ph.IM↗

arXiv2022

Architectural Implications of Embedding Dimension during GCN on CPU and GPU

Graph Neural Networks (GNNs) are a class of neural networks designed to extract information from the graphical structure of data. Graph Convolutional Networks (GCNs) are a widely used type of GNN for transductive graph learning problems which apply convolution to learn information from graphs. GCN is a challenging algorithm from an architecture perspective due to inherent sparsity, low data reuse, and massive memory capacity requirements. Traditional neural algorithms exploit the high compute capacity of GPUs to achieve high performance for both inference and training. The architectural decision to use a GPU for GCN inference is a question explored in this work. GCN on both CPU and GPU was characterized in order to better understand the implications of graph size, embedding dimension, and sampling on performance.

cs.LG↗

arXiv2022

Optimal 1D Ly$α$ Forest Power Spectrum Estimation -- II. KODIAQ, SQUAD & XQ-100

We measure the 1D Ly$\,α$ power spectrum $P_\mathrm{1D}$ from Keck Observatory Database of Ionized Absorption toward Quasars (KODIAQ), The Spectral Quasar Absorption Database (SQUAD) and XQ-100 quasars using the optimal quadratic estimator. We combine KODIAQ and SQUAD at the spectrum level, but perform a separate XQ-100 estimation to control its large resolution corrections in check. Our final analysis measures $P_\mathrm{1D}$ at scales $k<0.1\,$s$\,$km$^{-1}$ between redshifts $z=$ 2.0 -- 4.6 using 538 quasars. This sample provides the largest number of high-resolution, high-S/N observations; and combined with the power of optimal estimator it provides exceptional precision at small scales. These small-scale modes ($k\gtrsim 0.02\,$s$\,$km$^{-1}$), unavailable in Sloan Digital Sky Survey (SDSS) and Dark Energy Spectroscopic Instrument (DESI) analyses, are sensitive to the thermal state and reionization history of the intergalactic medium, as well as the nature of dark matter. As an example, a simple Fisher forecast analysis estimates that our results can improve small-scale cut off sensitivity by more than a factor of 2.

astro-ph.CO↗

arXiv2022

The effect of quasar redshift errors on Lyman-$α$ forest correlation functions

Using synthetic Lyman-$α$ forests from the Dark Energy Spectroscopic Instrument (DESI) survey, we present a study of the impact of errors in the estimation of quasar redshift on the Lyman-$α$ correlation functions. Estimates of quasar redshift have large uncertainties of a few hundred $\text{km s}^{-1}\,$ due to the broadness of the emission lines and the intrinsic shifts from other emission lines. We inject Gaussian random redshift errors into the mock quasar catalogues, and measure the auto-correlation and the Lyman-$α$-quasar cross-correlation functions. We find a smearing of the BAO feature in the radial direction, but changes in the peak position are negligible. However, we see a significant unphysical correlation for small separations transverse to the line of sight which increases with the amplitude of the redshift errors. We interpret this contamination as a result of the broadening of emission lines in the measured mean continuum, caused by quasar redshift errors, combined with the unrealistically strong clustering of the simulated quasars on small scales.

astro-ph.CO↗

arXiv2022

Constraining galaxy-halo connection with high-order statistics

We investigate using three-point statistics in constraining the galaxy-halo connection. We show that for some galaxy samples, the constraints on the halo occupation distribution parameters are dominated by the three-point function signal (over its two-point counterpart). We demonstrate this on mock catalogs corresponding to the Luminous Red Galaxies (LRGs), Emission-Line Galaxies (ELG), and quasars (QSOs) targeted by the Dark Energy Spectroscopic Instrument (DESI) Survey. The projected three-point function for triangle sides less up to 20$h^{-1}$ Mpc measured from a cubic Gpc of data can constrain the characteristic minimum mass of the LRGs with a precision of $0.46$ %. For comparison, similar constraints from the projected two-point function are $1.55$ %. The improvements for the ELGs and QSOs targets are more modest. In the case of the QSOs it is caused by the high shot-noise of the sample, and in the case of the ELGs, this is caused by the range of halo masses of the host halos. The most time-consuming part of our pipeline is the measurement of the three-point functions. We adopt a tabulation method, proposed in earlier works for the two-point function, to reduce significantly the required compute time for the three-point analysis.

astro-ph.CO↗

arXiv2022

Constraints on dark matter to dark radiation conversion in the late universe with DES-Y1 and external data

We study a phenomenological class of models where dark matter converts to dark radiation in the low redshift epoch. This class of models, dubbed DMDR, characterizes the evolution of comoving dark matter density with two extra parameters, and may be able to help alleviate the observed discrepancies between early- and late-time probes of the universe. We investigate how the conversion affects key cosmological observables such as the CMB temperature and matter power spectra. Combining 3x2pt data from Year 1 of the Dark Energy Survey, {\it Planck}-2018 CMB temperature and polarization data, supernovae (SN) Type Ia data from Pantheon, and baryon acoustic oscillation (BAO) data from BOSS DR12, MGS and 6dFGS, we place new constraints on the amount of dark matter that has converted to dark radiation and the rate of this conversion. The fraction of the dark matter that has converted since the beginning of the universe in units of the current amount of dark matter, $ζ$, is constrained at 68\% confidence level to be $<0.32$ for DES-Y1 3x2pt data, $<0.030$ for CMB+SN+BAO data, and $<0.037$ for the combined dataset. The probability that the DES and CMB+SN+BAO datasets are concordant increases from 4\% for the $Λ$CDM model to 8\% (less tension) for DMDR. The tension in $S_8 = σ_8 \sqrt{Ω_{\rm m}/0.3}$ between DES-Y1 3x2pt and CMB+SN+BAO is slightly reduced from $2.3σ$ to $1.9σ$. We find no reduction in the Hubble tension when the combined data is compared to distance-ladder measurements in the DMDR model. The maximum-posterior goodness-of-fit statistics of DMDR and $Λ$CDM model are comparable, indicating no preference for the DMDR cosmology over $Λ$CDM.

astro-ph.CO↗

arXiv2022

The MegaMapper: A Stage-5 Spectroscopic Instrument Concept for the Study of Inflation and Dark Energy

In this white paper, we present the MegaMapper concept. The MegaMapper is a proposed ground-based experiment to measure Inflation parameters and Dark Energy from galaxy redshifts at $2<z<5$. In order to achieve path-breaking results with a mid-scale investment, the MegaMapper combines existing technologies for critical path elements and pushes innovative development in other design areas. To this aim, we envision a 6.5-m Magellan-like telescope, with a newly designed wide field, coupled with DESI spectrographs, and small-pitch robots to achieve multiplexing of at least 26,000. This will match the expected achievable target density in the redshift range of interest and provide a 10x capability over the existing state-of the art, without a 10x increase in project budget.

astro-ph.IM↗

arXiv2022

A Spectroscopic Road Map for Cosmic Frontier: DESI, DESI-II, Stage-5

In this white paper, we present an experimental road map for spectroscopic experiments beyond DESI. DESI will be a transformative cosmological survey in the 2020s, mapping 40 million galaxies and quasars and capturing a significant fraction of the available linear modes up to z=1.2. DESI-II will pilot observations of galaxies both at much higher densities and extending to higher redshifts. A Stage-5 experiment would build out those high-density and high-redshift observations, mapping hundreds of millions of stars and galaxies in three dimensions, to address the problems of inflation, dark energy, light relativistic species, and dark matter. These spectroscopic data will also complement the next generation of weak lensing, line intensity mapping and CMB experiments and allow them to reach their full potential.

astro-ph.CO↗

arXiv2022

Target Selection and Validation of DESI Quasars

The Dark Energy Spectroscopic Instrument (DESI) survey will measure large-scale structures using quasars as direct tracers of dark matter in the redshift range 0.9 2.1. We present several methods to select candidate quasars for DESI, using input photometric imaging in three optical bands (g, r, z) from the DESI Legacy Imaging Surveys and two infrared bands (W1, W2) from the Wide-field Infrared Explorer (WISE). These methods were extensively tested during the Survey Validation of DESI. In this paper, we report on the results obtained with the different methods and present the selection we optimized for the DESI main survey. The final quasar target selection is based on a Random Forest algorithm and selects quasars in the magnitude range 16.5 99% purity for a nominal effective exposure time of ~1000s. With a 310 per sq. deg. target density, the main selection allows DESI to select more than 200 QSOs per sq. deg. (including 60 quasars with z>2.1), exceeding the project requirements by 20%. The redshift distribution of the selected quasars is in excellent agreement with quasar luminosity function predictions.

astro-ph.CO↗

arXiv2022

DESI Bright Galaxy Survey: Final Target Selection, Design, and Validation

Over the next five years, the Dark Energy Spectroscopic Instrument (DESI) will use 10 spectrographs with 5000 fibers on the 4m Mayall Telescope at Kitt Peak National Observatory to conduct the first Stage-IV dark energy galaxy survey. At $z < 0.6$, the DESI Bright Galaxy Survey (BGS) will produce the most detailed map of the Universe during the dark energy dominated epoch with redshifts of >10 million galaxies over 14,000 deg$^2$. In this work, we present and validate the final BGS target selection and survey design. From the Legacy Surveys, BGS will target a $r < 19.5$ magnitude-limited sample (BGS Bright); a fainter $19.5 < r < 20.175$ sample, color-selected to have high redshift efficiency (BGS Faint); and a smaller low-z quasar sample. BGS will observe these targets using exposure times, scaled to achieve uniform completeness, and visit each point on the footprint three times. We use observations from the Survey Validation programs conducted prior to the main survey along with realistic simulations to show that BGS can complete its strategy and make optimal use of `bright' time. We demonstrate that BGS targets have stellar contamination <1% and that their densities do not depend strongly on imaging properties. We also confirm that BGS Bright will achieve >80% fiber assignment efficiency. Finally, we show that BGS Bright and Faint will achieve >95% redshift success rates with no significant dependence on observing conditions. BGS meets the requirements for an extensive range of scientific applications. BGS will yield the most precise Baryon Acoustic Oscillations and Redshift-Space Distortions measurements at $z < 0.4$. It also presents opportunities to exploit new methods that require highly complete and dense galaxy samples (e.g. N-point statistics, multi-tracers). BGS further provides a powerful tool to study galaxy populations and the relations between galaxies and dark matter.

astro-ph.CO↗

arXiv2022

The DESI $N$-body Simulation Project -- II. Suppressing sample variance with fast simulations

Dark Energy Spectroscopic Instrument (DESI) will construct a large and precise three-dimensional map of our Universe. The survey effective volume reaches $\sim20\Gpchcube$. It is a great challenge to prepare high-resolution simulations with a much larger volume for validating the DESI analysis pipelines. \textsc{AbacusSummit} is a suite of high-resolution dark-matter-only simulations designed for this purpose, with $200\Gpchcube$ (10 times DESI volume) for the base cosmology. However, further efforts need to be done to provide a more precise analysis of the data and to cover also other cosmologies. Recently, the CARPool method was proposed to use paired accurate and approximate simulations to achieve high statistical precision with a limited number of high-resolution simulations. Relying on this technique, we propose to use fast quasi-$N$-body solvers combined with accurate simulations to produce accurate summary statistics. This enables us to obtain 100 times smaller variance than the expected DESI statistical variance at the scales we are interested in, e.g. $k < 0.3\hMpc$ for the halo power spectrum. In addition, it can significantly suppress the sample variance of the halo bispectrum. We further generalize the method for other cosmologies with only one realization in \textsc{AbacusSummit} suite to extend the effective volume $\sim 20$ times. In summary, our proposed strategy of combining high-fidelity simulations with fast approximate gravity solvers and a series of variance suppression techniques sets the path for a robust cosmological analysis of galaxy survey data.

astro-ph.CO↗