arXiv subjects

Robert Lilow

Publications and source records attributed to Robert Lilow.

At least 19 recordsLinked to original sources

arXiv2026

Mathematical Foundations of Modeling ETL Process Chains

Extract-Transform-Load (ETL) processes are core components of modern data processing infrastructures. The throughput of processed data records can be adjusted by changing the amount of allocated resources, i.e.~the number of parallel processing threads for each of the three ETL phases, but also depends on stochastic variations in the per-record processing times. In chains of multiple consecutive ETL processes, the relation between allocated resources and overall throughput is further complicated, for example by the occurrence of bottlenecks affecting all subsequent ETL processes. We develop a mathematical model of ETL process chains that is accurate at the level of time-aggregated throughput and suitable for efficient simulation. The process chain is represented as a controlled discrete-time Markov process on a directed acyclic graph whose edges are individual ETL processes. We model the mean throughput as a bounded, monotone function of the number of parallel threads, to capture the diminishing benefit of allocating more threads. We furthermore introduce a Flow Balance postulate linking number of threads, mean throughput, and mean processing time. The stochastic processing times are then modeled by non-negative heavy-tailed distributions around the mean processing time. This framework provides a principled simulator for ETL networks and a foundation for learning- and control-based resource allocation.

cs.DC↗

arXiv2024

Neural network reconstruction of density and velocity fields from the 2MASS Redshift Survey

We reconstruct the 3D matter density and peculiar velocity fields in the local Universe up to a distance of 200$\,h^{-1}\,$Mpc from the Two-Micron All-Sky Redshift Survey (2MRS), using a neural network (NN). We employed an NN with a U-net autoencoder architecture and a weighted mean squared error loss function trained separately to output either the density or velocity field for a given input grid of galaxy number counts. The NN was trained on mocks derived from the Quijote N-body simulations, incorporating redshift-space distortions (RSDs), galaxy bias, and selection effects closely mimicking the characteristics of 2MRS. The trained NN was benchmarked against a standard Wiener filter (WF) on a validation set of mocks before applying it to 2MRS. The NN reconstructions effectively approximate the mean posterior estimate of the true density and velocity fields conditioned on the observations. They consistently outperform the WF in terms of reconstruction accuracy and effectively capture the nonlinear relation between velocity and density. The NN-reconstructed bulk flow of the total survey volume exhibits a significant correlation with the true mock bulk flow, demonstrating that the NN is sensitive to information on super-survey scales encoded in the RSDs. When applied to 2MRS, the NN successfully recovers the main known clusters, some of which are partially in the Zone of Avoidance. The reconstructed bulk flows in spheres of different radii less than 100$\,h^{-1}\,$Mpc are in good agreement with a previous 2MRS analysis that required an additional external bulk flow component inferred from directly observed peculiar velocities. The NN-reconstructed peculiar velocity of the Local Group closely matches the observed Cosmic Microwave Background dipole in amplitude and Galactic latitude, and only deviates by 18${}^\circ$ in longitude. The NN-reconstructed fields are publicly available.

astro-ph.CO↗

arXiv2023

Large-scale density and velocity field reconstructions with neural networks

We assess a neural network (NN) method for reconstructing 3D cosmological density and velocity fields (target) from discrete and incomplete galaxy distributions (input). We employ second-order Lagrangian Perturbation Theory to generate a large ensemble of mock data to train an autoencoder (AE) architecture with a Mean Squared Error (MSE) loss function. The AE successfully captures nonlinear features arising from gravitational dynamics and the discreteness of the galaxy distribution. It preserves the positivity of the reconstructed density field and exhibits a weaker suppression of the power on small scales than the traditional linear Wiener filter (WF), which we use as a benchmark. In the density reconstruction, the reduction of the AE MSE relative to the WF is $\sim 15 \%$, whereas, for the velocity reconstruction, a relative reduction of up to a factor of two can be achieved. The AE is advantageous to the WF at recovering the distribution of the target fields, especially at the tails. In fact, trained with an MSE loss, any NN estimate approaches the unbiased mean of the underlying target given the input. This implies a slope of unity in the linear regression of the true on the NN-reconstructed field. Only for the special case of Gaussian fields, the NN and WF estimates are equivalent. Nonetheless, we also recover a linear regression slope of unity for the WF with non-Gaussian fields.

astro-ph.CO↗

arXiv2023

Ultracold plasmas from strongly anti-correlated Rydberg gases in the Kinetic Field Theory formalism

The dynamics of correlated systems is relevant in many fields ranging from cosmology to plasma physics. However, they are challenging to predict and understand even for classical systems due to the typically large numbers of particles involved. Here, we study the evolution of an ultracold, correlated many-body system with repulsive interactions and initial correlations set by the Rydberg blockade using the analytical framework of Kinetic Field Theory (KFT). The KFT formalism is based on the path-integral formulation for classical mechanics and was first developed and successfully used in cosmology to describe structure formation in Dark Matter. The theoretical framework offers a high flexibility regarding the initial configuration and interactions between particles and, in addition, is computationally cheap. More importantly, the analytic approach allows us to gain better insight into the processes which dominate the dynamics. In this work we show that KFT can be applied in a much more general context and study the evolution of a correlated ion plasma. We find good agreement between the analytical KFT results for the evolution of the correlation function and results obtained from numerical simulations. We use the correlation functions obtained with KFT to compute the temperature increase in the ionic system due to disorder-induced heating. For certain choices of parameters we observe that the effect can be reversed, leading to correlation cooling. Due to its numerical efficiency as compared to numerical simulations, a detailed study using KFT can help to constrain parameter spaces where disorder-induced heating is minimal in order to reach the regime of strong coupling.

cond-mat.quant-gas↗

arXiv2023

Baryon-photon interactions in Resummed Kinetic Field Theory

We explore how interactions between baryons and photons can be incorporated into Kinetic Field Theory (KFT), a description of cosmic structure formation based on classical Hamiltonian particle dynamics. In KFT, baryons are described as effective mesoscopic particles which represent fluid elements governed by the hydrodynamic equations. In this paper, we modify the mesoscopic particle model to include pressure effects exerted on baryonic matter through interactions with photons. As a proof of concept, we use this extended mesoscopic model to describe the tightly coupled baryon-photon fluid between matter-radiation equality and recombination. We show that this model can qualitatively reproduce the formation of baryon-acoustic oscillations in the cosmological power spectrum.

astro-ph.CO↗

arXiv2021

Constrained realizations of 2MRS density and peculiar velocity fields: growth rate and local flow

We generate constrained realizations (CRs) of the density and peculiar velocity fields within $200 \; h^{-1} \, \mathrm{Mpc}$ from the final release of the Two-Micron All-Sky Redshift Survey (2MRS) $-$ the densest all-sky redshift survey to date. The CRs are generated by combining a Wiener filter estimator in spherical Fourier-Bessel space with random realizations of log-normally distributed density fields and Poisson-sampled galaxy positions. The algorithm is tested and calibrated on a set of semi-analytic mock catalogs mimicking the environment of the Local Group (LG), to rigorously account for the statistical and systematic errors of the reconstruction method. By comparing our peculiar velocity CRs with the observed velocities from the Cosmicflows-3 catalog, we constrain the normalized linear growth rate to $f σ_8^\mathrm{lin} = 0.367 \pm 0.060$, which is consistent at the $1.1 σ$ level with the latest Planck results as well as other direct probes. Simultaneously, we estimate a bulk flow contribution from sources beyond the 2MRS reconstruction volume of $B^\mathrm{ext} = 199 \pm 68 \; \mathrm{km} \, \mathrm{s}^{-1}$ towards $l = 299 \pm 18^\circ$, $b = 8 \pm 19^\circ$. The total reconstructed velocity field at the position of the LG, smoothed with a $1 \; h^{-1} \, \mathrm{Mpc}$ Gaussian, is $685 \pm 75 \; \mathrm{km} \, \mathrm{s}^{-1}$ towards $l = 270.6 \pm 6.6^\circ$, $b = 35.5 \pm 7.2^\circ$, in good agreement with the observed CMB dipole. The total reconstructed bulk flow within different radii is compatible with other measurements. Within a $50 \; h^{-1} \, \mathrm{Mpc}$ Gaussian window we find a bulk flow of $274 \pm 50 \; \mathrm{km} \, \mathrm{s}^{-1}$ towards $l = 287 \pm 9^\circ$, $b = 11 \pm 10^\circ$. The code used to generate the CRs and obtain these results, dubbed CORAS, is made publicly available.

astro-ph.CO↗

arXiv2021

A first comparison of Kinetic Field Theory with Eulerian Standard Perturbation Theory

We present a detailed comparison of the newly developed particle-based Kinetic Field Theory framework for cosmic large-scale structure formation with the established formalism of Eulerian Standard Perturbation Theory. We highlight the qualitative differences of both approaches by a comparative analysis of the respective equations of motion and implementation of initial conditions. A natural starting point for a first quantitative comparison is given by the non-interacting regime of free-streaming kinematics. Our results suggest that Kinetic Field Theory contains a complete resummation of Standard Perturbation Theory in this regime. We further show that the exact free-streaming solution of Kinetic Field Theory can not be recovered in any finite order of Standard Perturbation Theory. Kinetic Field Theory should therefore provide a better starting point for perturbative treatments of non-linear structure formation.

astro-ph.CO↗

arXiv2021

A new measurement of the Hubble constant using Fast Radio Bursts

Fast radio bursts (FRBs) are very short and bright transients visible over extragalactic distances. The radio pulse undergoes dispersion caused by free electrons along the line of sight, most of which are associated with the large-scale structure (LSS). The total dispersion measure therefore increases with the line of sight and provides a distance estimate to the source. We present the first measurement of the Hubble constant using the dispersion measure -- redshift relation of FRBs with identified host counterpart and corresponding redshift information. A sample of nine currently available FRBs yields a constraint of $H_0 = 62.3 \pm 9.1 \,\rm{km} \,\rm{s}^{-1}\,\rm{Mpc}^{-1}$, accounting for uncertainty stemming from the LSS, host halo and Milky Way contributions to the observed dispersion measure. The main current limitation is statistical, and we estimate that a few hundred events with corresponding redshifts are sufficient for a per cent measurement of $H_0$. This is a number well within reach of ongoing FRB searches. We perform a forecast using a realistic mock sample to demonstrate that a high-precision measurement of the expansion rate is possible without relying on other cosmological probes. FRBs can therefore arbitrate the current tension between early and late time measurements of $H_0$ in the near future.

astro-ph.CO↗

arXiv2021

Consistent equivalence principle tests with fast radio bursts

Fast radio bursts (FRBs) are astrophysical transients of still debated origin. So far several hundred events have been detected, mostly at extragalactic distances, and this number is expected to grow significantly over the next years. The radio signals from the burst experience dispersion as they travel through the free electrons along the line-of-sight characterised by the dispersion measure (DM) of the radio pulse. In addition, each photon also experiences a gravitational Shapiro time delay while travelling through the potentials generated by the large-scale structure. If the equivalence principle (EP) holds, the Shapiro delay is the same for photons of all frequencies. In case the EP is broken, one would expect an additional dispersion to occur which could be either positive or negative for individual sources. Here we suggest to use angular statistics of the DM fluctuations to put constraints on the EP parametrized by the post-Newtonian parameter $γ$. Previous studies suffer from the problem that the gravitational potential responsible for the delay diverges in a cosmological setting, which our approach avoids. We carry out a forecast for a population of FRBs observable within the next years and show that any significant detection of the DM angular power spectrum will place constraints on the EP that are by a few orders of magnitude more stringent than current limits.

astro-ph.CO↗

arXiv2020

Kinetic field theory: Non-linear cosmic power spectra in the mean-field approximation

We use the recently developed Kinetic Field Theory (KFT) for cosmic structure formation to show how non-linear power spectra for cosmic density fluctuations can be calculated in a mean-field approximation to the particle interactions. Our main result is a simple, closed and analytic, approximate expression for this power spectrum. This expression has two parameters characterising non-linear structure growth which can be calibrated within KFT itself. Using this self-calibration, the non-linear power spectrum agrees with results obtained from numerical simulations to within typically $\lesssim10\,\%$ up to wave numbers $k\lesssim10\,h\,\mathrm{Mpc}^{-1}$ at redshift $z = 0$. Adjusting the two parameters to optimise agreement with numerical simulations, the relative difference to numerical results shrinks to typically $\lesssim 5\,\%$. As part of the derivation of our mean-field approximation, we show that the effective interaction potential between dark-matter particles relative to Zel'dovich trajectories is sourced by non-linear cosmic density fluctuations only, and is approximately of Yukawa rather than Newtonian shape.

astro-ph.CO↗

arXiv2020

Resummed Kinetic Field Theory: a model of coupled baryonic and dark matter

We present a new analytical description of cosmic structure formation in a mixture of dark and baryonic matter, using the framework of Kinetic Field Theory (KFT) -- a statistical field theory for classical particle dynamics. So far, KFT has only been able to describe a single type of particles, sufficient to consider structure growth due to the gravitational interactions between dark matter. However, the influence of baryonic gas dynamics becomes increasingly relevant when describing smaller scales. In this paper, we thus demonstrate how to extend the KFT formalism as well as a previously presented resummation scheme towards describing such mixtures of two particle species. Thereby, the gas dynamics of baryons are accounted for using the recently developed model of Mesoscopic Particle Hydrodynamics. Assuming a flat $Λ$CDM Universe and a simplified model for the thermal gas evolution, we demonstrate the validity of this approach by computing the linear evolution of the individual and total matter power spectra between the epoch of recombination and today. Our results correctly reproduce the expected behaviour, showing a suppression of both baryonic and dark matter structure growth on scales smaller than the baryonic Jeans length, in good agreement with results from the numerical Boltzmann solver CLASS. Nonlinear corrections within this approach will be investigated in upcoming works.

astro-ph.CO↗

arXiv2020

Probing primordial non-Gaussianity with Fast Radio Bursts

Fast radio bursts (FRBs) are astrophysical transients of currently unknown origin, and so far several events have been detected at extragalactic distances. The dispersion measure (DM) of the radio signal is a probe of the integrated electron density along the line of sight and therefore allows to map the electron distribution within the large-scale structure. Since a fraction of electrons gets expelled from galaxies by feedback, they are anticorrelated with halos at large scales and hence the angular DM correlations show a scale-dependent bias caused by primordial non-Gaussianity. Although the signal is weaker than in other probes like galaxy clustering, FRBs can potentially probe considerably larger volumes. We show that while studying the FRB clustering signal requires very large samples, correlations in the DM are cosmic-variance limited on large angular scales with only $\sim 10^{3-4}$ events. A tomographic analysis of the angular DM correlation function can constrain the local primordial bispectrum shape parameter $f_\mathrm{NL}$ to a precision down to ${f_\mathrm{NL}}\sim \mathcal{O}(1)$ depending on assumptions about the FRB redshift distribution and the astrophysical feedback on large scales. This makes FRBs a competitive probe to constrain inflationary physics.

astro-ph.CO↗

arXiv2019

Resummed Kinetic Field Theory: Using Mesoscopic Particle Hydrodynamics to Describe Baryonic Matter in a Cosmological Framework

Recently, Bartelmann et al. presented a 'Kinetic Field Theory' (KFT) formalism to tackle the difficulties of large scale structure formation. In this approach, the dynamics of a non-equilibrium ensemble of classical particles are examined based on methods of statistical field theory. So far, only contributions coming from dark matter were considered, which is assumed to pose an accurate description of our universe on very large scales. Nevertheless, going to smaller scales, also baryonic contributions have to be taken into account. Building on the ideas of Viermann et al. we present an effective particle model of hydrodynamics to describe baryonic matter in a cosmological framework. Using this model, the baryonic density contrast power spectrum is computed to lowest perturbative order within the resummed KFT framework of Lilow et al. We discuss the qualitative differences to the dark matter case and perform a quantitative comparison to the baryonic spectrum obtained from Eulerian perturbation theory. A subsequent paper will resolve the problem of coupling both theories describing dark and baryonic matter, respectively, to gain a full model of cosmic matter. Even though our focus is on cosmological systems only, we want to emphasize that all methods presented here are of a quite general fashion, making it applicable also to other fields.

astro-ph.CO↗

arXiv2019

Cosmic Structure Formation with Kinetic Field Theory

Kinetic Field Theory (KFT) is a statistical field theory for an ensemble of point-like classical particles in or out of equilibrium. We review its application to cosmological structure formation. Beginning with the construction of the generating functional of the theory, we describe in detail how the theory needs to be adapted to reflect the expanding spatial background and the homogeneous and isotropic, correlated initial conditions for cosmic structures. Based on the generating functional, we develop three main approaches to non-linear, late-time cosmic structures, which rest either on the Taylor expansion of an interaction operator, suitable averaging procedures for the interaction term, or a resummation of perturbation terms. We show how an analytic, parameter-free equation for the non-linear cosmic power spectrum can be derived. We explain how the theory can be used to derive the density profile of gravitationally bound structures and use it to derive power spectra of cosmic velocity densities. We further clarify how KFT relates to the BBGKY hierarchy. We then proceed to apply kinetic field theory to fluids, introduce a reformulation of KFT in terms of macroscopic quantities which leads to a resummation scheme, and use this to describe mixtures of gas and dark matter. We discuss how KFT can be applied to study cosmic structure formation with modified theories of gravity. As an example for an application to a non-cosmological particle ensemble, we show results on the spatial correlation function of cold Rydberg atoms derived from KFT.

astro-ph.CO↗

arXiv2019

Resummed Kinetic Field Theory: general formalism and linear structure growth from Newtonian particle dynamics

In earlier work, we have developed a nonequilibrium statistical field theory description of cosmic structure formation, dubbed Kinetic Field Theory (KFT), which is based on the Hamiltonian phase-space dynamics of classical particles and thus remains valid beyond shell-crossing. Here, we present an exact reformulation of the KFT framework that allows to resum an infinite subset of terms appearing in the original perturbative expansion of KFT. We develop the general formalism of this resummed KFT, including a diagrammatic language for the resummed perturbation theory, and compute the lowest-order results for the power spectra of the dark matter density contrast and momentum density. This allows us to derive analytically how the linear growth of the largest structures emerges from Newtonian particle dynamics alone, which, to our knowledge, is the first time this has been achieved.

astro-ph.CO↗

arXiv2017

Analytic calculation of the non-linear cosmic density-fluctuation power spectrum in the Born approximation

We derive a non-perturbative, closed, analytic equation for the non-linear power spectrum of cosmic density fluctuations. This result is based on our kinetic field theory (KFT) of cosmic structure formation and on evaluating particle interactions in the Born approximation. At redshift zero, relative deviations between our analytic result and typical numerical results are ~ 15 % on average up to wave numbers of k <= 10 h/Mpc. The theory underlying our analytic equation is fully specified once the statistical properties of the initial density and momentum fluctuations are set. It has no further adjustable parameters. Apart from this equation, our main result is that the characteristic non-linear deformations of the power spectrum at late cosmic times are determined by the initial momentum correlations and a partial compensation between diffusion damping and particle interactions.

astro-ph.CO↗

arXiv2017

Kinetic Field Theory: Exact free evolution of Gaussian phase-space correlations

In recent work we developed a description of cosmic large scale structure formation in terms of non-equilibrium ensembles of classical particles, with time evolution obtained in the framework of a statistical field theory. In these works, the initial Gaussian correlations between particles have so far been treated perturbatively or restricted to pure momentum correlations. Here we treat the correlations between all phase-space coordinates exactly by adopting a diagrammatic language for the different forms of correlations, directly inspired by the Mayer cluster expansion. We will demonstrate that explicit expressions for phase-space density cumulants of arbitrary $n$-point order, which fully capture the non-linear coupling of free streaming kinematics due to initial correlations, can be obtained from a simple set of Feynman rules. These cumulants will be the foundation for further investigations of interacting perturbation theory.

cond-mat.stat-mech↗

arXiv2017

Kinetic Field Theory: Effects of momentum correlations on the cosmic density-fluctuation power spectrum

In earlier work, we have developed a Kinetic Field Theory (KFT) for cosmological structure formation and showed that the non-linear density-fluctuation power spectrum known from numerical simulations can be reproduced quite well even if particle interactions are taken into account to first order only. Besides approximating gravitational interactions, we had to truncate the initial correlation hierarchy of particle momenta at the second order. Here, we substantially simplify KFT. We show that its central object, the free generating functional, can be factorized, taking the full hierarchy of momentum correlations into account. The factors appearing in the generating functional, which we identify as non-linearly evolved density-fluctuation power spectra, have a universal form and can thus be tabulated for fast access in perturbation schemes. In this paper, we focus on a complete evaluation of the free generating functional of KFT, not including particle interactions yet. This implies that the non-linearly evolved power spectra contain a damping term which reflects that structures are being wiped out at late times by free streaming. Once particle interactions will be taken into account, they will compensate this damping. If we suppress this damping in a way suggested by the fluctuation-dissipation relations of KFT, our results show that the complete hierarchy of initial momentum correlations is responsible for a large part of the characteristic non-linear deformation and the mode transport in the density-fluctuation power spectrum. Without any adjustable parameters, KFT accurately reproduces the scale at which non-linear evolution sets in. Finally, we further develop perturbation theory based on the factorization of the generating functional and propose a diagrammatic scheme for the perturbation terms.

astro-ph.CO↗