arXiv subjects

Layne C. Price

Publications and source records attributed to Layne C. Price.

At least 19 recordsLinked to original sources

arXiv2026

Align-RAG: Alignment Is All You Need for TSFM In-Context Learning

Retrieval-augmented forecasting promises to adapt frozen Time Series Foundation Models (TSFMs) to new domains without fine-tuning, but recent methods typically rely on learned fusion modules, i.e., trained adapters that merge retrieved examples into the backbone's forecast, based on the assumption that frozen backbones cannot dynamically incorporate retrieved context on their own. We show this assumption is unnecessary. We introduce Align-RAG, a training-free method that applies a closed-form per-pair amplitude rescaling and integer-lag phase shift to retrieved past-future windows before they enter a frozen backbone's context. With no learned parameters, Align-RAG outperforms the state-of-the-art trained retrieval adapter on a frozen Chronos-Bolt on all seven datasets of the standard benchmark (avg -3.75% MSE), showing that the gains previously attributed to learned fusion are recoverable without any training. Align-RAG further improves zero-shot MSE on four additional frozen TSFMs with various architectures by 2.5% to 13.7% per backbone with no per-backbone tuning. To probe why alignment helps, we compare the frozen backbone's prediction shift under aligned demonstrations to the closed-form ridge prediction shift on the same pairs. We find that aligned demonstrations induce prediction shifts that track a closed-form ridge predictor on the same pairs, with a future-shuffle control ruling out a futures-averaging account. Together, these results indicate that frozen TSFMs already support dynamic in-context use of retrievals, and that closed-form alignment should be the default baseline for retrieval-augmented forecasting before any fusion module is trained. Code available at: https://github.com/masadi-99/align-rag

cs.LG↗

arXiv2026

BoostLoRA: Growing Effective Rank by Boosting Adapters

Parameter-efficient fine-tuning (PEFT) methods face a tradeoff between adapter size and expressivity: ultra-low-parameter adapters are confined to fixed low-rank subspaces, capping performance even with extended training. We propose BoostLoRA, a gradient-boosting framework that overcomes this limit by iteratively training and merging minimal adapters on the examples the current model gets wrong. A ROTATE SVD basis strategy assigns each round to an orthogonal subspace, so cumulative effective rank grows linearly with the number of rounds while each adapter remains ultra-low-rank. After merging, adapters are discarded, leaving zero inference overhead. On Qwen2.5-3B, BoostLoRA reaches 89.1% on GSM8K and 68.8% on MATH-500, surpassing both the best single-shot ultra-low parameter adapter (TinyLoRA) and full fine-tuning; on code generation it reaches 57.2% on MBPP and 80.4% on HumanEval while full fine-tuning drops below the zero-shot baseline. We also demonstrate cross-architecture transfer on protein binding classification with ESM2-650M and cross-entropy training. BoostLoRA is, to our knowledge, the first PEFT method whose effective rank grows with training, separating per-round parameter cost from total representational capacity.

cs.LG↗

arXiv2025

NanoFlux: Adversarial Dual-LLM Evaluation and Distillation For Multi-Domain Reasoning

We present NanoFlux, a novel adversarial framework for generating targeted training data to improve LLM reasoning, where adversarially-generated datasets containing fewer than 200 examples outperform conventional fine-tuning approaches. The framework employs a competitive dynamic between models alternating as Attacker and Defender, supervised by a tool-augmented Judge, synthesizing multi-step questions with explanatory annotations that target specific reasoning capabilities. Fine-tuning a 4B-parameter model on NanoFlux-generated data yields performance gains across diverse domains compared to full-benchmark fine-tuning: +5.9% on mathematical reasoning (GSMHard), +3.6% on scientific reasoning (GenomeBench), and +16.6% on medical reasoning (MultiMedQA), while reducing computational requirements by 3-14x. Ablation studies reveal a non-monotonic relationship between dataset characteristics and model performance, uncovering domain-specific optimal points for question complexity and reasoning quality. NanoFlux automates training data generation through embedding-based novelty filtering, tool-augmented evaluation, and multi-hop reasoning, suggesting that future model improvements may lie in the intelligent synthesis of small, precisely targeted training datasets.

cs.LG↗

arXiv2023

Activity Classification Using Unsupervised Domain Transfer from Body Worn Sensors

Activity classification has become a vital feature of wearable health tracking devices. As innovation in this field grows, wearable devices worn on different parts of the body are emerging. To perform activity classification on a new body location, labeled data corresponding to the new locations are generally required, but this is expensive to acquire. In this work, we present an innovative method to leverage an existing activity classifier, trained on Inertial Measurement Unit (IMU) data from a reference body location (the source domain), in order to perform activity classification on a new body location (the target domain) in an unsupervised way, i.e. without the need for classification labels at the new location. Specifically, given an IMU embedding model trained to perform activity classification at the source domain, we train an embedding model to perform activity classification at the target domain by replicating the embeddings at the source domain. This is achieved using simultaneous IMU measurements at the source and target domains. The replicated embeddings at the target domain are used by a classification model that has previously been trained on the source domain to perform activity classification at the target domain. We have evaluated the proposed methods on three activity classification datasets PAMAP2, MHealth, and Opportunity, yielding high F1 scores of 67.19%, 70.40% and 68.34%, respectively when the source domain is the wrist and the target domain is the torso.

cs.LG↗

arXiv2022

Predicting Cellular Responses with Variational Causal Inference and Refined Relational Information

Predicting the responses of a cell under perturbations may bring important benefits to drug discovery and personalized therapeutics. In this work, we propose a novel graph variational Bayesian causal inference framework to predict a cell's gene expressions under counterfactual perturbations (perturbations that this cell did not factually receive), leveraging information representing biological knowledge in the form of gene regulatory networks (GRNs) to aid individualized cellular response predictions. Aiming at a data-adaptive GRN, we also developed an adjacency matrix updating technique for graph convolutional networks and used it to refine GRNs during pre-training, which generated more insights on gene relations and enhanced model performance. Additionally, we propose a robust estimator within our framework for the asymptotically efficient estimation of marginal perturbation effect, which is yet to be carried out in previous works. With extensive experiments, we exhibited the advantage of our approach over state-of-the-art deep learning models for individual response prediction.

cs.LG↗

arXiv2022

Variational Causal Inference

Estimating an individual's potential outcomes under counterfactual treatments is a challenging task for traditional causal inference and supervised learning approaches when the outcome is high-dimensional (e.g. gene expressions, impulse responses, human faces) and covariates are relatively limited. In this case, to construct one's outcome under a counterfactual treatment, it is crucial to leverage individual information contained in its observed factual outcome on top of the covariates. We propose a deep variational Bayesian framework that rigorously integrates two main sources of information for outcome construction under a counterfactual treatment: one source is the individual features embedded in the high-dimensional factual outcome; the other source is the response distribution of similar subjects (subjects with the same covariates) that factually received this treatment of interest.

stat.ML↗

arXiv2020

Robust posterior inference when statistically emulating forward simulations

Scientific analyses often rely on slow, but accurate forward models for observable data conditioned on known model parameters. While various emulation schemes exist to approximate these slow calculations, these approaches are only safe if the approximations are well understood and controlled. This workshop submission reviews and updates a previously published method, which has been used in cosmological simulations, to (1) train an emulator while simultaneously estimating posterior probabilities with MCMC and (2) explicitly propagate the emulation error into errors on the posterior probabilities for model parameters. We demonstrate how these techniques can be applied to quickly estimate posterior distributions for parameters of the $Λ$CDM cosmology model, while also gauging the robustness of the emulator approximation.

astro-ph.IM↗

arXiv2020

Discovering Invariances in Healthcare Neural Networks

We study the invariance characteristics of pre-trained predictive models by empirically learning transformations on the input that leave the prediction function approximately unchanged. To learn invariant transformations, we minimize the Wasserstein distance between the predictive distribution conditioned on the data instances and the predictive distribution conditioned on the transformed data instances. To avoid finding degenerate or perturbative transformations, we add a similarity regularization to discourage similarity between the data and its transformed values. We theoretically analyze the correctness of the algorithm and the structure of the solutions. Applying the proposed technique to clinical time series data, we discover variables that commonly-used LSTM models do not rely on for their prediction, especially when the LSTM is trained to be adversarially robust. We also analyze the invariances of BioBERT on clinical notes and discover words that it is invariant to.

cs.LG↗

arXiv2018

Effect of reheating on predictions following multiple-field inflation

We study the sensitivity of cosmological observables to the reheating phase following inflation driven by many scalar fields. We describe a method which allows semi-analytic treatment of the impact of perturbative reheating on cosmological perturbations using the sudden decay approximation. Focusing on $\mathcal{N}$-quadratic inflation, we show how the scalar spectral index and tensor-to-scalar ratio are affected by the rates at which the scalar fields decay into radiation. We find that for certain choices of decay rates, reheating following multiple-field inflation can have a significant impact on the prediction of cosmological observables.

astro-ph.CO↗

arXiv2017

Estimating Cosmological Parameters from the Dark Matter Distribution

A grand challenge of the 21st century cosmology is to accurately estimate the cosmological parameters of our Universe. A major approach to estimating the cosmological parameters is to use the large-scale matter distribution of the Universe. Galaxy surveys provide the means to map out cosmic large-scale structure in three dimensions. Information about galaxy locations is typically summarized in a "single" function of scale, such as the galaxy correlation function or power-spectrum. We show that it is possible to estimate these cosmological parameters directly from the distribution of matter. This paper presents the application of deep 3D convolutional networks to volumetric representation of dark-matter simulations as well as the results obtained using a recently proposed distribution regression framework, showing that machine learning techniques are comparable to, and can sometimes outperform, maximum-likelihood point estimates using "cosmological models". This opens the way to estimating the parameters of our Universe with higher accuracy.

astro-ph.CO↗

arXiv2017

The Spectrum of the Axion Dark Sector

Axions arise in many theoretical extensions of the Standard Model of particle physics, in particular the "string axiverse". If the axion masses, $m_a$, and (effective) decay constants, $f_a$, lie in specific ranges, then axions contribute to the cosmological dark matter and dark energy densities. We compute the background cosmological (quasi-)observables for models with a large number of axion fields, $n_{\rm ax}\sim \mathcal{O}(10-100)$, with the masses and decay constants drawn from statistical distributions. This reduces the number of parameters from $2n_{\rm ax}$ to a small number of "hyperparameters". We consider a number of distributions, from those motivated purely by statistical considerations, to those where the structure is specified according to a class of M-theory models. Using Bayesian methods we are able to constrain the hyperparameters of the distributions. In some cases the hyperparameters can be related to string theory, e.g. constraining the number ratio of axions to moduli, or the typical decay constant scale needed to provide the correct relic densities. Our methodology incorporates the use of both random matrix theory and Bayesian networks.

astro-ph.CO↗

arXiv2016

Ultracompact minihalos as probes of inflationary cosmology

Cosmological inflation generates primordial density perturbations on all scales, including those far too small to contribute to the cosmic microwave background. At these scales, isolated ultracompact minihalos of dark matter can form well before standard structure formation, if the perturbations have sufficient amplitude. Minihalos affect pulsar timing data and are potentially bright sources of gamma rays. The resulting constraints significantly extend the observable window of inflation in the presence of cold dark matter, coupling two of the key problems in modern cosmology.

astro-ph.CO↗

arXiv2016

Testing for New Physics: Neutrinos and the Primordial Power Spectrum

We test the sensitivity of neutrino parameter constraints from combinations of CMB and LSS data sets to the assumed form of the primordial power spectrum (PPS) using Bayesian model selection. Significantly, none of the tested combinations, including recent high-precision local measurements of $\mathrm{H}_0$ and cluster abundances, indicate a signal for massive neutrinos or extra relativistic degrees of freedom. For PPS models with a large, but fixed number of degrees of freedom, neutrino parameter constraints do not change significantly if the location of any features in the PPS are allowed to vary, although neutrino constraints are more sensitive to PPS features if they are known a priori to exist at fixed intervals in $\log k$. Although there is no support for a non-standard neutrino sector from constraints on both neutrino mass and relativistic energy density, we see surprisingly strong evidence for features in the PPS when it is constrained with data from Planck 2015, SZ cluster counts, and recent high-precision local measurements of $\mathrm{H}_0$. Conversely combining Planck with matter power spectrum and BAO measurements yields a much weaker constraint. Given that this result is sensitive to the choice of data this tension between SZ cluster counts, Planck and $\mathrm{H}_0$ measurements is likely an indication of unmodeled systematic bias that mimics PPS features, rather than new physics in the PPS or neutrino sector.

astro-ph.CO↗

arXiv2016

Reconstructing the redshift evolution of escaped ionizing flux from early galaxies with Planck and HST observations

While galaxies at $6 \lesssim z \lesssim 10$ are believed to dominate the epoch of cosmic reionization, the escape fraction of ionizing flux $f_\mathrm{esc}$ and the photon production rate $\dot n_γ$ from these galaxies must vary with redshift to simultaneously match CMB and low-redshift observations. We constrain $f_\mathrm{esc}(z)$ and $\dot n_γ(z)$ with Planck 2015 measurements of the Thomson optical depth $τ$, recent low multipole E-mode polarization measurements from Planck 2016, SDSS BAO data, and $3 \lesssim z \lesssim 10$ galaxy observations. We compare different galaxy luminosity functions that are calibrated to HST observations, using both parametric and non-parametric statistical methods that marginalize over the effective clumping factor $C_\mathrm{HII}$, the LyC production efficiency $ξ_\mathrm{ion}$, and the time-evolution of the UV limiting magnitude $dM_\mathrm{SF}/dz$. Using a power-law model, we find $f_\mathrm{esc} \lesssim 0.5$ at $z=8$ with slope $β\gtrsim 2.0$ at $68\%$ confidence with little dependence on the galaxy luminosity function or data, although there is non-negligible probability for no redshift evolution $β\sim 0$ or small escape fraction $f_\mathrm{esc} \sim 10^{-2}$. A non-parametric form for $f_\mathrm{esc}(z)$ evolves significantly with redshift, yielding $f_\mathrm{esc} \sim 0.2, 0.3, 0.6$ at $z=6,9,12$, respectively. However, a model-independent reconstruction of $\dot n_γ(z)$ predicts a suppressed escaped photon production rate at $z=9$ for the latest Planck data compared to the other models, implying a quicker period of reionization. We find evidence for redshift evolution in the limiting magnitude of the galaxy luminosity function for empirical models of the galaxy luminosity function.

astro-ph.CO↗

arXiv2016

Designing and testing inflationary models with Bayesian networks

Even simple inflationary scenarios have many free parameters. Beyond the variables appearing in the inflationary action, these include dynamical initial conditions, the number of fields, and couplings to other sectors. These quantities are often ignored but cosmological observables can depend on the unknown parameters. We use Bayesian networks to account for a large set of inflationary parameters, deriving generative models for the primordial spectra that are conditioned on a hierarchical set of prior probabilities describing the initial conditions, reheating physics, and other free parameters. We use $N_f$--quadratic inflation as an illustrative example, finding that the number of $e$-folds $N_*$ between horizon exit for the pivot scale and the end of inflation is typically the most important parameter, even when the number of fields, their masses and initial conditions are unknown, along with possible conditional dependencies between these parameters.

astro-ph.CO↗

arXiv2015

Power spectrum oscillations from Planck-suppressed operators in effective field theory motivated monodromy inflation

We consider a phenomenological model of inflation where the inflaton is the phase of a complex scalar field $Φ$. Planck-suppressed operators of $\mathcal O(f^5/M_\mathrm{pl})$ modify the geometry of the vev $\langle Φ\rangle$ at first order in the decay constant $f$, which adds a first order periodic term to the definition of the canonically normalized inflaton $ϕ$. This correction to the inflaton induces a fixed number of extra oscillatory terms in the potential $V \sim θ^p$. We derive the same result in a toy scenario where the vacuum $\langle Φ\rangle$ is an ellipse with an arbitrarily large eccentricity. These extra oscillations change the form of the power spectrum as a function of scale $k$ and provide a possible mechanism for differentiating EFT-motivated inflation from models where the angular shift symmetry is a gauge symmetry.

astro-ph.CO↗

arXiv2015

Learn-As-You-Go Acceleration of Cosmological Parameter Estimates

Cosmological analyses can be accelerated by approximating slow calculations using a training set, which is either precomputed or generated dynamically. However, this approach is only safe if the approximations are well understood and controlled. This paper surveys issues associated with the use of machine-learning based emulation strategies for accelerating cosmological parameter estimation. We describe a learn-as-you-go algorithm that is implemented in the Cosmo++ code and (1) trains the emulator while simultaneously estimating posterior probabilities; (2) identifies unreliable estimates, computing the exact numerical likelihoods if necessary; and (3) progressively learns and updates the error model as the calculation progresses. We explicitly describe and model the emulation error and show how this can be propagated into the posterior probabilities. We apply these techniques to the Planck likelihood and the calculation of $Λ$CDM posterior probabilities. The computation is significantly accelerated without a pre-defined training set and uncertainties in the posterior probabilities are subdominant to statistical fluctuations. We have obtained a speedup factor of $6.5$ for Metropolis-Hastings and $3.5$ for nested sampling. Finally, we discuss the general requirements for a credible error model and show how to update them on-the-fly.

astro-ph.IM↗

arXiv2014

Gravitational wave consistency relations for multifield inflation

We study the tensor spectral index $n_t$ and the tensor-to-scalar ratio $r$ in the simplest multifield extension to single-field, slow-roll inflation models. We show that multifield models with potentials $V \sim \sum_i λ_i |ϕ_i|^p$ have different predictions for $n_t/r$ than single-field models, even when all the couplings are equal $λ_i=λ_j$, due to the probabilistic nature of the fields' initial values. We analyze well-motivated prior probabilities for the $λ_i$ and initial conditions to make detailed predictions for the marginalized probability distribution of $n_t/r$. With $\mathcal O(100)$ fields and $p>3/4$, we find that $n_t/r$ differs from the single-field result of $n_t/r=-1/8$ at the 5$σ$ level. This gives a novel and testable prediction for the simplest multifield inflation models.

astro-ph.CO↗