arXiv subjects

Joonha Park

Publications and source records attributed to Joonha Park.

11 recordsLinked to original sources

arXiv2026

ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMs

Parameter-efficient fine-tuning (PEFT) of mixture-of-experts (MoE) models commonly attaches a separate low-rank adapter to each expert. This expert-wise design fragments adaptation in three ways: capacity is split across narrow low-rank updates, gradient supervision becomes sparse and imbalanced under sparse routing, and execution is decomposed into many small GEMMs. We find that such expert-wise separation is often unnecessary, as subsets of LoRA adapters become functionally similar during fine-tuning, revealing redundancy among expert-specific adapters. Based on this redundancy, we propose ACE (Adapter Consolidation across Experts), which groups redundant experts and replaces their expert-specific adapters with group-shared higher-rank LoRA modules under the same PEFT budget. ACE further introduces grouped adapter execution, which consolidates fragmented expert-wise adapter computations into fewer, larger group-level GEMMs. Across evaluations covering 12 datasets and four MoE backbones, ACE achieves the highest observed mean accuracy among the parameter-matched PEFT methods on the three backbones with complete baseline coverage, while providing $1.31\times$ to $1.48\times$ wall-clock training speedup over expert-wise LoRA without increasing peak memory. Our code is available at https://github.com/UbiquitousAILab/ACE.

cs.LG↗

arXiv2026

Premover: Fast Vision-Language-Action Control by Acting Before Instructions Are Complete

Vision-Language-Action (VLA) policies are typically evaluated as if the user had finished typing or speaking before the robot begins acting. In real deployment, however, users take several seconds to enter a request, leaving the policy idle for a substantial fraction of the interaction. We introduce Premover, a lightweight module that converts this idle window into useful precomputation. Premover keeps the VLA backbone frozen and attaches two small projection heads, one for image patches, one for language tokens, that map an intermediate layer of the backbone into a shared space. The resulting focus map is supervised by simulator-rendered target-object segmentation masks and applied as a per-patch reweighting of the next step's image tokens. A single scalar readiness threshold, trained jointly from streaming prefixes, decides when the policy should begin acting. On the LIBERO benchmark suite, Premover reduces mean wall-clock time from 34.0 to 29.4 seconds, a 13.6% reduction, while matching the full-prompt baseline's success rate (95.1% vs. 95.0%); naive premoving, by contrast, collapses to 66.4%.

cs.RO↗

arXiv2026

Modular Markov chain Monte Carlo with application to multimodal sampling

We develop a modular approach to Markov chain Monte Carlo (MCMC) sampling for unnormalized target densities. In this approach, Markov chains are constructed in parallel, each constrained to a subset of the target space. The Monte Carlo estimates from the constrained chains are then combined with appropriate weights, calculated from the transition probabilities between subsets. In addition to the computational advantages arising from its parallelized structure, this modular MCMC approach enables variance reduction for Monte Carlo estimation in settings where sampling from low-density regions is required. We develop a central limit theorem-type result for the resulting Monte Carlo estimates and propose a method for estimating their standard errors. Furthermore, by applying this modular sampling technique to simulated tempering, we propose a method for Monte Carlo estimation of expectations with respect to multimodal target distributions. This approach effectively addresses a well-known challenge of tempering-based methods: sampling efficiency can be greatly reduced when separated modes of the target distribution have different scales. We demonstrate the efficiency of the proposed methods through numerical examples, including one arising from Bayesian sparse regression with a spike-and-slab prior.

stat.CO↗

arXiv2026

A novel finite-sample testing procedure for composite null hypotheses via pointwise rejection

We propose a novel finite-sample procedure for testing composite null hypotheses. Traditional likelihood ratio tests based on asymptotic $\chi^2$ approximations often exhibit substantial bias in small samples. Our procedure rejects the composite null hypothesis $H_0: \theta \in \Theta_0$ if the simple null hypothesis $H_0: \theta = \theta_t$ is rejected for every $\theta_t$ in the null region $\Theta_0$, using an inflated significance level. We derive formulas that determine this inflated level so that the overall test approximately maintains the desired significance level even with small samples. Whereas the traditional likelihood ratio test applies when the null region is defined solely by equality constraints--that is, when it forms a manifold without boundary--the proposed approach extends to null hypotheses defined by both equality and inequality constraints. In addition, it accommodates null hypotheses expressed as unions of several component regions and can be applied to models involving nuisance parameters. Through several examples featuring nonstandard composite null hypotheses, we demonstrate numerically that the proposed test achieves accurate inference, exhibiting only a small gap between the actual and nominal significance levels for both small and large samples.

stat.ME↗

arXiv2024

A tutorial on spatiotemporal partially observed Markov process models via the R package spatPomp

We describe a computational framework for modeling and statistical inference on high-dimensional stochastic dynamic systems. Our primary motivation is the investigation of metapopulation dynamics arising from a collection of spatially distributed, interacting biological populations. To make progress on this goal, we embed it in a more general problem: inference for a collection of interacting partially observed nonlinear non-Gaussian stochastic processes. Each process in the collection is called a unit; in the case of spatiotemporal models, the units correspond to distinct spatial locations. The dynamic state for each unit may be discrete or continuous, scalar or vector valued. In metapopulation applications, the state can represent a structured population or the abundances of a collection of species at a single location. We consider models where the collection of states has a Markov property. A sequence of noisy measurements is made on each unit, resulting in a collection of time series. A model of this form is called a spatiotemporal partially observed Markov process (SpatPOMP). The R package spatPomp provides an environment for implementing SpatPOMP models, analyzing data using existing methods, and developing new inference approaches. Our presentation of spatPomp reviews various methodologies in a unifying notational framework. We demonstrate the package on a simple Gaussian system and on a nontrivial epidemiological model for measles transmission within and between cities. We show how to construct user-specified SpatPOMP models within spatPomp.

stat.ME↗

arXiv2023

Scalable simulation-based inference for implicitly defined models using a metamodel for Monte Carlo log-likelihood estimator

Models implicitly defined through a random simulator of a process have become widely used in scientific and industrial applications in recent years. However, simulation-based inference methods for such implicit models, like approximate Bayesian computation (ABC), often scale poorly as data size increases. We develop a scalable inference method for implicitly defined models using a metamodel for the Monte Carlo log-likelihood estimator derived from simulations. This metamodel characterizes both statistical and simulation-based randomness in the distribution of the log-likelihood estimator across different parameter values. Our metamodel-based method quantifies uncertainty in parameter estimation in a principled manner, leveraging the local asymptotic normality of the mean function of the log-likelihood estimator. We apply this method to construct accurate confidence intervals for parameters of partially observed Markov process models where the Monte Carlo log-likelihood estimator is obtained using the bootstrap particle filter. We numerically demonstrate that our method enables accurate and highly scalable parameter inference across several examples, including a mechanistic compartment model for infectious diseases.

stat.ME↗

arXiv2021

Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is widely used for sampling from high dimensional target distributions with densities known up to proportionality. While HMC exhibits favorable scaling properties in high dimensions, it struggles with strongly multimodal distributions. Tempering methods are commonly used to address multimodality, but they can be difficult to tune, especially in high dimensional settings. In this study, we propose a method that combines tempering with HMC to enable efficient sampling from high dimensional, strongly multimodal distributions. Our approach simulates the dynamics of a time-varying Hamiltonian in which the temperature increases and then decreases over time. In the first phase, the simulated trajectory gradually explores low-density regions farther from the mode; the second phase guides it back toward a local mode. We develop efficient tuning strategies based on a time-scale transformation under which the Hamiltonian becomes approximately stationary. This leads to a tempered Hamiltonian Monte Carlo (THMC) algorithm with automatic tuning. We demonstrate numerically that our method scales more effectively with dimension than adaptive parallel tempering and tempered sequential Monte Carlo. Finally, we apply our THMC to sample from strongly multimodal posterior distributions arising in Bayesian inference.

stat.CO↗

arXiv2021

Bagged filters for partially observed interacting systems

Bagging (i.e., bootstrap aggregating) involves combining an ensemble of bootstrap estimators. We consider bagging for inference from noisy or incomplete measurements on a collection of interacting stochastic dynamic systems. Each system is called a unit, and each unit is associated with a spatial location. A motivating example arises in epidemiology, where each unit is a city: the majority of transmission occurs within a city, with smaller yet epidemiologically important interactions arising from disease transmission between cities. Monte Carlo filtering methods used for inference on nonlinear non-Gaussian systems can suffer from a curse of dimensionality as the number of units increases. We introduce bagged filter (BF) methodology which combines an ensemble of Monte Carlo filters, using spatiotemporally localized weights to select successful filters at each unit and time. We obtain conditions under which likelihood evaluation using a BF algorithm can beat a curse of dimensionality, and we demonstrate applicability even when these conditions do not hold. BF can out-perform an ensemble Kalman filter on a coupled population dynamics model describing infectious disease transmission. A block particle filter also performs well on this task, though the bagged filter respects smoothness and conservation laws that a block particle filter can violate.

stat.ME↗

arXiv2020

Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter

We propose a method for inference on moderately high-dimensional, nonlinear, non-Gaussian, partially observed Markov process models for which the transition density is not analytically tractable. Markov processes with intractable transition densities arise in models defined implicitly by simulation algorithms. Widely used particle filter methods are applicable to nonlinear, non-Gaussian models but suffer from the curse of dimensionality. Improved scalability is provided by ensemble Kalman filter methods, but these are inappropriate for highly nonlinear and non-Gaussian models. We propose a particle filter method having improved practical and theoretical scalability with respect to the model dimension. This method is applicable to implicitly defined models having analytically intractable transition densities. Our method is developed based on the assumption that the latent process is defined in continuous time and that a simulator of this latent process is available. In this method, particles are propagated at intermediate time intervals between observations and are resampled based on a forecast likelihood of future observations. We combine this particle filter with parameter estimation methodology to enable likelihood-based inference for highly nonlinear spatiotemporal systems. We demonstrate our methodology on a stochastic Lorenz 96 model and a model for the population dynamics of infectious diseases in a network of linked regions.

stat.ME↗

arXiv2019

Markov chain Monte Carlo algorithms with sequential proposals

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis-Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

stat.CO↗

arXiv2017

Monte Carlo profile confidence intervals

Monte Carlo methods to evaluate and maximize the likelihood function enable the construction of confidence intervals and hypothesis tests, facilitating scientific investigation using models for which the likelihood function is intractable. When Monte Carlo error can be made small, by sufficiently exhaustive computation, then the standard theory and practice of likelihood-based inference applies. As data become larger, and models more complex, situations arise where no reasonable amount of computation can render Monte Carlo error negligible. We develop profile likelihood methodology to provide frequentist inferences that take into account Monte Carlo uncertainty. We investigate the role of this methodology in facilitating inference for computationally challenging dynamic latent variable models. We present three examples arising in the study of infectious disease transmission. These three examples demonstrate our methodology for inference on nonlinear dynamic models using genetic sequence data, panel time series data, and spatiotemporal data. We also discuss applicability to nonlinear time series analysis.

stat.ME↗