arXiv subjects

Manuel Pulido

Publications and source records attributed to Manuel Pulido.

15 recordsLinked to original sources

arXiv2026

Data-Driven Crowd Dynamics using Kinetic Theory and Ensemble-based Data Assimilation

Understanding pedestrian dynamics is critical for mitigating crowd-related risks and improving public safety. In this work, we propose a data-driven mesoscopic modeling framework that combines the kinetic theory of active particles with data assimilation techniques. The framework uses an ensemble Kalman filter to sequentially estimate the time-dependent spatial distribution of pedestrians and model parameters by fusing observations with the mesoscopic forward model state. Through a series of twin experiments, we show that the panic factor -- a key behavioral parameter -- is identifiable within this framework. We also evaluate the robustness of the approach by assimilating synthetic observations generated by an agent-based model (ABM). This setup introduces structural model error because the ABM is governed by microscopic rules that differ fundamentally from the mesoscopic kinetic equations. Despite this discrepancy, the ensemble Kalman filter, through the observation-based innovation term, successfully drives the kinetic model to track the observed pedestrian density while simultaneously recovering the panic factor. In this framework, observations act as a physical constraint on the evolution of the kinetic model. Crowd-dynamics models, including ABMs and kinetic models, often rely on phenomenological terms to describe social interactions, with parameters that are highly uncertain. Our findings indicate that in such systems, free-running simulations inevitably may diverge from the true state, whereas an online data-driven approach effectively constrains the system's trajectory to its underlying dynamical manifold.

physics.soc-ph↗

arXiv2025

The impact of fine tuning in LLaMA on hallucinations for named entity extraction in legal documentation

The extraction of information about traffic accidents from legal documents is crucial for quantifying insurance company costs. Extracting entities such as percentages of physical and/or psychological disability and the involved compensation amounts is a challenging process, even for experts, due to the subtle arguments and reasoning in the court decision. A two-step procedure is proposed: first, segmenting the document identifying the most relevant segments, and then extracting the entities. For text segmentation, two methodologies are compared: a classic method based on regular expressions and a second approach that divides the document into blocks of n-tokens, which are then vectorized using multilingual models for semantic searches (text-embedding-ada-002/MiniLM-L12-v2 ). Subsequently, large language models (LLaMA-2 7b, 70b, LLaMA-3 8b, and GPT-4 Turbo) are applied with prompting to the selected segments for entity extraction. For the LLaMA models, fine-tuning is performed using LoRA. LLaMA-2 7b, even with zero temperature, shows a significant number of hallucinations in extractions which are an important contention point for named entity extraction. This work shows that these hallucinations are substantially reduced after finetuning the model. The performance of the methodology based on segment vectorization and subsequent use of LLMs significantly surpasses the classic method which achieves an accuracy of 39.5%. Among open-source models, LLaMA-2 70B with finetuning achieves the highest accuracy 79.4%, surpassing its base version 61.7%. Notably, the base LLaMA-3 8B model already performs comparably to the finetuned LLaMA-2 70B model, achieving 76.6%, highlighting the rapid progress in model development. Meanwhile, GPT-4 Turbo achieves the highest accuracy at 86.1%.

cs.CL↗

arXiv2023

Transmission matrix parameter estimation of COVID-19 evolution with age compartments using ensemble-based data assimilation

The COVID-19 pandemic and its multiple outbreaks have challenged governments around the world. Much of the epidemiological modeling was based on pre-pandemic contact information of the population, which changed drastically due to governmental health measures, so called non-pharmaceutical interventions made to reduce transmission of the virus, like social distancing and complete lockdown. In this work, we evaluate an ensemble-based data assimilation framework applied to a meta-population model to infer the transmission of the disease between different population agegroups. We perform a set of idealized twin-experiments to investigate the performance of different possible parameterizations of the transmission matrix. These experiments show that it is not possible to unambiguously estimate all the independent parameters of the transmission matrix. However, under certain parameterizations, the transmission matrix in an age-compartmental model can be estimated. These estimated parameters lead to an increase of forecast accuracy in agegroups compartments assimilating age-dependent accumulated cases and deaths observed in Argentina compared to a single-compartment model, and reliable estimations of the effective reproduction number. The age-dependent data assimilation and forecasting of virus transmission may be important for an accurate prediction and diagnosis of health care demand.

q-bio.PE↗

arXiv2023

Online machine-learning forecast uncertainty estimation for sequential data assimilation

Quantifying forecast uncertainty is a key aspect of state-of-the-art numerical weather prediction and data assimilation systems. Ensemble-based data assimilation systems incorporate state-dependent uncertainty quantification based on multiple model integrations. However, this approach is demanding in terms of computations and development. In this work a machine learning method is presented based on convolutional neural networks that estimates the state-dependent forecast uncertainty represented by the forecast error covariance matrix using a single dynamical model integration. This is achieved by the use of a loss function that takes into account the fact that the forecast errors are heterodastic. The performance of this approach is examined within a hybrid data assimilation method that combines a Kalman-like analysis update and the machine learning based estimation of a state-dependent forecast error covariance matrix. Observing system simulation experiments are conducted using the Lorenz'96 model as a proof-of-concept. The promising results show that the machine learning method is able to predict precise values of the forecast covariance matrix in relatively high-dimensional states. Moreover, the hybrid data assimilation method shows similar performance to the ensemble Kalman filter outperforming it when the ensembles are relatively small.

physics.ao-ph↗

arXiv2022

Evaluation of Machine Learning Techniques for Forecast Uncertainty Quantification

Ensemble forecasting is, so far, the most successful approach to produce relevant forecasts with an estimation of their uncertainty. The main limitations of ensemble forecasting are the high computational cost and the difficulty to capture and quantify different sources of uncertainty, particularly those associated with model errors. In this work we perform toy-model and state-of-the-art model experiments to analyze to what extent artificial neural networks (ANNs) are able to model the different sources of uncertainty present in a forecast. In particular those associated with the accuracy of the initial conditions and those introduced by the model error. We also compare different training strategies: one based on a direct training using the mean and spread of an ensemble forecast as target, the other ones rely on an indirect training strategy using an analyzed state as target in which the uncertainty is implicitly learned from the data. Experiments using the Lorenz'96 model show that the ANNs are able to emulate some of the properties of ensemble forecasts like the filtering of the most unpredictable modes and a state-dependent quantification of the forecast uncertainty. Moreover, ANNs provide a reliable estimation of the forecast uncertainty in the presence of model error. Preliminary experiments conducted with a state-of-the-art forecasting system also confirm the ability of ANNs to produce a reliable quantification of the forecast uncertainty.

cs.LG↗

arXiv2021

Inference in epidemiological agent-based models using ensemble-based data assimilation

To represent the complex individual interactions in the dynamics of disease spread informed by data, the coupling of an epidemiological agent-based model with the ensemble Kalman filter is proposed. The statistical inference of the propagation of a disease by means of ensemble-based data assimilation systems has been studied in previous work. The models used are mostly compartmental models representing the mean field evolution through ordinary differential equations. These techniques allow to monitor the propagation of the infections from data and to estimate several parameters of epidemiological interest. However, there are many important features which are based on the individual interactions that cannot be represented in the mean field equations, such as social network and bubbles, contact tracing, isolating individuals in risk, and social network-based distancing strategies. Agent-based models can describe contact networks at an individual level, including demographic attributes such as age, neighbourhood, household, workplaces, schools, entertainment places, among others. Nevertheless, these models have several unknown parameters which are thus difficult to estimate. In this work, we propose the use of ensemble-based data assimilation techniques to calibrate an agent-based model using daily epidemiological data. This raises the challenge of having to adapt the agent populations to incorporate the information provided by the coarse-grained data. To do this, two stochastic strategies to correct the model predictions are developed. The ensemble Kalman filter with perturbed observations is used for the joint estimation of the state and some key epidemiological parameters. We conduct experiments with an agent based-model designed for COVID-19 and assess the proposed methodology on synthetic data and on COVID-19 daily reports from Ciudad Autónoma de Buenos Aires, Argentina.

stat.AP↗

arXiv2021

A Framework for Causal Discovery in non-intervenable systems

Many frameworks exist to infer cause and effect relations in complex nonlinear systems but a complete theory is lacking. A new framework is presented that is fully nonlinear, provides a complete information theoretic disentanglement of causal processes, allows for nonlinear interactions between causes, identifies the causal strength of missing or unknown processes, and can analyze systems that cannot be represented on Directed Acyclic Graphs. The basic building blocks are information theoretic measures such as (conditional) mutual information and a new concept called certainty that monotonically increases with the information available about the target process. The framework is presented in detail and compared with other existing frameworks, and the treatment of confounders is discussed. While there are systems with structures that the framework cannot disentangle, it is argued that any causal framework that is based on integrated quantities will miss out potentially important information of the underlying probability density functions. The framework is tested on several highly simplified stochastic processes to demonstrate how blocking and gateways are handled, and on the chaotic Lorentz 1963 system. We show that the framework provides information on the local dynamics, but also reveals information on the larger scale structure of the underlying attractor. Furthermore, by applying it to real observations related to the El-Nino-Southern-Oscillation system we demonstrate its power and advantage over other methodologies.

stat.ME↗

arXiv2020

A Review of Innovation-Based Methods to Jointly Estimate Model and Observation Error Covariance Matrices in Ensemble Data Assimilation

Data assimilation combines forecasts from a numerical model with observations. Most of the current data assimilation algorithms consider the model and observation error terms as additive Gaussian noise, specified by their covariance matrices Q and R, respectively. These error covariances, and specifically their respective amplitudes, determine the weights given to the background (i.e., the model forecasts) and to the observations in the solution of data assimilation algorithms (i.e., the analysis). Consequently, Q and R matrices significantly impact the accuracy of the analysis. This review aims to present and to discuss, with a unified framework, different methods to jointly estimate the Q and R matrices using ensemble-based data assimilation techniques. Most of the methodologies developed to date use the innovations, defined as differences between the observations and the projection of the forecasts onto the observation space. These methodologies are based on two main statistical criteria: (i) the method of moments, in which the theoretical and empirical moments of the innovations are assumed to be equal, and (ii) methods that use the likelihood of the observations, themselves contained in the innovations. The reviewed methods assume that innovations are Gaussian random variables, although extension to other distributions is possible for likelihood-based methods. The methods also show some differences in terms of levels of complexity and applicability to high-dimensional systems. The conclusion of the review discusses the key challenges to further develop estimation methods for Q and R. These challenges include taking into account time-varying error covariances, using limited observational coverage, estimating additional deterministic error terms, or accounting for correlated noises.

stat.ME↗

arXiv2020

Model error covariance estimation in particle and ensemble Kalman filters using an online expectation-maximization algorithm

The performance of ensemble-based data assimilation techniques that estimate the state of a dynamical system from partial observations depends crucially on the prescribed uncertainty of the model dynamics and of the observations. These are not usually known and have to be inferred. Many approaches have been proposed to tackle this problem, including fully Bayesian, likelihood maximization and innovation-based techniques. This work focuses on maximization of the likelihood function via the expectation-maximization (EM) algorithm to infer the model error covariance combined with ensemble Kalman filters and particle filters to estimate the state. The classical application of the EM algorithm in a data assimilation context involves filtering and smoothing a fixed batch of observations in order to complete a single iteration. This is an inconvenience when using sequential filtering in high-dimensional applications. Motivated by this, an adaptation of the algorithm that can process observations and update the parameters on the fly, with some underlying simplifications, is presented. The proposed technique was evaluated and achieved good performance in experiments with the Lorenz-63 and the 40-variable Lorenz-96 dynamical systems designed to represent some common scenarios in data assimilation such as non-linearity, chaoticity and model misspecification.

stat.CO↗

arXiv2019

Model uncertainty estimation using the expectation maximization algorithm and a particle flow filter

Model error covariances play a central role in the performance of data assimilation methods applied to nonlinear state-space models. However, these covariances are largely unknown in most of the applications. A misspecification of the model error covariance has a strong impact on the computation of the posterior probability density function, leading to unreliable estimations and even to a total failure of the assimilation procedure. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance, using a batch of observations. Based on the EM algorithm principles, the proposed method encompasses two stages: the expectation stage, in which a particle filter is used with the present estimate of the model error covariance as given to find the probability density function that maximizes the likelihood, followed by a maximization stage in which the expectation under the probability density function found in the expectation step is maximized as a function of the elements of the model error covariance. This novel algorithm here presented combines the EM with a fixed point algorithm and does not require a particle smoother to approximate the posterior densities. We demonstrate that the new method accurately and efficiently solves the linear model problem. Furthermore, for the chaotic nonlinear Lorenz-96 model the method is stable even for observation error covariance 10 times larger than the estimated model error covariance matrix, and also that it is successful in high-dimensional situations where the dimension of the estimated matrix is 1600.

stat.CO↗

arXiv2019

Kernel embedded nonlinear observational mappings in the variational mapping particle filter

Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point samples from an intermediate density to a posterior density. The local mappings that transform the intermediate densities are embedded in a reproducing kernel Hilbert space (RKHS). This variational mapping method requires the evaluation of the log-posterior density gradient and therefore the adjoint of the observational operator. In this work, we evaluate nonlinear observational mappings in the variational mapping method using two approximations that avoid the adjoint, an ensemble based approximation in which the gradient is approximated by the particle covariances in the state and observational spaces the so-called ensemble space and an RKHS approximation in which the observational mapping is embedded in an RKHS and the gradient is derived there. The approximations are evaluated for highly nonlinear observational operators and in a low-dimensional chaotic dynamical system. The RKHS approximation is shown to be highly successful and superior to the ensemble approximation.

stat.ML↗

arXiv2018

Inference of stochastic parameterizations for model error treatment using nested ensemble Kalman filters

Stochastic parameterizations are increasingly being used to represent the uncertainty associated with model errors in ensemble forecasting and data assimilation. One of the challenges associated with the use of these parameterizations is the optimization of the properties of the stochastic forcings within their formulation. In this work a hierarchical data assimilation approach based on two nested ensemble Kalman filters is proposed for inferring parameters associated with a stochastic parameterization. The proposed technique is based on the Rao-Blackwellization of the parameter estimation problem. The technique consists in using an ensemble of ensemble Kalman filters, each of them using a different set of stochastic parameter values. We show the ability of the technique to infer parameters related to the covariance structure of stochastic representations of model error in the Lorenz-96 dynamical system. The evaluation is conducted with stochastic twin experiments and imperfect model experiments with unresolved physics in the forecast model. The proposed technique performs successfully under different model error covariance structures. The technique is proposed to be applied offline as part of an a priori optimization of the data assimilation system and could in principle be extended to the estimation of other hyperparameters of a data assimilation system.

stat.CO↗

arXiv2018

Kernel embedding of maps for sequential Bayesian inference: The variational mapping particle filter

In this work, a novel sequential Monte Carlo filter is introduced which aims at efficient sampling of high-dimensional state spaces with a limited number of particles. Particles are pushed forward from the prior to the posterior density using a sequence of mappings that minimizes the Kullback-Leibler divergence between the posterior and the sequence of intermediate densities. The sequence of mappings represents a gradient flow. A key ingredient of the mappings is that they are embedded in a reproducing kernel Hilbert space, which allows for a practical and efficient algorithm. The embedding provides a direct means to calculate the gradient of the Kullback-Leibler divergence leading to quick convergence using well-known gradient-based stochastic optimization algorithms. Evaluation of the method is conducted in the chaotic Lorenz-63 system, the Lorenz-96 system, which is a coarse prototype of atmospheric dynamics, and an epidemic model that describes cholera dynamics. No resampling is required in the mapping particle filter even for long recursive sequences. The number of effective particles remains close to the total number of particles in all the experiments.

stat.ML↗

arXiv2017

Stochastic parameterization identification using ensemble Kalman filtering combined with expectation-maximization and Newton-Raphson maximum likelihood methods

For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal succesfully because both suffer from the collapse of the parameter posterior distribution. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of two methodologies: the maximization of the likelihood via Expectation-Maximization (EM) and Newton-Raphson (NR) algorithms which are mainly applied in the statistic and machine learning communities, and the ensemble Kalman filter (EnKF). The methods are derived using a Bayesian approach for a hidden Markov model. They are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof-of-concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorpored to represent the small-scale dynamics. The algorithms are able to identify an optimal stochastic parameterization with a good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.

physics.ao-ph↗

arXiv2015

DADA: Data Assimilation for the Detection and Attribution of Weather- and Climate-related Events

We describe a new approach allowing for systematic causal attribution of weather and climate-related events, in near-real time. The method is purposely designed to facilitate its implementation at meteorological centers by relying on data treatments that are routinely performed when numerically forecasting the weather. Namely, we show that causal attribution can be obtained as a by-product of so-called data assimilation procedures that are run on a daily basis to update the meteorological model with new atmospheric observations; hence, the proposed methodology can take advantage of the powerful computational and observational capacity of weather forecasting centers. We explain the theoretical rationale of this approach and sketch the most prominent features of a "data assimilation-based detection and attribution" (DADA) procedure. The proposal is illustrated in the context of the classical three-variable Lorenz model with additional forcing. Several theoretical and practical research questions that need to be addressed to make the proposal readily operational within weather forecasting centers are finally laid out.

stat.AP↗