arXiv subjects

Pedro Delicado

Publications and source records attributed to Pedro Delicado.

9 recordsLinked to original sources

arXiv2026

Autorelevance function and other feature relevance measures for univariate time series

We propose a model agnostic methodology to measure lag relevance in machine learning forecasting models applied to univariate time series. Particularly, we are working in the context of time series using the frameworks of Ghost variables and Shapley values, together with additive importance measures, to introduce the auto-relevance and partial auto-relevance functions as the lag importance values. Additionally, we propose a novel method to replace absent features in coalition based methods with a one step forecast from the same model. We evaluate these proposals under different simulations and real data cases. This combined framework perspective is particularly suitable for time series. In addition, to show our discoveries we use a pull of models from the seasonal ARMA family and recurrent neural networks. We found that the calculated relevance measures successfully demonstrate the expected lag structure in almost all cases.

stat.ML↗

arXiv2025

Modeling continuous monitoring glucose curves by Beta generalized non-parametric models

We present a functional data analysis approach for studying time-dependent, continuous glucose monitoring data with repeated measures for each individual in an experiment. After scaling the glucose concentration curves to the interval [0, 1], we model them by using a Beta distribution with two time-varying parameters. In this context, we develop a local linear maximum likelihood smoothing procedure that is valid when more than one parameter depends on time. Our approach requires much fewer observations than previous functional methods for this setting and is also applicable when only one individual (or a few) is available. We evaluate the performance of our estimator in terms of computation time and model fit using a synthetic dataset as well as a large, real clinical trial dataset. We also compare our method with existing methods in the literature. From a methodological point of view, we contribute to extend local likelihood estimation from one to two time-varying parameters by developing theoretical expressions for estimation and for approximating the leave-one-out cross-validation. Moreover, we show that this kernel-based approach competes with spline-based estimation methods, the dominant line of functional regression models today.

stat.ME↗

arXiv2024

Functional relevance based on the continuous Shapley value

The presence of artificial intelligence (AI) in our society is increasing, which brings with it the need to understand the behavior of AI mechanisms, including machine learning predictive algorithms fed with tabular data, text or images, among others. This work focuses on interpretability of predictive models based on functional data. Designing interpretability methods for functional data models implies working with a set of features whose size is infinite. In the context of scalar on function regression, we propose an interpretability method based on the Shapley value for continuous games, a mathematical formulation that allows for the fair distribution of a global payoff among a continuous set of players. The method is illustrated through a set of experiments with simulated and real data sets. The open source Python package ShapleyFDA is also presented.

stat.ML↗

arXiv2024

Functional Data Analysis on Wearable Sensor Data: A Systematic Review

Wearable devices and sensors have recently become a popular way to collect data, especially in the health sciences. The use of sensors allows patients to be monitored over a period of time with a high observation frequency. Due to the continuous-on-time structure of the data, novel statistical methods are recommended for the analysis of sensor data. One of the popular approaches in the analysis of wearable sensor data is functional data analysis. The main objective of this paper is to review functional data analysis methods applied to wearable device data according to the type of sensor. In addition, we introduce several freely available software packages and open databases of wearable device data to facilitate access to sensor data in different fields.

stat.ME↗

arXiv2023

SurvLIMEpy: A Python package implementing SurvLIME

In this paper we present SurvLIMEpy, an open-source Python package that implements the SurvLIME algorithm. This method allows to compute local feature importance for machine learning algorithms designed for modelling Survival Analysis data. Our implementation takes advantage of the parallelisation paradigm as all computations are performed in a matrix-wise fashion which speeds up execution time. Additionally, SurvLIMEpy assists the user with visualization tools to better understand the result of the algorithm. The package supports a wide variety of survival models, from the Cox Proportional Hazards Model to deep learning models such as DeepHit or DeepSurv. Two types of experiments are presented in this paper. First, by means of simulated data, we study the ability of the algorithm to capture the importance of the features. Second, we use three open source survival datasets together with a set of survival algorithms in order to demonstrate how SurvLIMEpy behaves when applied to different models.

stat.ML↗

arXiv2021

Wildfires vegetation recovery through satellite remote sensing and Functional Data Analysis

In recent years wildfires have caused havoc across the world, especially aggravated in certain regions, due to climate change. Remote sensing has become a powerful tool for monitoring fires, as well as for measuring their effects on vegetation over the following years. We aim to explain the dynamics of wildfires' effects on a vegetation index (previously estimated by causal inference through synthetic controls) from pre-wildfire available information (mainly proceeding from satellites). For this purpose, we use regression models from Functional Data Analysis, where wildfire effects are considered functional responses, depending on elapsed time after each wildfire, while pre-wildfire information acts as scalar covariates. Our main findings show that vegetation recovery after wildfires is a slow process, affected by many pre-wildfire conditions, among which the richness and diversity of vegetation is one of the best predictors for the recovery.

stat.AP↗

arXiv2021

Estimating heterogeneous wildfire effects using synthetic controls and satellite remote sensing

Wildfires have become one of the biggest natural hazards for environments worldwide. The effects of wildfires are heterogeneous, meaning that the magnitude of their effects depends on many factors such as geographical region, climate and land cover/vegetation type. Yet, which areas are more affected by these events remains unclear. Here we present a novel application of the Generalised Synthetic Control (GSC) method that enables quantification and prediction of vegetation changes due to wildfires through a time-series analysis of in situ and satellite remote sensing data. We apply this method to medium to large wildfires ($>$ 1000 acres) in California throughout a time-span of two decades (1996--2016). The method's ability for estimating counterfactual vegetation characteristics for burned regions is explored in order to quantify abrupt system changes. We find that the GSC method is better at predicting vegetation changes than the more traditional approach of using nearby regions to assess wildfire impacts. We evaluate the GSC method by comparing its predictions of spectral vegetation indices to observations during pre-wildfire periods and find improvements in correlation coefficient from $R^2 = 0.66$ to $R^2 = 0.93$ in Normalised Difference Vegetation Index (NDVI), from $R^2 = 0.48$ to $R^2 = 0.81$ for Normalised Burn Ratio (NBR), and from $R^2 = 0.49$ to $R^2 = 0.85$ for Normalised Difference Moisture Index (NDMI). Results show greater changes in NDVI, NBR, and NDMI post-fire on regions classified as having a lower Burning Index. The GSC method also reveals that wildfire effects on vegetation can last for more than a decade post-wildfire, and in some cases never return to their previous vegetation cycles within our study period. Lastly, we discuss the usefulness of using GSC in remote sensing analyses.

stat.AP↗

arXiv2020

Multidimensional Scaling for Big Data

We present a set of algorithms implementing multidimensional scaling (MDS) for large data sets. MDS is a family of dimensionality reduction techniques using a $n \times n$ distance matrix as input, where $n$ is the number of individuals, and producing a low dimensional configuration: a $n\times r$ matrix with $r<<n$. When $n$ is large, MDS is unaffordable with classical MDS algorithms because of their extremely large memory and time requirements. We compare six non-standard algorithms intended to overcome these difficulties. They are based on the central idea of partitioning the data set into small pieces, where classical MDS methods can work. Two of these algorithms are original proposals. In order to check the performance of the algorithms as well as to compare them, we have done a simulation study. Additionally, we have used the algorithms to obtain an MDS configuration for EMNIST: a real large data set with more than $800000$ points. We conclude that all the algorithms are appropriate to use for obtaining an MDS configuration, but we recommend using one of our proposals since it is a fast algorithm with satisfactory statistical properties when working with big data. An R package implementing the algorithms has been created.

stat.CO↗

arXiv2020

Understanding complex predictive models with Ghost Variables

We propose a procedure for assigning a relevance measure to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check the out of sample performance. First, the individual relevance of each variable is computed by comparing the predictions in the test set, given by the model that includes all the variables with those of another model in which the variable of interest is substituted by its ghost variable, defined as the prediction of this variable by using the rest of explanatory variables. Second, we check the joint effects among the variables by using the eigenvalues of a relevance matrix that is the covariance matrix of the vectors of individual effects. It is shown that in simple models, as linear or additive models, the proposed measures are related to standard measures of significance of the variables and in neural networks models (and in other algorithmic prediction models) the procedure provides information about the joint and individual effects of the variables that is not usually available by other methods. The procedure is illustrated with simulated examples and the analysis of a large real data set.

stat.ML↗