arXiv subjects

Camila P. E. de Souza

Publications and source records attributed to Camila P. E. de Souza.

15 recordsLinked to original sources

arXiv2026

Bandwidth-free nonparametric density estimation for grouped data

In some situations, data is collected under systematical and technical constraints due to uncertainty in experimental reports, intermittent measurements, confidentiality, and non-detects. For this reason, it might not be possible to retrieve or receive the data in a conventional format but rather in a grouped form where only the number of occurrences is known within intervals. The challenge is to estimate the density of the underlying ungrouped data based on the observed grouped data with no information regarding the underlying distribution. To overcome this problem, this study introduces a mean-adjusted log-concave (MALC) density estimation method for univariate grouped data, aiming to provide a bandwidth-free non-parametric approach that does not rely on specific distributional assumptions. The performance of the MALC method is evaluated through simulations across various distributions with different sample sizes and grid widths. The results demonstrate the robustness and effectiveness of the MALC approach in grouped data analysis, offering a broader range of applications over traditional methods.

stat.ME↗

arXiv2026

Bayesian Variable Selection for Function-on-Scalar Regression Models: a comparative analysis

In this work, we developed a new Bayesian method for variable selection in function-on-scalar regression (FOSR). Our method uses a hierarchical Bayesian structure and latent variables to enable an adaptive covariate selection process for FOSR. Extensive simulation studies show the proposed method's main properties, such as its accuracy in estimating the coefficients and high capacity to select variables correctly. Furthermore, we conducted a substantial comparative analysis with the main competing methods, the BGLSS (Bayesian Group Lasso with Spike and Slab prior) method, the group LASSO (Least Absolute Shrinkage and Selection Operator), the group MCP (Minimax Concave Penalty), and the group SCAD (Smoothly Clipped Absolute Deviation). Our results demonstrate that the proposed methodology is superior in correctly selecting covariates compared with the existing competing methods while maintaining a satisfactory level of goodness of fit. In contrast, the competing methods could not balance selection accuracy with goodness of fit. We also considered a COVID-19 dataset and some socioeconomic data from Brazil as an application and obtained satisfactory results. In short, the proposed Bayesian variable selection model is highly competitive, showing significant predictive and selective quality.

stat.ME↗

arXiv2026

Variational Inference for Variable Selection in Scalar-on-Function Regression

In practical regression applications, multiple covariates are often measured, but not all may be associated with the response variable. Identifying and including only the relevant covariates in the model is crucial for improving prediction accuracy. In this work, we develop a variational inference approach for estimation and variable selection in scalar-on-function regression, involving only functional covariates, and in partially functional regression models that also include scalar covariates. Specifically, we develop a variational expectation-maximization (VEM) algorithm, with a variational Bayes procedure implemented in the E-step to obtain approximate marginal posterior distributions for most model parameters, except for the regularization parameters, which are updated in the M-step. Our method accurately identifies relevant covariates while maintaining strong predictive performance, as demonstrated through extensive simulation studies across diverse scenarios. Compared with alternative approaches, including BGLSS (Bayesian Group Lasso with Spike-and-Slab priors), grLASSO (group Least Absolute Shrinkage and Selection Operator), grMCP (group Minimax Concave Penalty), and grSCAD (group Smoothly Clipped Absolute Deviation), our approach achieves a superior balance between goodness-of-fit and sparsity in most scenarios. We further illustrate its practical utility through real-data applications involving spectral analysis of sugar samples and weather measurements from Japan.

stat.ME↗

arXiv2025

Fast Bayesian Basis Selection for Functional Data Representation with Correlated Errors

Functional data analysis finds widespread application across various fields. While functional data are intrinsically infinite-dimensional, in practice, they are observed only at a finite set of points, typically over a dense grid. As a result, smoothing techniques are often used to approximate the observed data as functions. In this work, we propose a novel Bayesian approach for selecting basis functions for smoothing one or multiple curves simultaneously. Our method differentiates from other Bayesian approaches in two key ways: (i) by accounting for correlated errors and (ii) by developing a variational Expectation-Maximization (VEM) algorithm, which is faster than Markov chain Monte Carlo (MCMC) methods such as Gibbs sampling. Simulation studies demonstrate that our method effectively identifies the true underlying structure of the data across various scenarios, and it is applicable to different types of functional data. Our VEM algorithm not only recovers the basis coefficients and the correct set of basis functions but also estimates the existing within-curve correlation. When applied to the motorcycle, LIDAR (LIght Detection And Ranging) experiment and Canadian weather datasets, our method demonstrates comparable, and in some cases superior, performance in terms of adjusted R2 compared to regression splines, smoothing splines, least absolute shrinkage and selection operator (LASSO) and Bayesian LASSO. Our proposed method is implemented in R and codes are available at https://github.com/acarolcruz/VB-Bases-Selection

stat.ME↗

arXiv2025

Model-based Clustering of Multi-Dimensional Zero-Inflated Counts via the EM Algorithm

Zero-inflated count data arise in various fields, including health, biology, economics, and the social sciences. These data are often modelled using probabilistic distributions such as zero-inflated Poisson (ZIP), zero-inflated negative binomial (ZINB), or zero-inflated binomial (ZIB). To account for heterogeneity in the data, it is often useful to cluster observations into groups that may explain underlying differences in the data-generating process. This paper focuses on model-based clustering for zero-inflated counts when observations are structured in a matrix form rather than a vector. We propose a clustering framework based on mixtures of ZIP or ZINB distributions, with both the count and zero components depending on cluster assignments. Our approach incorporates covariates through a log-linear structure for the mean parameter and includes a size factor to adjust for differences in total sampling or exposure. Model parameters and cluster assignments are estimated via the Expectation-Maximization (EM) algorithm. We assess the performance of our proposed methodology through simulation studies evaluating clustering accuracy and estimator properties, followed by applications to publicly available datasets.

stat.ME↗

arXiv2025

BayesCPclust: A Bayesian Approach for Clustering Constant-Wise Change-Point Data

Change-point models deal with ordered data sequences. Their primary goal is to infer the locations where an aspect of the data sequence changes. In this paper, we propose and implement a nonparametric Bayesian model for clustering observations based on their constant-wise change-point profiles via Gibbs sampler. Our model incorporates a Dirichlet Process on the constant-wise change-point structures to cluster observations while simultaneously performing multiple change-point estimation. Additionally, our approach controls the number of clusters in the model, not requiring the specification of the number of clusters a priori. Satisfactory clustering and estimation results were obtained when evaluating our method under various simulated scenarios and on a real dataset from single-cell genomic sequencing. Our proposed methodology is implemented as an R package called BayesCPclust and is available from the Comprehensive R Archive Network at https://CRAN.R-project.org/package=BayesCPclust.

stat.ME↗

arXiv2024

Fast variational Bayesian inference for correlated survival data: an application to invasive mechanical ventilation duration analysis

Correlated survival data are prevalent in various clinical settings and have been extensively discussed in literature. One of the most common types of correlated survival data is clustered survival data, where the survival times from individuals in a cluster are associated. Our study is motivated by invasive mechanical ventilation data from different intensive care units (ICUs) in Ontario, Canada, forming multiple clusters. The survival times from patients within the same ICU cluster are correlated. To address this association, we introduce a shared frailty log-logistic accelerated failure time model that accounts for intra-cluster correlation through a cluster-specific random intercept. We present a novel, fast variational Bayes (VB) algorithm for parameter inference and evaluate its performance using simulation studies varying the number of clusters and their sizes. We further compare the performance of our proposed VB algorithm with the h-likelihood method and a Markov Chain Monte Carlo (MCMC) algorithm. The proposed algorithm delivers satisfactory results and demonstrates computational efficiency over the MCMC algorithm. We apply our method to the ICU ventilation data from Ontario to investigate the ICU site random effect on ventilation duration.

stat.ME↗

arXiv2023

Variational Bayesian analysis of survival data using a log-logistic accelerated failure time model

The log-logistic regression model is one of the most commonly used accelerated failure time (AFT) models in survival analysis, for which statistical inference methods are mainly established under the frequentist framework. Recently, Bayesian inference for log-logistic AFT models using Markov chain Monte Carlo (MCMC) techniques has also been widely developed. In this work, we develop an alternative approach to MCMC methods and infer the parameters of the log-logistic AFT model via a mean-field variational Bayes (VB) algorithm. A piecewise approximation technique is embedded in deriving the VB algorithm to achieve conjugacy. The proposed VB algorithm is evaluated and compared with typical frequentist inferences and MCMC inference using simulated data under various scenarios. A publicly available dataset is employed for illustration. We demonstrate that the proposed VB algorithm can achieve good estimation accuracy and has a lower computational cost compared with MCMC methods.

stat.ME↗

arXiv2022

J-PLUS: A catalogue of globular cluster candidates around the M81/M82/NGC3077 triplet of galaxies

Globular clusters (GCs) are proxies of the formation assemblies of their host galaxies. However, few studies exist targeting GC systems of spiral galaxies up to several effective radii. Through 12-band Javalambre Photometric Local Universe Survey (J-PLUS) imaging, we study the point sources around the M81/M82/NGC3077 triplet in search of new GC candidates. We develop a tailored classification scheme to search for GC candidates based on their similarity to known GCs via a principal components analysis (PCA) projection. Our method accounts for missing data and photometric errors. We report 642 new GC candidates in a region of 3.5 deg$^2$ around the triplet, ranked according to their Gaia astrometric proper motions when available. We find tantalising evidence for an overdensity of GC candidate sources forming a bridge connecting M81 and M82. Finally, the spatial distribution of the GC candidates $(g-i)$ colours is consistent with halo/intra-cluster GCs, i.e. it gets bluer as they get further from the closest galaxy in the field. We further employ a regression-tree based model to estimate the metallicity distribution of the GC candidates based on their J-PLUS bands. The metallicity distribution of the sample candidates is broad and displays a bump towards the metal-rich end. Our list increases the population of GC candidates around the triplet by 3-fold, stresses the usefulness of multi-band surveys in finding these objects, and provides a testbed for further studies analysing their spatial distribution around nearby (spirals) galaxies.

astro-ph.GA↗

arXiv2022

Bayesian Adaptive Selection of Basis Functions for Functional Data Representation

Considering the context of functional data analysis, we developed and applied a new Bayesian approach via Gibbs sampler to select basis functions for a finite representation of functional data. The proposed methodology uses Bernoulli latent variables to assign zero to some of the basis function coefficients with a positive probability. This procedure allows for an adaptive basis selection since it can determine the number of bases and which should be selected to represent functional data. Moreover, the proposed procedure measures the uncertainty of the selection process and can be applied to multiple curves simultaneously. The methodology developed can deal with observed curves that may differ due to experimental error and random individual differences between subjects, which one can observe in a real dataset application involving daily numbers of COVID-19 cases in Brazil. Simulation studies show the main properties of the proposed method, such as its accuracy in estimating the coefficients and the strength of the procedure to find the true set of basis functions. Despite having been developed in the context of functional data analysis, we also compared the proposed model via simulation with the well-established LASSO and Bayesian LASSO, which are methods developed for non-functional data.

stat.ME↗

arXiv2021

Parameter Estimation for Grouped Data Using EM and MCEM Algorithms

Nowadays, the confidentiality of data and information is of great importance for many companies and organizations. For this reason, they may prefer not to release exact data, but instead to grant researchers access to approximate data. For example, rather than providing the exact measurements of their clients, they may only provide researchers with grouped data, that is, the number of clients falling in each of a set of non-overlapping measurement intervals. The challenge is to estimate the mean and variance structure of the hidden ungrouped data based on the observed grouped data. To tackle this problem, this work considers the exact observed data likelihood and applies the Expectation-Maximization (EM) and Monte-Carlo EM (MCEM) algorithms for cases where the hidden data follow a univariate, bivariate, or multivariate normal distribution. Simulation studies are conducted to evaluate the performance of the proposed EM and MCEM algorithms. The well-known Galton data set is considered as an application example.

stat.ME↗

arXiv2021

Aggregated functional data model applied on clustering and disaggregation of UK electrical load profiles

Understanding electrical energy demand at the consumer level plays an important role in planning the distribution of electrical networks and offering of off-peak tariffs, but observing individual consumption patterns is still expensive. On the other hand, aggregated load curves are normally available at the substation level. The proposed methodology separates substation aggregated loads into estimated mean consumption curves, called typical curves, including information given by explanatory variables. In addition, a model-based clustering approach for substations is proposed based on the similarity of their consumers typical curves and covariance structures. The methodology is applied to a real substation load monitoring dataset from the United Kingdom and tested in eight simulated scenarios.

stat.AP↗

arXiv2017

Switching nonparametric regression models for multi-curve data

We develop and apply an approach for analyzing multi-curve data where each curve is driven by a latent state process. The state at any particular point determines a smooth function, forcing the individual curve to switch from one function to another. Thus each curve follows what we call a switching nonparametric regression model. We develop an EM algorithm to estimate the model parameters. We also obtain standard errors for the parameter estimates of the state process. We consider several types of state processes: independent and identically distributed, independent but depending on a covariate and Markov. Simulation studies show the frequentist properties of our estimates. We apply our methods to a data set of a building's power usage.

stat.ME↗

arXiv2014

Analysis of Aggregated Functional Data from Mixed Populations with Application to Energy Consumption

Understanding the energy consumption patterns of different types of consumers is essential in any planning of energy distribution. However, obtaining consumption information for single individuals is often either not possible or too expensive. Therefore, we consider data from aggregations of energy use, that is, from sums of individuals' energy use, where each individual falls into one of C consumer classes. Unfortunately, the exact number of individuals of each class may be unknown: consumers do not always report the appropriate class, due to various factors including differential energy rates for different consumer classes. We develop a methodology to estimate the expected energy use of each class as a function of time and the true number of consumers in each class. We also provide some measure of uncertainty of the resulting estimates. To accomplish this, we assume that the expected consumption is a function of time that can be well approximated by a linear combination of B-splines. Individual consumer perturbations from this baseline are modeled as B-splines with random coefficients. We treat the reported numbers of consumers in each category as random variables with distribution depending on the true number of consumers in each class and on the probabilities of a consumer in one class reporting as another class. We obtain maximum likelihood estimates of all parameters via a maximization algorithm. We introduce a special numerical trick for calculating the maximum likelihood estimates of the true number of consumers in each class. We apply our method to a data set and study our method via simulation.

stat.AP↗

arXiv2013

Switching Nonparametric Regression Models and the Motorcycle Data revisited

We propose a methodology to analyze data arising from a curve that, over its domain, switches among J states. We consider a sequence of response variables, where each response y depends on a covariate x according to an unobserved state z. The states form a stochastic process and their possible values are j=1,...,J. If z equals j the expected response of y is one of J unknown smooth functions evaluated at x. We call this model a switching nonparametric regression model. We develop an EM algorithm to estimate the parameters of the latent state process and the functions corresponding to the J states. We also obtain standard errors for the parameter estimates of the state process. We conduct simulation studies to analyze the frequentist properties of our estimates. We also apply the proposed methodology to the well-known motorcycle data set treating the data as coming from more than one simulated accident run with unobserved run labels.

stat.ME↗