arXiv subjects

Jean-Michel Poggi

Publications and source records attributed to Jean-Michel Poggi.

10 recordsLinked to original sources

arXiv2026

Conformal Prediction for Hierarchical Data

We consider conformal prediction for multivariate data and focus on hierarchical data, where some components are linear combinations of others. Intuitively, the hierarchical structure can be leveraged to reduce the size of prediction regions for the same coverage level. We implement this intuition by including a projection step (also called a reconciliation step) in the split conformal prediction [SCP] procedure, and prove that the resulting prediction regions are indeed globally smaller. We do so both under the classic objective of joint coverage and under a new and challenging task: component-wise coverage, for which efficiency results are more difficult to obtain. The associated strategies and their analyses are based both on the literature of SCP and of forecast reconciliation, which we connect. We also illustrate the theoretical findings, for different scales of hierarchies on simulated data.

stat.ML↗

arXiv2026

Geographically Weighted Regression for Air Quality Low-Cost Sensor Calibration

This article focuses on the use of Geographically Weighted Regression (GWR) method to correct air quality low-cost sensors measurements. Those sensors are of major interest in the current era of high-resolution air quality monitoring at urban scale, but require calibration using reference analyzers. The results for NO2 are provided along with comments on the estimated GWR model and the spatial content of the estimated coefficients. The study has been carried out using the publicly available SensEURCity dataset in Antwerp, which is especially relevant since it includes 9 reference stations and 34 low-cost sensors collocated and deployed within the city.

stat.AP↗

arXiv2025

Debiasing physico-chemical models in air quality monitoring by combining different pollutant concentration measures

Air quality monitoring requires to produce accurate estimation of nitrogen dioxide or fine particulate matter concentration maps, at different moments. A typical strategy is to combine different types of data. On the one hand, concentration maps produced by deterministic physicochemical models at urban scale, and on the other hand, concentration measures made at different points, different moments, and by different devices. These measures are provided first by a small number of reference stations, which give reliable measurements of the concentration, and second by a larger number of micro-sensors, which give biased and noisier measurements. The proposed approach consists in modeling the bias of the physicochemical model and estimating the parameters of this bias using all the available concentration measures. Our model relies on a partition of the geographical space of interest into different zones within which the bias is assumed to be modeled by a single affine transformation of the actual concentration. Our approach allows to improve the concentration maps provided by the deterministic models but also to understand the behavior of micro-sensors and their contribution in improving air quality monitoring. We introduce the model, detail its implementation and experiment it through numerical results using datasets collected in Grenoble (France).

stat.AP↗

arXiv2017

Random Forests for Big Data

Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like linear regression models, clustering methods and bootstrapping schemes. Based on decision trees combined with aggregation and bootstrap ideas, random forests were introduced by Breiman in 2001. They are a powerful nonparametric statistical method allowing to consider in a single and versatile framework regression problems, as well as two-class and multi-class classification problems. Focusing on classification problems, this paper proposes a selective review of available proposals that deal with scaling random forests to Big Data problems. These proposals rely on parallel environments or on online adaptations of random forests. We also describe how related quantities -- such as out-of-bag error and variable importance -- are addressed in these methods. Then, we formulate various remarks for random forests in the Big Data context. Finally, we experiment five variants on two massive datasets (15 and 120 millions of observations), a simulated one as well as real world data. One variant relies on subsampling while three others are related to parallel implementations of random forests and involve either various adaptations of bootstrap to Big Data or to "divide-and-conquer" approaches. The fifth variant relates on online learning of random forests. These numerical experiments lead to highlight the relative performance of the different variants, as well as some of their limitations.

stat.ML↗

arXiv2017

Arbres CART et Forêts aléatoires, Importance et sélection de variables

Two algorithms proposed by Leo Breiman : CART trees (Classification And Regression Trees for) introduced in the first half of the 80s and random forests emerged, meanwhile, in the early 2000s, are the subject of this article. The goal is to provide each of the topics, a presentation, a theoretical guarantee, an example and some variants and extensions. After a preamble, introduction recalls objectives of classification and regression problems before retracing some predecessors of the Random Forests. Then, a section is devoted to CART trees then random forests are presented. Then, a variable selection procedure based on permutation variable importance is proposed. Finally the adaptation of random forests to the Big Data context is sketched.

stat.ME↗

arXiv2015

Clustering electricity consumers using high-dimensional regression mixture models

Massive informations about individual (household, small and medium enterprise) consumption are now provided with new metering technologies and the smart grid. Two major exploitations of these data are load profiling and forecasting at different scales on the grid. Customer segmentation based on load classification is a natural approach for these purposes. We propose here a new methodology based on mixture of high-dimensional regression models. The novelty of our approach is that we focus on uncovering classes or clusters corresponding to different regression models. As a consequence, these classes could then be exploited for profiling as well as forecasting in each class or for bottom-up forecasts in a unified view. We consider a real dataset of Irish individual consumers of 4,225 meters, each with 48 half-hourly meter reads per day over 1 year: from 1st January 2010 up to 31st December 2010, to demonstrate the feasibility of our approach.

stat.AP↗

arXiv2014

A prediction interval for a function-valued forecast model

Starting from the information contained in the shape of the load curves, we have proposed a flexible nonparametric function-valued fore-cast model called KWF (Kernel+Wavelet+Functional) well suited to handle nonstationary series. The predictor can be seen as a weighted average of futures of past situations, where the weights increase with the similarity between the past situations and the actual one. In addi-tion, this strategy provides with a simultaneous multiple horizon pre-diction. These weights induce a probability distribution that can be used to produce bootstrap pseudo predictions. Prediction intervals are constructed after obtaining the corresponding bootstrap pseudo pre-diction residuals. We develop two propositions following directly the KWF strategy and compare it to two alternative ways coming from proposals of econometricians. They construct simultaneous prediction intervals using multiple comparison corrections through the control of the family wise error (FWE) or the false discovery rate. Alternatively, such prediction intervals can be constructed bootstrapping joint prob-ability regions. In this work we propose to obtain prediction intervals for the KWF model that are simultaneously valid for the H predic-tion horizons that corresponds with the corresponding path forecast, making a connection between functional time series and the econome-tricians' framework.

stat.ME↗

arXiv2011

Clustering functional data using wavelets

We present two methods for detecting patterns and clusters in high dimensional time-dependent functional data. Our methods are based on wavelet-based similarity measures, since wavelets are well suited for identifying highly discriminant local time and scale features. The multiresolution aspect of the wavelet transform provides a time-scale decomposition of the signals allowing to visualize and to cluster the functional data into homogeneous groups. For each input function, through its empirical orthogonal wavelet transform the first method uses the distribution of energy across scales generate a handy number of features that can be sufficient to still make the signals well distinguishable. Our new similarity measure combined with an efficient feature selection technique in the wavelet domain is then used within more or less classical clustering algorithms to effectively differentiate among high dimensional populations. The second method uses dissimilarity measures between the whole time-scale representations and are based on wavelet-coherence tools. The clustering is then performed using a k-centroid algorithm starting from these dissimilarities. Practical performance of these methods that jointly designs both the feature selection in the wavelet domain and the classification distance is demonstrated through simulations as well as daily profiles of the French electricity power demand.

stat.ME↗

arXiv2010

Classification supervisée en grande dimension. Application à l'agrément de conduite automobile

This work is motivated by a real work problem: objectivization. It consists in explaining the subjective drivability using physical criteria coming from signals measured during experiments. We suggest an approach for the discriminant variables selection trying to take advantage of the functional nature of the data. The porblem is ill-posed, since the number of explanatory variables is hugely greater than the sample size. The strategy proceeds in three steps: a signal preprocessing including wavelet denoising and synchronization, dimensionality reduction by compression using a common wavelet basis, and finally the selection of useful variables using a stepwise strategy involving successive applications of the CART method.

math.ST↗

arXiv2008

Random Forests: some methodological insights

This paper examines from an experimental perspective random forests, the increasingly used statistical method for classification and regression problems introduced by Leo Breiman in 2001. It first aims at confirming, known but sparse, advice for using random forests and at proposing some complementary remarks for both standard problems as well as high dimensional ones for which the number of variables hugely exceeds the sample size. But the main contribution of this paper is twofold: to provide some insights about the behavior of the variable importance index based on random forests and in addition, to propose to investigate two classical issues of variable selection. The first one is to find important variables for interpretation and the second one is more restrictive and try to design a good prediction model. The strategy involves a ranking of explanatory variables using the random forests score of importance and a stepwise ascending variable introduction strategy.

stat.ML↗