arXiv subjects

Shojaeddin Chenouri

Publications and source records attributed to Shojaeddin Chenouri.

12 recordsLinked to original sources

arXiv2025

An Efficient Continual Learning Framework for Multivariate Time Series Prediction Tasks with Application to Vehicle State Estimation

In continual time series analysis using neural networks, catastrophic forgetting (CF) of previously learned models when training on new data domains has always been a significant challenge. This problem is especially challenging in vehicle estimation and control, where new information is sequentially introduced to the model. Unfortunately, existing work on continual learning has not sufficiently addressed the adverse effects of catastrophic forgetting in time series analysis, particularly in multivariate output environments. In this paper, we present EM-ReSeleCT (Efficient Multivariate Representative Selection for Continual Learning in Time Series Tasks), an enhanced approach designed to handle continual learning in multivariate environments. Our approach strategically selects representative subsets from old and historical data and incorporates memory-based continual learning techniques with an improved optimization algorithm to adapt the pre-trained model on new information while preserving previously acquired information. Additionally, we develop a sequence-to-sequence transformer model (autoregressive model) specifically designed for vehicle state estimation. Moreover, we propose an uncertainty quantification framework using conformal prediction to assess the sensitivity of the memory size and to showcase the robustness of the proposed method. Experimental results from tests on an electric Equinox vehicle highlight the superiority of our method in continually learning new information while retaining prior knowledge, outperforming state-of-the-art continual learning methods. Furthermore, EM-ReSeleCT significantly reduces training time, a critical advantage in continual learning applications.

cs.LG↗

arXiv2025

Graph-based Change Point Detection for Functional Data

Modeling functions that are sequentially observed as functional time series is becoming increasingly common. In such models, it is often crucial to ensure data homogeneity. We investigate the sensitivity of graph-based change point detection for changes in the distribution of functional data that demarcate homogeneous regions. Related test statistics and thresholds for detection are given. A key factor in the efficacy of such tests is the graph construction. Practical considerations for constructing a graph on arbitrary data are explored. Simulation experiments investigate tuning parameters for graph construction and evaluate the graph-based methods in comparison to existing functional methods. In addition to sensitivity of lower and higher order changes, robustness to the tuning parameter choices, and practical recommendations, are shown. Applications to multi-year pedestrian counts, high-frequency asset returns, and continuous electricity prices corroborate the simulation results.

stat.ME↗

arXiv2025

A Comprehensive Framework for Statistical Inference in Measurement System Assessment Studies

Measurement system analysis aims to quantify the variability in data attributable to the measurement system and evaluate its contribution to overall data variability. This paper conducts a rigorous theoretical investigation of the statistical methods used in such analyses, focusing on variance components and other critical parameters. While established techniques exist for single-variable cases, a systematic theoretical exploration of their properties has been largely overlooked. This study addresses this gap by examining estimators for variance components and other key parameters in measurement system assessment, analyzing their statistical properties, and providing new insights into their reliability, performance, and applicability.

stat.AP↗

arXiv2023

On Monitoring High-Dimensional Processes with Individual Observations

Modern data collecting methods and computation tools have made it possible to monitor high-dimensional processes. In this article, Phase II monitoring of high-dimensional processes is investigated when the available number of samples collected in Phase I is limitted in comparison to the number of variables. A new charting statistic for high-dimensional multivariate processes based on the diagonal elements of the underlying covariance matrix is introduced and a unified procedure for Phase I and II by employing a self-starting control chart is proposed. To remedy the effect of outliers, we adopt a robust procedure for parameter estimation in Phase I and introduce the appropriate consistent estimators. The statistical performance of the proposed method is evaluated in Phase II through average run length (ARL) criterion in the absence and presence of outliers and reveals that the proposed control chart scheme effectively detects various kinds of shifts in the process mean. Finally, we illustrate the applicability of our proposed method via a real-world example.

stat.ME↗

arXiv2022

Phase I Analysis of High-Dimensional Processes in the Presence of Outliers

One of the significant challenges in monitoring the quality of products today is the high dimensionality of quality characteristics. In this paper, we address Phase I analysis of high-dimensional processes with individual observations when the available number of samples collected over time is limited. Using a new charting statistic, we propose a robust procedure for parameter estimation in Phase I. This robust procedure is efficient in parameter estimation in the presence of outliers or contamination in the data. A consistent estimator is proposed for parameter estimation and a finite sample correction coefficient is derived and evaluated through simulation. We assess the statistical performance of the proposed method in Phase I in terms of the probability of signal criterion. This assessment is carried out in the absence and presence of outliers. We show that, in both phases, the proposed control chart scheme effectively detects various kinds of shifts in the process mean. Besides, we present two real-world examples to illustrate the applicability of our proposed method.

stat.ME↗

arXiv2021

Functional Boxplots for Outlier Detection in Additive Manufacturing

Additive manufacturing (AM), also known as 3D printing, is one of the most promising digital manufacturing technologies, thanks to its potential to produce highly complex geometries rapidly. AM has been promoted from a prototyping methodology to a serial production platform for which precise process monitoring and control strategies to guarantee the accuracy of products are required. This need has motivated practitioners to focus on designing process monitoring tools to improve the accuracy of produced geometries. In line with the emerging interest, in the current investigation, a novel strategy is proposed which uses functional representation of in-plane contours to come up with statistical boxplots with the goal of detecting outlying AM products. The method can be used for process monitoring during AM production to automatically detect defective products in an online fashion. To ensure the considered method has an acceptable potential, different complex 3D geometries are considered and undergo different types of stochastic perturbations to collect data for outlier detection. The results of the conducted simulation are very promising and reveal the reliability of the proposed method for detecting products with statistically significant deformation.

stat.AP↗

arXiv2021

Detection of Change Points in Piecewise Polynomial Signals Using Trend Filtering

While many approaches have been proposed for discovering abrupt changes in piecewise constant signals, few methods are available to capture these changes in piecewise polynomial signals. In this paper, we propose a change point detection method, PRUTF, based on trend filtering. By providing a comprehensive dual solution path for trend filtering, PRUTF allows us to discover change points of the underlying signal for either a given value of the regularization parameter or a specific number of steps of the algorithm. We demonstrate that the dual solution path constitutes a Gaussian bridge process that enables us to derive an exact and efficient stopping rule for terminating the search algorithm. We also prove that the estimates produced by this algorithm are asymptotically consistent in pattern recovery. This result holds even in the case of staircases (consecutive change points of the same sign) in the signal. Finally, we investigate the performance of our proposed method for various signals and then compare its performance against some state-of-the-art methods in the context of change point detection. We apply our method to three real-world datasets including the UK House Price Index (HPI), the GISS surface Temperature Analysis (GISTEMP) and the Coronavirus disease (COVID-19) pandemic.

stat.ME↗

arXiv2021

Valid Post-Detection Inference for Change Points Identified Using Trend Filtering

There are many research works and methods about change point detection in the literature. However, there are only a few that provide inference for such change points after being estimated. This work mainly focuses on a statistical analysis of change points estimated by the PRUTF algorithm, which incorporates trend filtering to determine change points in piecewise polynomial signals. This paper develops a methodology to perform statistical inference, such as computing p-values and constructing confidence intervals in the newly developed post-selection inference framework. Our work concerns both cases of known and unknown error variance. As pointed out in the post-selection inference literature, the length of such confidence intervals are undesirably long. To resolve this shortcoming, we also provide two novel strategies, global post-detection, and local post-detection which are based on the intrinsic properties of change points. We run our proposed methods on real as well as simulated data to evaluate their performances.

stat.ME↗

arXiv2021

A non-alternating graph hashing algorithm for large scale image search

In the era of big data, methods for improving memory and computational efficiency have become crucial for successful deployment of technologies. Hashing is one of the most effective approaches to deal with computational limitations that come with big data. One natural way for formulating this problem is spectral hashing that directly incorporates affinity to learn binary codes. However, due to binary constraints, the optimization becomes intractable. To mitigate this challenge, different relaxation approaches have been proposed to reduce the computational load of obtaining binary codes and still attain a good solution. The problem with all existing relaxation methods is resorting to one or more additional auxiliary variables to attain high quality binary codes while relaxing the problem. The existence of auxiliary variables leads to coordinate descent approach which increases the computational complexity. We argue that introducing these variables is unnecessary. To this end, we propose a novel relaxed formulation for spectral hashing that adds no additional variables to the problem. Furthermore, instead of solving the problem in original space where number of variables is equal to the data points, we solve the problem in a much smaller space and retrieve the binary codes from this solution. This trick reduces both the memory and computational complexity at the same time. We apply two optimization techniques, namely projected gradient and optimization on manifold, to obtain the solution. Using comprehensive experiments on four public datasets, we show that the proposed efficient spectral hashing (ESH) algorithm achieves highly competitive retrieval performance compared with state of the art at low complexity.

cs.CV↗

arXiv2021

Robust nonparametric hypothesis tests for differences in the covariance structure of functional data

We develop a group of robust, nonparametric hypothesis tests which detect differences between the covariance operators of several populations of functional data. These tests, called FKWC tests, are based on functional data depth ranks. These tests work well even when the data is heavy tailed, which is shown both in simulation and theoretically. These tests offer several other benefits, they have a simple distribution under the null hypothesis, they are computationally cheap and they possess transformation invariance properties. We show that under general alternative hypotheses these tests are consistent under mild, nonparametric assumptions. As a result of this work, we introduce a new functional depth function called L2-root depth which works well for the purposes of detecting differences in magnitude between covariance kernels. We present an analysis of the FKWC test using L2-root depth under local alternatives. In simulation, when the true covariance kernels have strictly positive eigenvalues, we show that these tests have higher power than their competitors, while still maintaining their nominal size. We also provide a methods for computing sample size and performing multiple comparisons.

stat.ME↗

arXiv2017

A New Method for Performance Analysis in Nonlinear Dimensionality Reduction

In this paper, we develop a local rank correlation measure which quantifies the performance of dimension reduction methods. The local rank correlation is easily interpretable, and robust against the extreme skewness of nearest neighbor distributions in high dimensions. Some benchmark datasets are studied. We find that the local rank correlation closely corresponds to our visual interpretation of the quality of the output. In addition, we demonstrate that the local rank correlation is useful in estimating the intrinsic dimensionality of the original data, and in selecting a suitable value of tuning parameters used in some algorithms.

stat.ME↗

arXiv2017

Optimal estimation in functional linear regression for sparse noise-contaminated data

In this paper, we propose a novel approach to fit a functional linear regression in which both the response and the predictor are functions of a common variable such as time. We consider the case that the response and the predictor processes are both sparsely sampled on random time points and are contaminated with random errors. In addition, the random times are allowed to be different for the measurements of the predictor and the response functions. The aforementioned situation often occurs in the longitudinal data settings. To estimate the covariance and the cross-covariance functions we use a regularization method over a reproducing kernel Hilbert space. The estimate of the cross-covarinace function is used to obtain an estimate of the regression coefficient function and also functional singular components. We derive the convergence rates of the proposed cross-covariance, the regression coefficient and the singular component function estimators. Furthermore, we show that, under some regularity conditions, the estimator of the coefficient function has a minimax optimal rate. We conduct a simulation study and demonstrate merits of the proposed method by comparing it to some other existing methods in the literature. We illustrate the method by an example of an application to a well known multicenter AIDS Cohort Study.

stat.ME↗