arXiv subjects

Munir Hiabu

Publications and source records attributed to Munir Hiabu.

13 recordsLinked to original sources

arXiv2024

Fast Estimation of Partial Dependence Functions using Trees

Many existing interpretation methods are based on Partial Dependence (PD) functions that, for a pre-trained machine learning model, capture how a subset of the features affects the predictions by averaging over the remaining features. Notable methods include Shapley additive explanations (SHAP) which computes feature contributions based on a game theoretical interpretation and PD plots (i.e., 1-dim PD functions) that capture average marginal main effects. Recent work has connected these approaches using a functional decomposition and argues that SHAP values can be misleading since they merge main and interaction effects into a single local effect. However, a major advantage of SHAP compared to other PD-based interpretations has been the availability of fast estimation techniques, such as \texttt{TreeSHAP}. In this paper, we propose a new tree-based estimator, \texttt{FastPD}, which efficiently estimates arbitrary PD functions. We show that \texttt{FastPD} consistently estimates the desired population quantity -- in contrast to path-dependent \texttt{TreeSHAP} which is inconsistent when features are correlated. For moderately deep trees, \texttt{FastPD} improves the complexity of existing methods from quadratic to linear in the number of observations. By estimating PD functions for arbitrary feature subsets, \texttt{FastPD} can be used to extract PD-based interpretations such as SHAP, PD plots and higher-order interaction effects.

cs.LG↗

arXiv2024

Pure interaction effects unseen by Random Forests

Random Forests are widely claimed to capture interactions well. However, some simple examples suggest that they perform poorly in the presence of certain pure interactions that the conventional CART criterion struggles to capture during tree construction. Motivated from this, it is argued that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions. In a simulation study these variants are compared to conventional Random Forests and Extremely Randomized Trees. The results validate that the modifications considered enhance the model's fitting ability in scenarios where pure interactions play a crucial role. Finally, the methods are applied to real datasets.

stat.ML↗

arXiv2024

Consistency of Random Forest Type Algorithms under a Probabilistic Impurity Decrease Condition

This paper derives a unifying theorem establishing consistency results for a broad class of tree-based algorithms. It improves current results in two aspects. First of all, it can be applied to algorithms that vary from traditional Random Forests due to additional randomness for choosing splits, extending split options, allowing partitions into more than two cells in a single iteration step, and combinations of those. In particular, we prove consistency for Extremely Randomized Trees, Interaction Forests and Oblique Regression Trees using our general theorem. Secondly, it can be used to demonstrate consistency for a larger function class compared to previous results on Random Forests if one allows for additional random splits. Our results are based on the extension of the recently introduced notion of sufficient impurity decrease to a probabilistic sufficient impurity decrease condition.

math.ST↗

arXiv2023

A machine learning approach based on survival analysis for IBNR frequencies in non-life reserving

We introduce new approaches for forecasting IBNR (Incurred But Not Reported) frequencies by leveraging individual claims data, which includes accident date, reporting delay, and possibly additional features for every reported claim. A key element of our proposal involves computing development factors, which may be influenced by both the accident date and other features. These development factors serve as the basis for predictions. While we assume close to continuous observations of accident date and reporting delay, the development factors can be expressed at any level of granularity, such as months, quarters, or year and predictions across different granularity levels exhibit coherence. The calculation of development factors relies on the estimation of a hazard function in reverse development time, and we present three distinct methods for estimating this function: the Cox proportional hazard model, a feed-forward neural network, and eXtreme gradient boosting. In all three cases, estimation is based on the same partial likelihood that accommodates left truncation and ties in the data. While the first case is a semi-parametric model that assumes in parts a log linear structure, the two machine learning approaches only assume that the baseline and the other factors are multiplicatively separable. Through an extensive simulation study and real-world data application, our approach demonstrates promising results.

stat.ME↗

arXiv2023

Identifiability and estimation of the competing risks model under exclusion restrictions

The non-identifiability of the competing risks model requires researchers to work with restrictions on the model to obtain informative results. We present a new identifiability solution based on an exclusion restriction. Many areas of applied research use methods that rely on exclusion restrcitions. It appears natural to also use them for the identifiability of competing risks models. By imposing the exclusion restriction couple with an Archimedean copula, we are able to avoid any parametric restriction on the marginal distributions. We introduce a semiparametric estimation approach for the nonparametric marginals and the parametric copula. Our simulation results demonstrate the usefulness of the suggested model, as the degree of risk dependence can be estimated without parametric restrictions on the marginal distributions.

stat.ME↗

arXiv2023

Random Planted Forest: a directly interpretable tree ensemble

We introduce a novel interpretable tree based algorithm for prediction in a regression setting. Our motivation is to estimate the unknown regression function from a functional decomposition perspective in which the functional components correspond to lower order interaction terms. The idea is to modify the random forest algorithm by keeping certain leaves after they are split instead of deleting them. This leads to non-binary trees which we refer to as planted trees. An extension to a forest leads to our random planted forest algorithm. Additionally, the maximum number of covariates which can interact within a leaf can be bounded. If we set this interaction bound to one, the resulting estimator is a sum of one-dimensional functions. In the other extreme case, if we do not set a limit, the resulting estimator and corresponding model place no restrictions on the form of the regression function. In a simulation study we find encouraging prediction and visualisation properties of our random planted forest method. We also develop theory for an idealized version of random planted forests in cases where the interaction bound is low. We show that if it is smaller than three, the idealized version achieves asymptotically optimal convergence rates up to a logarithmic factor. Code is available on GitHub https://github.com/PlantedML/randomPlantedForest.

stat.ML↗

arXiv2023

Unifying local and global model explanations by functional decomposition of low dimensional structures

We consider a global representation of a regression or classification function by decomposing it into the sum of main and interaction components of arbitrary order. We propose a new identification constraint that allows for the extraction of interventional SHAP values and partial dependence plots, thereby unifying local and global explanations. With our proposed identification, a feature's partial dependence plot corresponds to the main effect term plus the intercept. The interventional SHAP value of feature $k$ is a weighted sum of the main component and all interaction components that include $k$, with the weights given by the reciprocal of the component's dimension. This brings a new perspective to local explanations such as SHAP values which were previously motivated by game theory only. We show that the decomposition can be used to reduce direct and indirect bias by removing all components that include a protected feature. Lastly, we motivate a new measure of feature importance. In principle, our proposed functional decomposition can be applied to any machine learning model, but exact calculation is only feasible for low-dimensional structures or ensembles of those. We provide an algorithm and efficient implementation for gradient-boosted trees (xgboost) and random planted forest. Conducted experiments suggest that our method provides meaningful explanations and reveals interactions of higher orders. The proposed methods are implemented in an R package, available at \url{https://github.com/PlantedML/glex}.

cs.LG↗

arXiv2023

Smooth Backfitting for Additive Hazard Rates

Smooth backfitting was first introduced in an additive regression setting via a direct projection alternative to the classic backfitting method by Buja, Hastie and Tibshirani. This paper translates the original smooth backfitting concept to a survival model considering an additively structured hazard. The model allows for censoring and truncation patterns occurring in many applications such as medical studies or actuarial reserving. Our estimators are shown to be a projection of the data into the space of multivariate hazard functions with smooth additive components. Hence, our hazard estimator is the closest nonparametric additive fit even if the actual hazard rate is not additive. This is different to other additive structure estimators where it is not clear what is being estimated if the model is not true. We provide full asymptotic theory for our estimators. We propose an implementation of estimators that show good performance in practice.

math.ST↗

arXiv2023

Replicating and extending chain-ladder via an age-period-cohort structure on the claim development in a run-off triangle

This paper introduces yet another stochastic model replicating chain-ladder estimates and furthermore considers extensions that add flexibility to the modeling. In its simplest form, the proposed model replicates the chain-ladder's development factors using a GLM model with averaged hazard rates running in reversed development time as response. This is in contrast to the existing reserving literature within the GLM framework where claim amounts are modeled as response. Modeling the averaged hazard rate corresponds to modeling the claim development and is arguably closer to the actual chain-ladder algorithm. Furthermore, since exposure does not need to be modeled, the model only has half the number of parameters compared to when modeling the claim amounts. This lesser complexity can be used to easily introduce model extensions that may better fit the data. We provide a new R-package, $\texttt{clmplus}$, where the models are implemented and can be fed with run-off triangles. We conduct an empirical study on 30 publicly available run-off triangles making a case for the benefit of having $\texttt{clmplus}$ in the actuary's toolbox.

stat.AP↗

arXiv2022

Local linear smoothing in additive models as data projection

We discuss local linear smooth backfitting for additive non-parametric models. This procedure is well known for achieving optimal convergence rates under appropriate smoothness conditions. In particular, it allows for the estimation of each component of an additive model with the same asymptotic accuracy as if the other components were known. The asymptotic discussion of local linear smooth backfitting is rather complex because typically an overwhelming notation is required for a detailed discussion. In this paper we interpret the local linear smooth backfitting estimator as a projection of the data onto a linear space with a suitably chosen semi-norm. This approach simplifies both the mathematical discussion as well as the intuitive understanding of properties of this version of smooth backfitting.

math.ST↗

arXiv2020

Smooth backfitting of proportional hazards with multiplicative components

Smooth backfitting has proven to have a number of theoretical and practical advantages in structured regression. Smooth backfitting projects the data down onto the structured space of interest providing a direct link between data and estimator. This paper introduces the ideas of smooth backfitting to survival analysis in a proportional hazard model, where we assume an underlying conditional hazard with multiplicative components. We develop asymptotic theory for the estimator and we use the smooth backfitter in a practical application, where we extend recent advances of in-sample forecasting methodology by allowing more information to be incorporated, while still obeying the structured requirements of in-sample forecasting.

math.ST↗

arXiv2020

Continuous chain-ladder with paid data

We introduce a continuous-time framework for the prediction of outstanding liabilities, in which chain-ladder development factors arise as a histogram estimator of a cost-weighted hazard function running in reversed development time. We use this formulation to show that under our assumptions on the individual data chain-ladder is consistent. Consistency is understood in the sense that both the number of observed claims grows to infinity and the level of aggregation tends to zero. We propose alternatives to chain-ladder development factors by replacing the histogram estimator with kernel smoothers and by estimating a cost-weighted density instead of a cost-weighted hazard. Finally, we provide a real-data example and a simulation study confirming the strengths of the proposed alternatives.

stat.AP↗

arXiv2019

Non-Smooth Backfitting for Excess Risk Additive Regression Model with Two Survival Time-Scales

We present a new backfitting algorithm estimating the complex structured non-parametric survival model of Scheike (2001) without having to use smoothing. The considered model is a non-parametric survival model with two time-scales that are equivalent up to a constant that varies over the subjects. Covariate effects are modelled linearly on each time scale by additive Aalen models. Estimators of the cumulative intensities on the two time-scales are suggested by solving local estimating equations jointly on the two time-scales. We are able to estimate the cumulative intensities solving backfitting estimating equations without using smoothing methods and we provide large sample properties and simultaneous confidence bands. The model is applied to data on myocardial infarction providing a separation of the two effects stemming from time since diagnosis and age.

stat.ME↗