Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance
This paper studies the finite-sample bias of plug-in estimators for joint entropy, conditional entropy, and mutual information for finitely supported discrete random variables. Using the univariate reduction framework developed in our companion works, we derive explicit first-order bias formulas. For a pair (X,Y) with support sizes r and s, the biases are respectively -(rs-1)/(2n), -r(s-1)/(2n), and (r-1)(s-1)/(2n). We then propose bias-corrected estimators for the three measures and prove that the uncorrected and corrected versions are asymptotically equivalent, differing by a deterministic term of order O(1/n); consequently, they share the same asymptotic distribution. A simulation study validates the theoretical results, including a dedicated independence study and the relationship with Wilks' theorem. An application to a motor insurance portfolio, where X is the driver's age class and Y the claim severity class, illustrates the improvement brought by the bias correction for risk classification.