A Benchmark Study of Classical and Dual Polynomial Regression (DPR)-Based Probability Density Estimation Technique

Abstract

The probability density function (PDF) plays a central role in statistical and machine learning modeling. Real-world data often deviates from Gaussian assumptions, exhibiting skewness and exponential decay. To evaluate how well different density estimation methods capture such irregularities, we generated six unimodal datasets from diverse distributions that reflect real-world anomalies. These were compared using parametric methods (Pearson Type I and Normal distribution) as well as non-parametric approaches, including histograms, kernel density estimation (KDE), and our proposed method. To accelerate computation, we implemented GPU-based versions of KDE (tKDE) and histogram estimation (tHDE) in TensorFlow, both of which outperform Python SciPy's KDE. Prior work demonstrated the use of piecewise modeling for density estimation, such as local polynomial regression; however, these methods are computationally intensive. Based on the concept of piecewise modeling, we developed a computationally efficient model, the Dual Polynomial Regression (DPR) method, which leverages tKDE or tHDE for training. DPR employs the piecewise strategy to split the PDF at its mode and fit polynomial regressions to the left and right halves independently, enabling better capture of the asymmetric shape of the unimodal distribution. We used the Mean Squared Error (MSE), Jensen-Shannon Divergence (JSD), and Pearson's correlation coefficient, with reference to the baseline PDF, to validate accuracy. We verified normalization using Area Under the Curve (AUC) and computational overhead via execution time. Validation on real-world systolic and diastolic data from 300,000 unique patients shows that the DPR of order 4, trained with tKDE, offers the best balance between accuracy and computational overhead.