POSSE-kNN: Pathwise Out-of-Bag Selected Subspace Ensembles for Binary Classification

Abstract

Nearest neighbour classification is attractive for tabular data, but its performance can deteriorate when a fixed query centred neighbourhood does not follow the local class geometry. This study evaluates POSSE-$k$NN, a pathwise $k$ nearest neighbour ensemble that combines bootstrap sampling, random feature subspaces, out-of-bag (OOB) screening, and selective voting. Within each randomized candidate, pathwise selection first chooses the training observation nearest to the query and then chooses each subsequent neighbour relative to the observation accepted at the preceding step. After the candidate is fully specified, its OOB error is computed. Five hundred candidates are generated, ranked by OOB error, and the best 25% are retained. The method is evaluated on ten binary benchmark datasets using repeated 70/30 train/test partitions and six established comparators. Across the dataset level means, POSSE-kNN attains an accuracy of 0.740, Cohen's kappa of 0.412, and a Brier score of 0.175, giving the best aggregate result for all three criteria. It has the highest unrounded mean accuracy and kappa on eight datasets; RkNN and SVM lead on the other two. A neighbourhood size analysis on three datasets shows stable behaviour for k={3, 5, 7} when the path geometry is informative and identifies a case in which alternative neighbourhood rules are preferable.