arXiv subjects

Mikey Ferguson

Publications and source records attributed to Mikey Ferguson.

2 recordsLinked to original sources

arXiv2026

Optimal Combinatorial Testing with Constraints: The Balancing Act

Imagine that you are in front of a cockpit with several on-off buttons. If you were to thoroughly test it, you would need to try a prohibitive number of configurations. But since most bugs in practice can be isolated to interactions among few components, having tests that cover every possible pairwise configuration is a good start. However, this is a problem that goes from easy to NP-hard as soon as some pairwise configurations are forbidden. In this paper, we revisit unconstrained combinatorial testing with pairwise coverage on binary parameters and contrast it with the constrained case, showing and conjecturing properties that either are upheld or change from one to the other. In particular, we discuss the extent to which it remains a good idea -- and sometimes indeed optimal -- to have every button almost as many times on as off to minimize testing. We propose the first exact algorithm based on integer programming and a faster heuristic that often produces optimal solutions, both outperforming or competitive with their baselines.

math.OC↗

arXiv2022

Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm

Pruning techniques have been successfully used in neural networks to trade accuracy for sparsity. However, the impact of network pruning is not uniform: prior work has shown that the recall for underrepresented classes in a dataset may be more negatively affected. In this work, we study such relative distortions in recall by hypothesizing an intensification effect that is inherent to the model. Namely, that pruning makes recall relatively worse for a class with recall below accuracy and, conversely, that it makes recall relatively better for a class with recall above accuracy. In addition, we propose a new pruning algorithm aimed at attenuating such effect. Through statistical analysis, we have observed that intensification is less severe with our algorithm but nevertheless more pronounced with relatively more difficult tasks, less complex models, and higher pruning ratios. More surprisingly, we conversely observe a de-intensification effect with lower pruning ratios, which indicates that moderate pruning may have a corrective effect to such distortions.

cs.LG↗