A Theory-grounded Hybrid Neural Network Integrating Complementary Estimation Mechanisms for Stable Visual Object TrackingA

Abstract

Hybrid neural networks (HNNs) that integrate artificial neural networks (ANNs) with brain-inspired neural networks have achieved broad success across perception and control tasks. However, much of the current success is confined to neuron-scale hybridization, where discrete, spike-based coding fundamentally limits applicability to continuous-state estimation tasks. In neuroscience, continuous attractor neural networks (CANNs) represent continuous states through neural ensembles, pointing to a population-scale route for HNNs to address this limitation. Yet, principled methodologies for ANN-CANN integration remain largely underexplored. In this work, we propose a theory-grounded ANN-CANN hybridization framework and instantiate it as a hybrid tracking neural network (HTNN) for visual object tracking, a representative continuous-state estimation task. The framework aligns ANN response maps with CANN dynamics in the same state space, enabling the two heterogeneous branches to interact through the shared state representation. Furthermore, we uncover a functional bias-variance complementarity: data-driven ANNs provide asymptotically unbiased estimates, while CANN estimates are low-variance but temporally lagged. By operationalizing this complementarity, HTNN achieves stable and accurate tracking across nine visual tracking benchmarks, consistently outperforming single-network baselines and existing hybrid models. Notably, these performance gains are robustly maintained even under diverse environmental variations, including occlusion, motion blur, and background interference. Through this proof-of-concept study, our framework offers a generalizable foundation for advancing HNNs toward population-scale hybridization.