Generalizability of Artificial Neural Network Models in Ecological Applications: Predicting Nest Occurrence and Breeding Success of the Red-winged Blackbird Agelaius phoeniceus

Abstract

Separate artificial neural network (ANN) models were developed from data in two geographical regions and years apart for a marsh-nesting bird, the red-winged blackbird Agelaius phoeniceus. Each model was independently tested on the spatially and temporally distinct data from the other region to determine how generalizable it was. The first model was developed to predict occurrence of nests in two wetlands on Lake Erie, Ohio in 1995 and 1996. The second model was developed to predict breeding success in two marshes in Connecticut, USA in 1969 and 1970. Independent variables were vegetation durability, stem density, stem/nest height, distance to open water, distance to edge, and water depth. With input variable relevances, sensitivity analyses and neural interpretation diagrams we were able to understand how the different models predicted nest occurrence and breeding success and compare their differences and similarities. Both models also predicted increasing nest occurrence/breeding success with increasing water depth under the nest and increasing distance to edge. However, relationships for prediction differed in the models. Generalizability of the models was poor except when the marshes had similar values of important variables in the model. ANN models performed better than generalized linear models (GLM) on marshes with similar structures. Generalizability of the models did not differ in nest occurrence and breeding success data. Extensive testing also showed that the GLMs were not necessarily more generalizable than ANNs, suggesting that ANN models make good definitions of a study system but are too specific to generalize well to other ecologically complex systems unless input variable distributions are very similar.