Integrative characterization of the topography of V4 neural codes using deep learning approaches

Abstract

Area V4 is a mid-level stage of the macaque ventral visual stream, known to encode intermediate visual features such as color, curvature, corners, texture, three-dimensional (3D) solids, and local form. Classical neurophysiological studies have typically examined these dimensions in isolation, contrasting V4 selectivity for shape versus texture, 3D solid surfaces versus two-dimensional (2D) flat patterns, or object form versus texture. Yet how these tunings relate to one another within individual neurons, and how they are jointly organized across the cortical surface, remain unknown. For instance, does a neuron selective for 2D contour-defined shape prefer 3D solid surfaces or 2D flat surfaces? How are preferences for such heterogeneous attributes arranged in a common topographic map? To address these questions, we leverage V4 "digital twins" -- deep neural network models fitted to large-scale, wide-field calcium imaging data comprising tens of thousands of natural images. These digital twins allow us to systematically probe not only the stimulus dimensions explored in earlier studies, but also new, multidimensional stimulus sets that reveal additional aspects of the V4 code. In this study, we find that neural pixels preferring 2D contour-defined shapes also tend to prefer 3D surface shape defined by shading or texture gradients and by object form. In contrast, pixels preferring 2D texture tend to prefer flat surfaces defined by uniform texture or reflectance. We propose that this division of labor suggests that V4 may decompose the encoding of geometrical shape and surface appearance of visual stimuli into distinct populations of neurons, organized as interleaved clusters in the V4 topographic map.