arXiv subjects

Shiming Tang

Publications and source records attributed to Shiming Tang.

4 recordsLinked to original sources

arXiv2025

Integrative characterization of the topography of V4 neural codes using deep learning approaches

Area V4 is a mid-level stage of the macaque ventral visual stream, known to encode intermediate visual features such as color, curvature, corners, texture, three-dimensional (3D) solids, and local form. Classical neurophysiological studies have typically examined these dimensions in isolation, contrasting V4 selectivity for shape versus texture, 3D solid surfaces versus two-dimensional (2D) flat patterns, or object form versus texture. Yet how these tunings relate to one another within individual neurons, and how they are jointly organized across the cortical surface, remain unknown. For instance, does a neuron selective for 2D contour-defined shape prefer 3D solid surfaces or 2D flat surfaces? How are preferences for such heterogeneous attributes arranged in a common topographic map? To address these questions, we leverage V4 "digital twins" -- deep neural network models fitted to large-scale, wide-field calcium imaging data comprising tens of thousands of natural images. These digital twins allow us to systematically probe not only the stimulus dimensions explored in earlier studies, but also new, multidimensional stimulus sets that reveal additional aspects of the V4 code. In this study, we find that neural pixels preferring 2D contour-defined shapes also tend to prefer 3D surface shape defined by shading or texture gradients and by object form. In contrast, pixels preferring 2D texture tend to prefer flat surfaces defined by uniform texture or reflectance. We propose that this division of labor suggests that V4 may decompose the encoding of geometrical shape and surface appearance of visual stimuli into distinct populations of neurons, organized as interleaved clusters in the V4 topographic map.

q-bio.NC↗

arXiv2024

Self-Attention-Based Contextual Modulation Improves Neural System Identification

Convolutional neural networks (CNNs) have been shown to be state-of-the-art models for visual cortical neurons. Cortical neurons in the primary visual cortex are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs integrate global contextual information to model contextual modulation via two mechanisms: successive convolutions and a fully connected readout layer. In this paper, we find that self-attention (SA), an implementation of non-local network mechanisms, can improve neural response predictions over parameter-matched CNNs in two key metrics: tuning curve correlation and peak tuning. We introduce peak tuning as a metric to evaluate a model's ability to capture a neuron's top feature preference. We factorize networks to assess each context mechanism, revealing that information in the local receptive field is most important for modeling overall tuning, but surround information is critically necessary for characterizing the tuning peak. We find that self-attention can replace posterior spatial-integration convolutions when learned incrementally, and is further enhanced in the presence of a fully connected readout layer, suggesting that the two context mechanisms are complementary. Finally, we find that decomposing receptive field learning and contextual modulation learning in an incremental manner may be an effective and robust mechanism for learning surround-center interactions.

cs.CV↗

arXiv2023

A large calcium-imaging dataset reveals a systematic V4 organization for natural scenes

The visual system evolved to process natural scenes, yet most of our understanding of the topology and function of visual cortex derives from studies using artificial stimuli. To gain deeper insights into visual processing of natural scenes, we utilized widefield calcium-imaging of primate V4 in response to many natural images, generating a large dataset of columnar-scale responses. We used this dataset to build a digital twin of V4 via deep learning, generating a detailed topographical map of natural image preferences at each cortical position. The map revealed clustered functional domains for specific classes of natural image features. These ranged from surface-related attributes like color and texture to shape-related features such as edges, curvature, and facial features. We validated the model-predicted domains with additional widefield calcium-imaging and single-cell resolution two-photon imaging. Our study illuminates the detailed topological organization and neural codes in V4 that represent natural scenes.

q-bio.NC↗

arXiv2019

Complexity and Diversity in Sparse Code Priors Improve Receptive Field Characterization of Macaque V1 Neurons

System identification techniques -- projection pursuit regression models (PPRs) and convolutional neural networks (CNNs) -- provide state-of-the-art performance in predicting visual cortical neurons' responses to arbitrary input stimuli. However, the constituent kernels recovered by these methods are often noisy and lack coherent structure, making it difficult to understand the underlying component features of a neuron's receptive field. In this paper, we show that using a dictionary of diverse kernels with complex shapes learned from natural scenes based on efficient coding theory, as the front-end for PPRs and CNNs can improve their performance in neuronal response prediction as well as algorithmic data efficiency and convergence speed. Extensive experimental results also indicate that these sparse-code kernels provide important information on the component features of a neuron's receptive field. In addition, we find that models with the complex-shaped sparse code front-end are significantly better than models with a standard orientation-selective Gabor filter front-end for modeling V1 neurons that have been found to exhibit complex pattern selectivity. We show that the relative performance difference due to these two front-ends can be used to produce a sensitive metric for detecting complex selectivity in V1 neurons.

q-bio.QM↗