arXiv subjects

Shreya Saha

Publications and source records attributed to Shreya Saha.

8 recordsLinked to original sources

arXiv2026

Integrated representational signatures strengthen specificity in brains and models

The extent to which different neural or artificial neural networks (models) rely on equivalent representations to support similar tasks remains a central question in neuroscience and machine learning. Prior work has typically compared systems using a single representational similarity metric, yet each captures only one facet of representational structure. To address this, we leverage a suite of representational similarity metrics-each capturing a distinct facet of representational correspondence, such as geometry, unit-level tuning, or linear decodability-and assess brain region or model separability using multiple complementary measures. Metrics that preserve geometric or tuning structure (e.g., RSA, Soft Matching) yield stronger region-based discrimination, whereas more flexible mappings such as Linear Predictivity show weaker separation. These findings suggest that geometry and tuning encode brain-region- or model-family-specific signatures, while linearly decodable information tends to be more globally shared across regions or models. To integrate these complementary representational facets, we adapt Similarity Network Fusion (SNF), a framework originally developed for multi-omics data integration. SNF produces substantially sharper regional and model family-level separation than any single metric and yields robust composite similarity profiles. Moreover, clustering cortical regions using SNF-derived similarity scores reveals a clearer hierarchical organization that aligns closely with established anatomical and functional hierarchies of the visual cortex-surpassing the correspondence achieved by individual metrics.

q-bio.NC↗

arXiv2026

Comparing and Integrating Different Notions of Representational Correspondence in Neural Systems

The extent to which different biological and artificial neural systems rely on equivalent internal representations to support similar tasks remains a central question in neuroscience and machine learning. Prior work typically compares systems using a single representational similarity metric, even though different metrics emphasize distinct facets of representational correspondence. Here we evaluate a suite of representational similarity measures by asking how well each metric recovers known structure across two domains: for artificial models, whether procedurally dissimilar models (differing in architecture or training paradigm) are assigned lower similarity than procedurally matched models; and for neural data, whether responses from distinct cortical regions are separated while responses from the same region align across subjects. Across both vision models and neural recordings, metrics that preserve representational geometry or tuning structure more reliably separate this structure than more flexible mappings such as linear predictivity. To integrate these complementary facets, we adapt Similarity Network Fusion, originally developed for multi-omics integration, to combine similarity graphs across metrics. The resulting fused similarity yields sharper separation of procedurally defined model families and, when applied to neural data, recovers a clearer hierarchical organization of the ventral visual stream that aligns more closely with established anatomical and functional hierarchies than single metrics. Overall, this approach reveals which dimensions of representational correspondence recover meaningful structure in models and brains, and how complementary notions of similarity can be integrated.

cs.CV↗

arXiv2026

Barycentric alignment for instance-level comparison of neural representations

Comparing representations across neural networks is challenging because representations admit symmetries, such as arbitrary reordering of units or rotations of activation space, that obscure underlying equivalence between models. We introduce a barycentric alignment framework that quotients out these nuisance symmetries to construct a universal embedding space across many models. Unlike existing similarity measures, which summarize relationships over entire stimulus sets, this framework enables similarity to be defined at the level of individual stimuli, revealing inputs that elicit convergent versus divergent representations across models. Using this instance-level notion of similarity, we identify systematic input properties that predict representational convergence versus divergence across vision and language model families. We also construct universal embedding spaces for brain representations across individuals and cortical regions, enabling instance-level comparison of representational agreement across stages of the human visual hierarchy. Finally, we apply the same barycentric alignment framework to purely unimodal vision and language models and find that post-hoc alignment into a shared space yields image text similarity scores that closely track human cross-modal judgments and approach the performance of contrastively trained vision-language models. This strikingly suggests that independently learned representations already share sufficient geometric structure for human-aligned cross-modal comparison. Together, these results show that resolving representational similarity at the level of individual stimuli reveals phenomena that cannot be detected by set-level comparison metrics.

cs.LG↗

arXiv2025

Measuring the Measures: Discriminative Capacity of Representational Similarity Metrics Across Model Families

Representational similarity metrics are fundamental tools in neuroscience and AI, yet we lack systematic comparisons of their discriminative power across model families. We introduce a quantitative framework to evaluate representational similarity measures based on their ability to separate model families-across architectures (CNNs, Vision Transformers, Swin Transformers, ConvNeXt) and training regimes (supervised vs. self-supervised). Using three complementary separability measures-dprime from signal detection theory, silhouette coefficients and ROC-AUC, we systematically assess the discriminative capacity of commonly used metrics including RSA, linear predictivity, Procrustes, and soft matching. We show that separability systematically increases as metrics impose more stringent alignment constraints. Among mapping-based approaches, soft-matching achieves the highest separability, followed by Procrustes alignment and linear predictivity. Non-fitting methods such as RSA also yield strong separability across families. These results provide the first systematic comparison of similarity metrics through a separability lens, clarifying their relative sensitivity and guiding metric choice for large-scale model and brain comparisons.

cs.LG↗

arXiv2025

Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness

The human language system represents both linguistic forms and meanings, but the abstractness of the meaning representations remains debated. Here, we searched for abstract representations of meaning in the language cortex by modeling neural responses to sentences using representations from vision and language models. When we generate images corresponding to sentences and extract vision model embeddings, we find that aggregating across multiple generated images yields increasingly accurate predictions of language cortex responses, sometimes rivaling large language models. Similarly, averaging embeddings across multiple paraphrases of a sentence improves prediction accuracy compared to any single paraphrase. Enriching paraphrases with contextual details that may be implicit (e.g., augmenting "I had a pancake" to include details like "maple syrup") further increases prediction accuracy, even surpassing predictions based on the embedding of the original sentence, suggesting that the language system maintains richer and broader semantic representations than language models. Together, these results demonstrate the existence of highly abstract, form-independent meaning representations within the language cortex.

cs.CL↗

arXiv2025

Modeling the Human Visual System: Comparative Insights from Response-Optimized and Task-Optimized Vision Models, Language Models, and different Readout Mechanisms

Over the past decade, predictive modeling of neural responses in the primate visual system has advanced significantly, largely driven by various DNN approaches. These include models optimized directly for visual recognition, cross-modal alignment through contrastive objectives, neural response prediction from scratch, and large language model embeddings.Likewise, different readout mechanisms, ranging from fully linear to spatial-feature factorized methods have been explored for mapping network activations to neural responses. Despite the diversity of these approaches, it remains unclear which method performs best across different visual regions. In this study, we systematically compare these approaches for modeling the human visual system and investigate alternative strategies to improve response predictions. Our findings reveal that for early to mid-level visual areas, response-optimized models with visual inputs offer superior prediction accuracy, while for higher visual regions, embeddings from LLMs based on detailed contextual descriptions of images and task-optimized models pretrained on large vision datasets provide the best fit. Through comparative analysis of these modeling approaches, we identified three distinct regions in the visual cortex: one sensitive primarily to perceptual features of the input that are not captured by linguistic descriptions, another attuned to fine-grained visual details representing semantic information, and a third responsive to abstract, global meanings aligned with linguistic content. We also highlight the critical role of readout mechanisms, proposing a novel scheme that modulates receptive fields and feature maps based on semantic content, resulting in an accuracy boost of 3-23% over existing SOTAs for all models and brain regions. Together, these findings offer key insights into building more precise models of the visual system.

cs.NE↗

arXiv2025

BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields

Reconstruction of deformable scenes from endoscopic videos is important for many applications such as intraoperative navigation, surgical visual perception, and robotic surgery. It is a foundational requirement for realizing autonomous robotic interventions for minimally invasive surgery. However, previous approaches in this domain have been limited by their modular nature and are confined to specific camera and scene settings. Our work adopts the Neural Radiance Fields (NeRF) approach to learning 3D implicit representations of scenes that are both dynamic and deformable over time, and furthermore with unknown camera poses. We demonstrate this approach on endoscopic surgical scenes from robotic surgery. This work removes the constraints of known camera poses and overcomes the drawbacks of the state-of-the-art unstructured dynamic scene reconstruction technique, which relies on the static part of the scene for accurate reconstruction. Through several experimental datasets, we demonstrate the versatility of our proposed model to adapt to diverse camera and scene settings, and show its promise for both current and future robotic surgical systems.

cs.CV↗

arXiv2024

Signatures of topology in generic transport measurements for Rarita-Schwinger-Weyl semimetals

We investigate how the signatures of the topological properties of the bandstructures for nodal-point semimetals are embedded in the response coefficients, arising in two distinct experimental set-ups, by taking the Rarita-Schwinger-Weyl (RSW) semimetal as an example. The first scenario involves the computation of third-rank tensors representing second-order response coefficients, relating the charge/thermal current densities to the combined effects of the gradient of the chemical potential and an external electric field/temperature gradient. On the premises that internode scatterings can be ignored, the relaxation-time approximation leads to a quantized value for the nonvanishing components of each of these nonlinear response tensors, characterizing a single untilted RSW node. Furthermore, the final expressions turn out to be insensitive to the specific values of the chemical potential and the temperature. The second scenario involves computing the magnetoelectric conductivity under the action of collinear electric ($\mathbf E$) and magnetic ($\mathbf B$) fields, representing a planar Hall set-up. In particular, our focus is in bringing out the dependence of the linear-in-$|\mathbf B|$ parts of the conductivity tensor on the intrinsic topological properties of the bandstructure, which are nonvanishing only in the presence of a nonzero tilt in the energy spectrum.

cond-mat.mes-hall↗