arXiv subjects

Navid Shervani-Tabar

Publications and source records attributed to Navid Shervani-Tabar.

4 recordsLinked to original sources

arXiv2025

Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints

Deep neural networks have achieved impressive performance through carefully engineered training strategies. Nonetheless, such methods lack parallels in biological neural circuits, relying heavily on non-local credit assignment, precise initialization, normalization layers, batch processing, and large datasets. Biologically plausible plasticity rules, such as random feedback alignment, often suffer from instability and unbounded weight growth without these engineered methods, while Hebbian-type schemes fail to provide goal-oriented credit. In this study, we demonstrate that incorporating Oja's plasticity rule into error-driven training yields stable, efficient learning in feedforward and recurrent architectures, obviating the need for carefully engineered tricks. Our results show that Oja's rule preserves richer activation subspaces, mitigates exploding or vanishing signals, and improves short-term memory in recurrent networks. Notably, meta-learned local plasticity rules incorporating Oja's principle not only match but surpass standard backpropagation in data-scarce regimes. These findings reveal a biologically grounded pathway bridging engineered deep networks and plausible synaptic mechanisms.

q-bio.NC↗

arXiv2023

Multiresolution Approximation of a Bayesian Inverse Problem using Second-Generation Wavelets

Bayesian approaches are one of the primary methodologies to tackle an inverse problem in high dimensions. Such an inverse problem arises in hydrology to infer the permeability field given flow data in a porous media. It is common practice to decompose the unknown field into some basis and infer the decomposition parameters instead of directly inferring the unknown. Given the multiscale nature of permeability fields, wavelets are a natural choice for parameterizing them. This study uses a Bayesian approach to incorporate the statistical sparsity that characterizes discrete wavelet coefficients. First, we impose a prior distribution incorporating the hierarchical structure of the wavelet coefficient and smoothness of reconstruction via scale-dependent hyperparameters. Then, Sequential Monte Carlo (SMC) method adaptively explores the posterior density on different scales, followed by model selection based on Bayes Factors. Finally, the permeability field is reconstructed from the coefficients using a multiresolution approach based on second-generation wavelets. Here, observations from the pressure sensor grid network are computed via Multilevel Adaptive Wavelet Collocation Method (AWCM). Results highlight the importance of prior modeling on parameter estimation in the inverse problem.

stat.ME↗

arXiv2023

Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways

Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.

q-bio.NC↗

arXiv2021

Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder

Recent advances in artificial intelligence have propelled the development of innovative computational materials modeling and design techniques. Generative deep learning models have been used for molecular representation, discovery, and design. In this work, we assess the predictive capabilities of a molecular generative model developed based on variational inference and graph theory in the small data regime. Physical constraints that encourage energetically stable molecules are proposed. The encoding network is based on the scattering transform with adaptive spectral filters to allow for better generalization of the model. The decoding network is a one-shot graph generative model that conditions atom types on molecular topology. A Bayesian formalism is considered to capture uncertainties in the predictive estimates of molecular properties. The model's performance is evaluated by generating molecules with desired target properties.

physics.chem-ph↗