arXiv subjects

Nithin Nagaraj

Publications and source records attributed to Nithin Nagaraj.

At least 37 records · Page 2Linked to original sources

arXiv2023

To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks

Reducing the size of a neural network (pruning) by removing weights without impacting its performance is an important problem for resource-constrained devices. In the past, pruning was typically accomplished by ranking or penalizing weights based on criteria like magnitude and removing low-ranked weights before retraining the remaining ones. Pruning strategies may also involve removing neurons from the network in order to achieve the desired reduction in network size. We formulate pruning as an optimization problem with the objective of minimizing misclassifications by selecting specific weights. To accomplish this, we have introduced the concept of chaos in learning (Lyapunov exponents) via weight updates and exploiting causality to identify the causal weights responsible for misclassification. Such a pruned network maintains the original performance and retains feature explainability.

cs.LG↗

arXiv2023

Kolam Simulation using Angles at Lattice Points

Kolam is a ritual art form practised by people in South India and consists of rule-bound geometric patterns of dots and lines. Single loop Kolams are mathematical closed loop patterns drawn over a grid of dots and conforming to certain heuristics. In this work, we propose a novel encoding scheme where we map the angular movements of Kolam at lattice points into sequences containing $4$ distinct symbols. This is then used to simulate single loop Kolam procedure via turtle moves in accordance with the desired angular direction at specific points. We thus obtain sequential codes for Kolams, unique up to cyclic permutations. We specify the requirements for the algorithm and indicate the general methodology. We demonstrate a sample of Kolams using our algorithm with a software implementation in Python.

cs.IT↗

arXiv2022

Fairly Constricted Multi-Objective Particle Swarm Optimization

It has been well documented that the use of exponentially-averaged momentum (EM) in particle swarm optimization (PSO) is advantageous over the vanilla PSO algorithm. In the single-objective setting, it leads to faster convergence and avoidance of local minima. Naturally, one would expect that the same advantages of EM carry over to the multi-objective setting. Hence, we extend the state of the art Multi-objective optimization (MOO) solver, SMPSO, by incorporating EM in it. As a consequence, we develop the mathematical formalism of constriction fairness which is at the core of extended SMPSO algorithm. The proposed solver matches the performance of SMPSO across the ZDT, DTLZ and WFG problem suites and even outperforms it in certain instances.

cs.NE↗

arXiv2022

Granger Causality for Compressively Sensed Sparse Signals

Compressed sensing is a scheme that allows for sparse signals to be acquired, transmitted and stored using far fewer measurements than done by conventional means employing Nyquist sampling theorem. Since many naturally occurring signals are sparse (in some domain), compressed sensing has rapidly seen popularity in a number of applied physics and engineering applications, particularly in designing signal and image acquisition strategies, e.g., magnetic resonance imaging, quantum state tomography, scanning tunneling microscopy, analog to digital conversion technologies. Contemporaneously, causal inference has become an important tool for the analysis and understanding of processes and their interactions in many disciplines of science, especially those dealing with complex systems. Direct causal analysis for compressively sensed data is required to avoid the task of reconstructing the compressed data. Also, for some sparse signals, such as for sparse temporal data, it may be difficult to discover causal relations directly using available data-driven/ model-free causality estimation techniques. In this work, we provide a mathematical proof that structured compressed sensing matrices, specifically Circulant and Toeplitz, preserve causal relationships in the compressed signal domain, as measured by Granger Causality. We then verify this theorem on a number of bivariate and multivariate coupled sparse signal simulations which are compressed using these matrices. We also demonstrate a real world application of network causal connectivity estimation from sparse neural spike train recordings from rat prefrontal cortex.

eess.SP↗

arXiv2022

Neurochaos Feature Transformation and Classification for Imbalanced Learning

Learning from limited and imbalanced data is a challenging problem in the Artificial Intelligence community. Real-time scenarios demand decision-making from rare events wherein the data are typically imbalanced. These situations commonly arise in medical applications, cybersecurity, catastrophic predictions etc. This motivates the development of learning algorithms capable of learning from imbalanced data. Human brain effortlessly learns from imbalanced data. Inspired by the chaotic neuronal firing in the human brain, a novel learning algorithm namely Neurochaos Learning (NL) was recently proposed. NL is categorized in three blocks: Feature Transformation, Neurochaos Feature Extraction (CFX), and Classification. In this work, the efficacy of neurochaos feature transformation and extraction for classification in imbalanced learning is studied. We propose a unique combination of neurochaos based feature transformation and extraction with traditional ML algorithms. The explored datasets in this study revolve around medical diagnosis, banknote fraud detection, environmental applications and spoken-digit classification. In this study, experiments are performed in both high and low training sample regime. In the former, five out of nine datasets have shown a performance boost in terms of macro F1-score after using CFX features. The highest performance boost obtained is 25.97% for Statlog (Heart) dataset using CFX+Decision Tree. In the low training sample regime (from just one to nine training samples per class), the highest performance boost of 144.38% is obtained for Haberman's Survival dataset using CFX+Random Forest. NL offers enormous flexibility of combining CFX with any ML classifier to boost its performance, especially for learning tasks with limited and imbalanced data.

cs.NE↗

arXiv2022

Cause-Effect Preservation and Classification using Neurochaos Learning

Discovering cause-effect from observational data is an important but challenging problem in science and engineering. In this work, a recently proposed brain inspired learning algorithm namely-\emph{Neurochaos Learning} (NL) is used for the classification of cause-effect from simulated data. The data instances used are generated from coupled AR processes, coupled 1D chaotic skew tent maps, coupled 1D chaotic logistic maps and a real-world prey-predator system. The proposed method consistently outperforms a five layer Deep Neural Network architecture for coupling coefficient values ranging from $0.1$ to $0.7$. Further, we investigate the preservation of causality in the feature extracted space of NL using Granger Causality (GC) for coupled AR processes and and Compression-Complexity Causality (CCC) for coupled chaotic systems and real-world prey-predator dataset. This ability of NL to preserve causality under a chaotic transformation and successfully classify cause and effect time series (including a transfer learning scenario) is highly desirable in causal machine learning applications.

cs.LG↗

arXiv2021

Learning Generalized Causal Structure in Time-series

The science of causality explains/determines 'cause-effect' relationship between the entities of a system by providing mathematical tools for the purpose. In spite of all the success and widespread applications of machine-learning (ML) algorithms, these algorithms are based on statistical learning alone. Currently, they are nowhere close to 'human-like' intelligence as they fail to answer and learn based on the important "Why?" questions. Hence, researchers are attempting to integrate ML with the science of causality. Among the many causal learning issues encountered by ML, one is that these algorithms are dumb to the temporal order or structure in data. In this work we develop a machine learning pipeline based on a recently proposed 'neurochaos' feature learning technique (ChaosFEX feature extractor), that helps us to learn generalized causal-structure in given time-series data.

cs.LG↗

arXiv2021

Problems with information theoretic approaches to causal learning

The language of information theory is favored in both causal reasoning and machine learning frameworks. But, is there a better language than this? In this study, we demonstrate the pitfalls of infotheoretic estimation using first order statistics on (short) sequences for causal learning. We recommend the use of data compression based approaches for causality testing since these make very little assumptions on data as opposed to infotheoretic measures, and are more robust to finite data length effects. We conclude with a discussion on the challenges posed in modeling the effects of conditioning process $X$ with another process $Y$ in causal machine learning. Specifically, conditioning can increase 'confusion' which can be difficult to model by classical information theory. A conscious causal agent creates new choices, decisions and meaning which poses huge challenges for AI.

cs.IT↗

arXiv2021

Causal Analysis of Carnatic Music: A Preliminary Study

The musicological analysis of Carnatic music is challenging, owing to its rich structure and complexity. Automated \textit{rāga} classification, pitch detection, tonal analysis, modelling and information retrieval of this form of southern Indian classical music have, however, made significant progress in recent times. A causal analysis to investigate the musicological structure of Carnatic compositions and the identification of the relationships embedded in them have never been previously attempted. In this study, we propose a novel framework for causal discovery, using a compression-complexity measure. Owing to the limited number of compositions available, however, we generated surrogates to further facilitate the analysis of the prevailing causal relationships. Our analysis indicates that the context-free grammar, inferred from more complex compositions, such as the \textit{Mē\d{l}akarta} \textit{rāga}, are a \textit{structural cause} for the \textit{Janya} \textit{rāga}. We also analyse certain special cases of the \textit{Janya rāga} in order to understand their origins and structure better.

cs.SD↗

arXiv2021

Causal Discovery using Compression-Complexity Measures

Causal inference is one of the most fundamental problems across all domains of science. We address the problem of inferring a causal direction from two observed discrete symbolic sequences $X$ and $Y$. We present a framework which relies on lossless compressors for inferring context-free grammars (CFGs) from sequence pairs and quantifies the extent to which the grammar inferred from one sequence compresses the other sequence. We infer $X$ causes $Y$ if the grammar inferred from $X$ better compresses $Y$ than in the other direction. To put this notion to practice, we propose three models that use the Compression-Complexity Measures (CCMs) - Lempel-Ziv (LZ) complexity and Effort-To-Compress (ETC) to infer CFGs and discover causal directions without demanding temporal structures. We evaluate these models on synthetic and real-world benchmarks and empirically observe performances competitive with current state-of-the-art methods. Lastly, we present two unique applications of the proposed models for causal inference directly from pairs of genome sequences belonging to the SARS-CoV-2 virus. Using a large number of sequences, we show that our models capture directed causal information exchange between sequence pairs, presenting novel opportunities for addressing key issues such as contact-tracing, motif discovery, evolution of virulence and pathogenicity in future applications.

cs.LG↗

arXiv2021

When Noise meets Chaos: Stochastic Resonance in Neurochaos Learning

Chaos and Noise are ubiquitous in the Brain. Inspired by the chaotic firing of neurons and the constructive role of noise in neuronal models, we for the first time connect chaos, noise and learning. In this paper, we demonstrate Stochastic Resonance (SR) phenomenon in Neurochaos Learning (NL). SR manifests at the level of a single neuron of NL and enables efficient subthreshold signal detection. Furthermore, SR is shown to occur in single and multiple neuronal NL architecture for classification tasks - both on simulated and real-world spoken digit datasets. Intermediate levels of noise in neurochaos learning enables peak performance in classification tasks thus highlighting the role of SR in AI applications, especially in brain inspired learning architectures.

q-bio.NC↗

arXiv2020

A Neurochaos Learning Architecture for Genome Classification

There has been empirical evidence of presence of non-linearity and chaos at the level of single neurons in biological neural networks. The properties of chaotic neurons inspires us to employ them in artificial learning systems. Here, we propose a Neurochaos Learning (NL) architecture, where the neurons used to extract features from data are 1D chaotic maps. ChaosFEX+SVM, an instance of this NL architecture, is proposed as a hybrid combination of chaos and classical machine learning algorithm. We formally prove that a single layer of NL with a finite number of 1D chaotic neurons satisfies the Universal Approximation Theorem with an exact value for the number of chaotic neurons needed to approximate a discrete real valued function with finite support. This is made possible due to the topological transitivity property of chaos and the existence of uncountably infinite number of dense orbits for the chosen 1D chaotic map. The chaotic neurons in NL get activated under the presence of an input stimulus (data) and output a chaotic firing trajectory. From such chaotic firing trajectories of individual neurons of NL, we extract Firing Time, Firing Rate, Energy and Entropy that constitute ChaosFEX features. These ChaosFEX features are then fed to a Support Vector Machine with linear kernel for classification. The effectiveness of chaotic feature engineering performed by NL (ChaosFEX+SVM) is demonstrated for synthetic and real world datasets in the low and high training sample regimes. Specifically, we consider the problem of classification of genome sequences of SARS-CoV-2 from other coronaviruses (SARS-CoV-1, MERS-CoV and others). With just one training sample per class for 1000 random trials of training, we report an average macro F1-score > 0.99 for the classification of SARS-CoV-2 from SARS-CoV-1 genome sequences. Robustness of ChaosFEX features to additive noise is also demonstrated.

cs.NE↗

arXiv2019

Measuring Causality: The Science of Cause and Effect

Determining and measuring cause-effect relationships is fundamental to most scientific studies of natural phenomena. The notion of causation is distinctly different from correlation which only looks at association of trends or patterns in measurements. In this article, we review different notions of causality and focus especially on measuring causality from time series data. Causality testing finds numerous applications in diverse disciplines such as neuroscience, econometrics, climatology, physics and artificial intelligence.

stat.ME↗

arXiv2019

ChaosNet: A Chaos based Artificial Neural Network Architecture for Classification

Inspired by chaotic firing of neurons in the brain, we propose ChaosNet -- a novel chaos based artificial neural network architecture for classification tasks. ChaosNet is built using layers of neurons, each of which is a 1D chaotic map known as the Generalized Luroth Series (GLS) which has been shown in earlier works to possess very useful properties for compression, cryptography and for computing XOR and other logical operations. In this work, we design a novel learning algorithm on ChaosNet that exploits the topological transitivity property of the chaotic GLS neurons. The proposed learning algorithm gives consistently good performance accuracy in a number of classification tasks on well known publicly available datasets with very limited training samples. Even with as low as 7 (or fewer) training samples/class (which accounts for less than 0.05% of the total available data), ChaosNet yields performance accuracies in the range 73.89 % - 98.33 %. We demonstrate the robustness of ChaosNet to additive parameter noise and also provide an example implementation of a 2-layer ChaosNet for enhancing classification accuracy. We envisage the development of several other novel learning algorithms on ChaosNet in the near future.

cs.LG↗

arXiv2019

Causal Stability and Synchronization

Synchronization of chaos arises between coupled dynamical systems and is very well understood as a temporal phenomena which leads the coupled systems to converge or develop a dependence with time. In this work, we provide a complementary spatial perspective to this phenomenon by introducing the novel idea of causal stability. We then propose and prove a causal stability synchronization theorem as a necessary and sufficient condition for synchronization. We also provide an empirical criteria to identify synchronizing variables in coupled identical chaotic dynamical systems based on causality analysis on time series data of the driving system alone.

math.DS↗

arXiv2019

Evolution of Novel Activation Functions in Neural Network Training with Applications to Classification of Exoplanets

We present analytical exploration of novel activation functions as consequence of integration of several ideas leading to implementation and subsequent use in habitability classification of exoplanets. Neural networks, although a powerful engine in supervised methods, often require expensive tuning efforts for optimized performance. Habitability classes are hard to discriminate, especially when attributes used as hard markers of separation are removed from the data set. The solution is approached from the point of investigating analytical properties of the proposed activation functions. The theory of ordinary differential equations and fixed point are exploited to justify the "lack of tuning efforts" to achieve optimal performance compared to traditional activation functions. Additionally, the relationship between the proposed activation functions and the more popular ones is established through extensive analytical and empirical evidence. Finally, the activation functions have been implemented in plain vanilla feed-forward neural network to classify exoplanets.

astro-ph.IM↗

arXiv2019

A Novel Chaos Theory Inspired Neuronal Architecture

The practical success of widely used machine learning (ML) and deep learning (DL) algorithms in Artificial Intelligence (AI) community owes to availability of large datasets for training and huge computational resources. Despite the enormous practical success of AI, these algorithms are only loosely inspired from the biological brain and do not mimic any of the fundamental properties of neurons in the brain, one such property being the chaotic firing of biological neurons. This motivates us to develop a novel neuronal architecture where the individual neurons are intrinsically chaotic in nature. By making use of the topological transitivity property of chaos, our neuronal network is able to perform classification tasks with very less number of training samples. For the MNIST dataset, with as low as $0.1 \%$ of the total training data, our method outperforms ML and matches DL in classification accuracy for up to $7$ training samples/class. For the Iris dataset, our accuracy is comparable with ML algorithms, and even with just two training samples/class, we report an accuracy as high as $95.8 \%$. This work highlights the effectiveness of chaos and its properties for learning and paves the way for chaos-inspired neuronal architectures by closely mimicking the chaotic nature of neurons in the brain.

q-bio.NC↗

arXiv2018

Causality Testing: A Data Compression Framework

Causality testing, the act of determining cause and effect from measurements, is widely used in physics, climatology, neuroscience, econometrics and other disciplines. As a result, a large number of causality testing methods based on various principles have been developed. Causal relationships in complex systems are typically accompanied by entropic exchanges which are encoded in patterns of dynamical measurements. A data compression algorithm which can extract these encoded patterns could be used for inferring these relations. This motivates us to propose, for the first time, a generic causality testing framework based on data compression. The framework unifies existing causality testing methods and enables us to innovate a novel Compression-Complexity Causality measure. This measure is rigorously tested on simulated and real-world time series and is found to overcome the limitations of Granger Causality and Transfer Entropy, especially for noisy and non-synchronous measurements. Additionally, it gives insight on the `kind' of causal influence between input time series by the notions of positive and negative causality.

physics.data-an↗