arXiv subjects

Soumya Banerjee

Publications and source records attributed to Soumya Banerjee.

34 records · Page 2Linked to original sources

arXiv2023

Automatic Recognition of Learning Resource Category in a Digital Library

Digital libraries often face the challenge of processing a large volume of diverse document types. The manual collection and tagging of metadata can be a time-consuming and error-prone task. To address this, we aim to develop an automatic metadata extractor for digital libraries. In this work, we introduce the Heterogeneous Learning Resources (HLR) dataset designed for document image classification. The approach involves decomposing individual learning resources into constituent document images (sheets). These images are then processed through an OCR tool to extract textual representation. State-of-the-art classifiers are employed to classify both the document image and its textual content. Subsequently, the labels of the constituent document images are utilized to predict the label of the overall document.

cs.DL↗

arXiv2023

Streaming LifeLong Learning With Any-Time Inference

Despite rapid advancements in lifelong learning (LLL) research, a large body of research mainly focuses on improving the performance in the existing \textit{static} continual learning (CL) setups. These methods lack the ability to succeed in a rapidly changing \textit{dynamic} environment, where an AI agent needs to quickly learn new instances in a `single pass' from the non-i.i.d (also possibly temporally contiguous/coherent) data streams without suffering from catastrophic forgetting. For practical applicability, we propose a novel lifelong learning approach, which is streaming, i.e., a single input sample arrives in each time step, single pass, class-incremental, and subject to be evaluated at any moment. To address this challenging setup and various evaluation protocols, we propose a Bayesian framework, that enables fast parameter update, given a single training example, and enables any-time inference. We additionally propose an implicit regularizer in the form of snap-shot self-distillation, which effectively minimizes the forgetting further. We further propose an effective method that efficiently selects a subset of samples for online memory rehearsal and employs a new replay buffer management scheme that significantly boosts the overall performance. Our empirical evaluations and ablations demonstrate that the proposed method outperforms the prior works by large margins.

cs.LG↗

arXiv2021

Class Incremental Online Streaming Learning

A wide variety of methods have been developed to enable lifelong learning in conventional deep neural networks. However, to succeed, these methods require a `batch' of samples to be available and visited multiple times during training. While this works well in a static setting, these methods continue to suffer in a more realistic situation where data arrives in \emph{online streaming manner}. We empirically demonstrate that the performance of current approaches degrades if the input is obtained as a stream of data with the following restrictions: $(i)$ each instance comes one at a time and can be seen only once, and $(ii)$ the input data violates the i.i.d assumption, i.e., there can be a class-based correlation. We propose a novel approach (CIOSL) for the class-incremental learning in an \emph{online streaming setting} to address these challenges. The proposed approach leverages implicit and explicit dual weight regularization and experience replay. The implicit regularization is leveraged via the knowledge distillation, while the explicit regularization incorporates a novel approach for parameter regularization by learning the joint distribution of the buffer replay and the current sample. Also, we propose an efficient online memory replay and replacement buffer strategy that significantly boosts the model's performance. Extensive experiments and ablation on challenging datasets show the efficacy of the proposed method.

cs.LG↗

arXiv2021

Variational Rejection Particle Filtering

We present a variational inference (VI) framework that unifies and leverages sequential Monte-Carlo (particle filtering) with \emph{approximate} rejection sampling to construct a flexible family of variational distributions. Furthermore, we augment this approach with a resampling step via Bernoulli race, a generalization of a Bernoulli factory, to obtain a low-variance estimator of the marginal likelihood. Our framework, Variational Rejection Particle Filtering (VRPF), leads to novel variational bounds on the marginal likelihood, which can be optimized efficiently with respect to the variational parameters and generalizes several existing approaches in the VI literature. We also present theoretical properties of the variational bound and demonstrate experiments on various models of sequential data, such as the Gaussian state-space model and variational recurrent neural net (VRNN), on which VRPF outperforms various existing state-of-the-art VI methods.

cs.LG↗

arXiv2020

Segmenting Scientific Abstracts into Discourse Categories: A Deep Learning-Based Approach for Sparse Labeled Data

The abstract of a scientific paper distills the contents of the paper into a short paragraph. In the biomedical literature, it is customary to structure an abstract into discourse categories like BACKGROUND, OBJECTIVE, METHOD, RESULT, and CONCLUSION, but this segmentation is uncommon in other fields like computer science. Explicit categories could be helpful for more granular, that is, discourse-level search and recommendation. The sparsity of labeled data makes it challenging to construct supervised machine learning solutions for automatic discourse-level segmentation of abstracts in non-bio domains. In this paper, we address this problem using transfer learning. In particular, we define three discourse categories BACKGROUND, TECHNIQUE, OBSERVATION-for an abstract because these three categories are the most common. We train a deep neural network on structured abstracts from PubMed, then fine-tune it on a small hand-labeled corpus of computer science papers. We observe an accuracy of 75% on the test corpus. We perform an ablation study to highlight the roles of the different parts of the model. Our method appears to be a promising solution to the automatic segmentation of abstracts, where the labeled data is sparse.

cs.CL↗

arXiv2017

Combinatorial Models for the Variety of Complete Quadrics

We develop several combinatorial models that are useful in the study of the $SL_n$-variety $\mathcal{X}$ of complete quadrics. Barred permutations parameterize the fixed points of the action of a maximal torus $T$ of $SL_n$, while $μ$-involutions parameterize the orbits of a Borel subgroup of $SL_n$. Using these combinatorial objects, we characterize the $T$-stable curves and surfaces on $\mathcal{X}$, compute the $T$-equivariant $K$-theory of $\mathcal{X}$, and describe a Białynicki-Birula cell decomposition for $\mathcal{X}$. Furthermore, we give a computational characterization of the Bruhat order on Borel orbits in $\mathcal{X}$.

math.AG↗

arXiv2016

Analysis of a Planetary Scale Scientific Collaboration Dataset Reveals Novel Patterns

Scientific collaboration networks are an important component of scientific output and contribute significantly to expanding our knowledge and to the economy and gross domestic product of nations. Here we examine a dataset from the Mendeley scientific collaboration network. We analyze this data using a combination of machine learning techniques and dynamical models. We find interesting clusters of countries with different characteristics of collaboration. Some of these clusters are dominated by developed countries that have higher number of self connections compared with connections to other countries. Another cluster is dominated by impoverished nations that have mostly connections and collaborations with other countries but fewer self connections. We also propose a complex systems dynamical model that explains these characteristics. Our model explains how the scientific collaboration networks of impoverished and developing nations change over time. We also find interesting patterns in the behaviour of countries that may reflect past foreign policies and contemporary geopolitics. Our model and analysis gives insights and guidelines into how scientific development of developing countries can be guided. This is intimately related to fostering economic development of impoverished nations and creating a richer and more prosperous society.

cs.SI↗

arXiv2016

Optimal Strategies for Virus Propagation

This paper explores a number of questions regarding optimal strategies evolved by viruses upon entry into a vertebrate host. The infected cell life cycle consists of a non-productively infected stage in which it is producing virions but not releasing them and of a productively infected stage in which it is just releasing virions. The study explores why the infected cell cycle should be so delineated, something which is akin to a classic bang-bang control or all-or-none principle. The times spent in each of these stages represent a viral strategy to optimize peak viral load. Increasing the time spent in the non-productively infected phase (τ1) would lead to a concomitant increase in peak viremia. However increasing this time would also invite a more vigorous response from Cytotoxic T-Lymphocytes (CTLs). Simultaneously, if there is a vigorous antibody response, then we might expect τ1 to be high, in order that the virus builds up its population and conversely if there is a weak antibody response, τ1 might be small. These tradeoffs are explored using a mathematical model of virus propagation using Ordinary Differential Equations (ODEs). The study raises questions about whether common viruses have actually settled into an optimum, the role for reliability and whether experimental infections of hosts with non-endemic strains could help elicit answers about viral progression.

q-bio.CB↗

arXiv2016

A Biologically Inspired Model of Distributed Online Communication Supporting Efficient Search and Diffusion of Innovation

We inhabit a world that is not only small but supports efficient decentralized search - an individual using local information can establish a line of communication with another completely unknown individual. Here we augment a hierarchical social network model with communication between and within communities. We argue that organization into communities would decrease overall decentralized search times. We take inspiration from the biological immune system which organizes search for pathogens in a hybrid modular strategy. Our strategy has relevance in search for rare amounts of information in online social networks. Our work also has implications for design of efficient online networks that could have an impact on networks of human collaboration, scientific collaboration and networks used in targeted manhunts. Real world systems, like online social networks, have high associated delays for long-distance links, since they are built on top of physical networks. Such systems have been shown to densify. Hence such networks will have a communication cost due to space and the requirement of maintaining connections. We have incorporated such a non-spatial cost to communication. We introduce the notion of a community size that increases with the size of the system, which is shown to reduce the time to search for information in networks. Our final strategy balances search times and participation costs and is shown to decrease time to find information in decentralized search in online social networks. Our strategy also balances strong-ties and weak-ties over long distances and may ultimately lead to more productive and innovative networks of human communication and enterprise. We hope that this work will lay the foundation for strategies aimed at producing global scale human interaction networks that are sustainable and lead to a more networked, diverse and prosperous society.

cs.SI↗

arXiv2015

A Multi-Agent System Approach to Load-Balancing and Resource Allocation for Distributed Computing

In this research we use a decentralized computing approach to allocate and schedule tasks on a massively distributed grid. Using emergent properties of multi-agent systems, the algorithm dynamically creates and dissociates clusters to serve the changing resource demands of a global task queue. The algorithm is compared to a standard First-in First-out (FIFO) scheduling algorithm. Experiments done on a simulator show that the distributed resource allocation protocol (dRAP) algorithm outperforms the FIFO scheduling algorithm on time to empty queue, average waiting time and CPU utilization. Such a decentralized computing approach holds promise for massively distributed processing scenarios like SETI@home and Google MapReduce.

cs.NE↗

arXiv2011

Biologically Inspired Design Principles for Scalable, Robust, Adaptive, Decentralized Search and Automated Response (RADAR)

Distributed search problems are ubiquitous in Artificial Life (ALife). Many distributed search problems require identifying a rare and previously unseen event and producing a rapid response. This challenge amounts to finding and removing an unknown needle in a very large haystack. Traditional computational search models are unlikely to find, nonetheless, appropriately respond to, novel events, particularly given data distributed across multiple platforms in a variety of formats and sources with variable and unknown reliability. Biological systems have evolved solutions to distributed search and response under uncertainty. Immune systems and ant colonies efficiently scale up massively parallel search with automated response in highly dynamic environments, and both do so using distributed coordination without centralized control. These properties are relevant to ALife, where distributed, autonomous, robust and adaptive control is needed to design robot swarms, mobile computing networks, computer security systems and other distributed intelligent systems. They are also relevant for searching, tracking the spread of ideas, and understanding the impact of innovations in online social networks. We review design principles for Scalable Robust, Adaptive, Decentralized search with Automated Response (Scalable RADAR) in biology. We discuss how biological RADAR scales up efficiently, and then discuss in detail how modular search in the immune system can be mimicked or built upon in ALife. Such search mechanisms are particularly useful when components have limited capacity to communicate and social or physical distance makes long distance communication more costly.

cs.NE↗

arXiv2010

Immune System Inspired Strategies for Distributed Systems

Many components of the IS are constructed as modular units which do not need to communicate with each other such that the number of components increases but the size remains constant. However, a sub-modular IS architecture in which lymph node number and size both increase sublinearly with body size is shown to efficiently balance the requirements of communication and migration, consistent with experimental data. We hypothesize that the IS architecture optimizes the tradeoff between local search for pathogens and global response using antibodies. Similar to natural immune systems, physical space and resource are also important constraints on Artificial Immune Systems (AIS), especially distributed systems applications used to connect low-powered sensors using short-range wireless communication. AIS problems like distributed robot control will also require a sub-modular architecture to efficiently balance the tradeoff between local search for a solution and global response or proliferation of the solution between different components.

cs.DC↗

arXiv2010

Scale Invariance of Immune System Response Rates and Times: Perspectives on Immune System Architecture and Implications for Artificial Immune Systems

Most biological rates and times decrease systematically with organism body size. We use an ordinary differential equation (ODE) model of West Nile Virus in birds to show that pathogen replication rates decline with host body size, but natural immune system (NIS) response rates do not change systematically with body size. This is surprising since the NIS has to search for small quantities of pathogens through larger physical spaces in larger organisms, and also respond by producing larger absolute quantities of antibody in larger organisms. We call this scale-invariant detection and response. We hypothesize that the NIS has evolved an architecture to efficiently neutralize pathogens. We investigate a range of architectures using an Agent Based Model (ABM). We find that a sub-modular NIS architecture, in which lymph node number and size both increase sublinearly with body size, efficiently balances the tradeoff between local pathogen detection and global response using antibodies. This leads to nearly scale-invariant detection and response, consistent with experimental data. Similar to the NIS, physical space and resources are also important constraints on Artificial Immune Systems (AIS), especially distributed systems applications used to connect low-powered sensors using short-range wireless communication. We show that AIS problems, like distributed robot control, will also require a sub-modular architecture to efficiently balance the tradeoff between local search for a solution and global response or proliferation of the solution between different components. This research has wide applicability in other distributed systems AIS applications.

q-bio.QM↗

arXiv2010

Modular RADAR: An Immune System Inspired Search and Response Strategy for Distributed Systems

The Natural Immune System (NIS) is a distributed system that solves challenging search and response problems while operating under constraints imposed by physical space and resource availability. Remarkably, NIS search and response times do not scale appreciably with the physical size of the animal in which its search is conducted. Many distributed systems are engineered to solve analogous problems, and the NIS demonstrates how such engineered systems can achieve desirable scalability. We hypothesize that the architecture of the NIS, composed of a hierarchical decentralized detection network of lymph nodes (LN) facilitates efficient search and response. A sub-modular architecture in which LN numbers and size both scale with organism size is shown to efficiently balance tradeoffs between local antigen detection and global antibody production, leading to nearly scale-invariant detection and response. We characterize the tradeoffs as balancing local and global communication and show that similar tradeoffs exist in distributed systems like LN inspired artificial immune system (AIS) applications and peer-to-peer (P2P) systems. Taking inspiration from the architecture of the NIS, we propose a modular RADAR (Robust Adaptive Decentralized search with Automated Response) strategy for distributed systems. We demonstrate how two existing distributed systems (a LN inspired multi-robot control application and a P2P system) can be improved by a modular RADAR strategy. Such a sub-modular architecture is shown to balance the tradeoffs between local communication (within artificial LNs and P2P clusters) and global communication (between artificial LNs and P2P clusters), leading to efficient search and response.

cs.DC↗

arXiv2009

A Hybrid Agent Based and Differential Equation Model of Body Size Effects on Pathogen Replication and Immune System Response

Many emerging pathogens infect multiple host species, and multi-host pathogens may have very different dynamics in different host species. This research addresses how pathogen replication rates and Immune System (IS) response times are constrained by host body size. An Ordinary Differential Equation (ODE) model is used to show that pathogen replication rates decline with host body size but IS response rates remain invariant with body size. An Agent-Based Model (ABM) is used to investigate two models of IS architecture that could explain scale invariance of IS response rates. A stage structured hybrid model is proposed that strikes a balance between the detailed representation of an ABM and computational tractability of an ODE, by using them in the initial and latter stages of an infection, respectively.

q-bio.OT↗

arXiv2009

An Immune System Inspired Approach to Automated Program Verification

An immune system inspired Artificial Immune System (AIS) algorithm is presented, and is used for the purposes of automated program verification. Relevant immunological concepts are discussed and the field of AIS is briefly reviewed. It is proposed to use this AIS algorithm for a specific automated program verification task: that of predicting shape of program invariants. It is shown that the algorithm correctly predicts program invariant shape for a variety of benchmarked programs.

cs.NE↗