arXiv subjects

Srijan Sengupta

Publications and source records attributed to Srijan Sengupta.

32 records · Page 2Linked to original sources

arXiv2023

Word Embeddings as Statistical Estimators

Word embeddings are a fundamental tool in natural language processing. Currently, word embedding methods are evaluated on the basis of empirical performance on benchmark data sets, and there is a lack of rigorous understanding of their theoretical properties. This paper studies word embeddings from a statistical theoretical perspective, which is essential for formal inference and uncertainty quantification. We propose a copula-based statistical model for text data and show that under this model, the now-classical Word2Vec method can be interpreted as a statistical estimation method for estimating the theoretical pointwise mutual information (PMI). Next, by building on the work of Levy and Goldberg (2014), we develop a missing value-based estimator as a statistically tractable and interpretable alternative to the Word2Vec approach. The estimation error of this estimator is comparable to Word2Vec and improves upon the truncation-based method proposed by Levy and Goldberg (2014). The proposed estimator also performs comparably to Word2Vec in a benchmark sentiment analysis task on the IMDb Movie Reviews data set.

stat.ME↗

arXiv2021

Social Sensors in Epidemiological Networks via Graph Eigenvectors

In this paper, we consider epidemiological networks which are used for modeling the transmission of contagious diseases through a population. Specifically, we study the so-called social sensors problem: given an epidemiological network, can we find a small set of nodes such that by monitoring disease transmission on these nodes, we can get ahead of the overall epidemic in the full population? In spite of its societal relevance, there has not been much statistical work on this problem, and we aim to provide an exposition that will hopefully stimulate interest in the research community. Furthermore, by leveraging classical results in spectral graph theory, we propose a novel method for finding social sensors, which achieves substantial improvement over existing methods in both synthetic and real-world epidemiological networks.

cs.SI↗

arXiv2021

Scalable Community Extraction of Text Networks for Automated Grouping in Medical Databases

Networks are ubiquitous in today's world. Community structure is a well-known feature of many empirical networks, and a lot of statistical methods have been developed for community detection. In this paper, we consider the problem of community structure in text networks,which is greatly relevant in medical errors and patient safety databases. We adapt a well-known community extraction method to develop a scalable algorithm for community extraction in large text databases. The application of our method on a real-world patient safety report database demonstrates that the groups generated from community extraction are much more accurate than manual tagging by frontline workers.

cs.SI↗

arXiv2021

The Value of Summary Statistics for Anomaly Detection in Temporally-Evolving Networks: A Performance Evaluation Study

Network data has emerged as an active research area in statistics. Much of the focus of ongoing research has been on static networks that represent a single snapshot or aggregated historical data unchanging over time. However, most networks result from temporally-evolving systems that exhibit intrinsic dynamic behavior. Monitoring such temporally-varying networks to detect anomalous changes has applications in both social and physical sciences. In this work, we perform an evaluation study of summary statistics for anomaly detection in temporally-evolving networks by incorporating principles from statistical process monitoring. In contrast to previous studies, we deliberately incorporate temporal auto-correlation in our study. Other considerations in our comprehensive assessment include types and duration of anomaly, model type, and sparsity in temporally-evolving networks. We conclude that summary statistics can be valuable tools for network monitoring and often perform better than more complicated statistics.

stat.AP↗

arXiv2020

Scalable Estimation of Epidemic Thresholds via Node Sampling

Infectious or contagious diseases can be transmitted from one person to another through social contact networks. In today's interconnected global society, such contagion processes can cause global public health hazards, as exemplified by the ongoing Covid-19 pandemic. It is therefore of great practical relevance to investigate the network trans-mission of contagious diseases from the perspective of statistical inference. An important and widely studied boundary condition for contagion processes over networks is the so-called epidemic threshold. The epidemic threshold plays a key role in determining whether a pathogen introduced into a social contact network will cause an epidemic or die out. In this paper, we investigate epidemic thresholds from the perspective of statistical network inference. We identify two major challenges that are caused by high computational and sampling complexity of the epidemic threshold. We develop two statistically accurate and computationally efficient approximation techniques to address these issues under the Chung-Lu modeling framework. The second approximation, which is based on random walk sampling, further enjoys the advantage of requiring data on a vanishingly small fraction of nodes. We establish theoretical guarantees for both methods and demonstrate their empirical superiority.

cs.SI↗

arXiv2020

Online Social Deception and Its Countermeasures for Trustworthy Cyberspace: A Survey

We are living in an era when online communication over social network services (SNSs) have become an indispensable part of people's everyday lives. As a consequence, online social deception (OSD) in SNSs has emerged as a serious threat in cyberspace, particularly for users vulnerable to such cyberattacks. Cyber attackers have exploited the sophisticated features of SNSs to carry out harmful OSD activities, such as financial fraud, privacy threat, or sexual/labor exploitation. Therefore, it is critical to understand OSD and develop effective countermeasures against OSD for building a trustworthy SNSs. In this paper, we conducted an extensive survey, covering (i) the multidisciplinary concepts of social deception; (ii) types of OSD attacks and their unique characteristics compared to other social network attacks and cybercrimes; (iii) comprehensive defense mechanisms embracing prevention, detection, and response (or mitigation) against OSD attacks along with their pros and cons; (iv) datasets/metrics used for validation and verification; and (v) legal and ethical concerns related to OSD research. Based on this survey, we provide insights into the effectiveness of countermeasures and the lessons from existing literature. We conclude this survey paper with an in-depth discussions on the limitations of the state-of-the-art and recommend future research directions in this area.

cs.CR↗

arXiv2019

Toward epidemic thresholds on temporal networks: a review and open questions

Epidemiological contact network models have emerged as an important tool in understanding and predicting the spread of infectious disease, due to their capacity to engage individual heterogeneity that may underlie essential dynamics of a particular host-pathogen system. Just as fundamental are the changes that real-world contact networks undergo over time, both independently of and in response to pathogen spreading. These dynamics play a central role in determining whether a disease will die out or become an epidemic within a population, known as the epidemic threshold. In this paper, we provide an overview of methods to predict the epidemic threshold for temporal contact network models, and discuss areas that remain unexplored.

physics.soc-ph↗

arXiv2019

Statistical Evaluation of Spectral Methods for Anomaly Detection in Networks

Monitoring of networks for anomaly detection has attracted a lot of attention in recent years especially with the rise of connected devices and social networks. This is of importance as anomaly detection could span a wide range of application, from detecting terrorist cells in counter-terrorism efforts to phishing attacks in social network circles. For this reason, numerous techniques for anomaly detection have been introduced. However, application of these techniques to more complex network models is hindered by various challenges such as the size of the network being investigated, how much apriori information is needed, the size of the anomalous graph, among others. A recent technique introduced by Miller et al, which relies on a spectral framework for anomaly detection, has the potential to address many of these challenges. In their discussion of the spectral framework, three algorithms were proposed that relied on the eigenvalues and eigenvectors of the residual matrix of a binary network. The authors demonstrated the ability to detect anomalous subgraphs that were less than 1% of the network size. However, to date, there is little work that has been done to evaluate the statistical performance of these algorithms. This study investigates the statistical properties of the spectral methods, specifically the Chi-square and L1 norm algorithm proposed by Miller. We will analyze the performance of the algorithm using simulated networks and also extend the method's application to count networks. Finally we will make some methodological improvements and recommendations to both algorithms.

stat.AP↗

arXiv2017

Application of Fractional Derivatives in Characterization of ECG graphs of Right Ventricular Hypertrophy Patients

There are many functions which are continuous everywhere but non-differentiable at some or all points such functions are termed as unreachable functions. Graphs representing such unreachable functions are called unreachable graphs. For example, ECG is such an unreachable graph. Classical calculus fails in their characterization as derivatives do not exist at the unreachable points. Such unreachable functions can be characterized by fractional calculus as fractional derivatives exist at those unreachable points where classical derivatives do not exist. Definition of fractional derivatives has been proposed by several mathematicians like Grunwald-Letinikov, Riemann-Liouville, Caputo, and Jumarie to develop the theory of fractional calculus. In this paper, we have used Jumarie type fractional derivative and consequently the phase transition (P.T.) which is the difference between left fractional derivative and right fractional derivatives to characterize those points. A comparative study has been done between normal ECG sample and problematic ECG sample (Right Ventricular Hypertrophy) by the help of the above mentioned mathematical tool.

q-bio.OT↗

arXiv2017

Characterization of Left Ventricular Hypertrophy via Fractional Derivatives

In this paper, we have used the concepts of the fractional derivative of rough curves to characterize ECG of LVH patients and compared the results with normal ECGs. In mathematical language, an ECG is a rough curve having Q, R, S points as non-differentiable points where classical derivatives do not exist but fractional derivatives exist. We have calculated both left and right modified Riemann-Liouville fractional derivatives and their differences termed as phase transition at those non-differentiable points of V1, V2, V5, and V6 leads.Investigation shows that phase transition is higher for LVH patients than normal ones. This may be a method of determination of risk factor of LVH patients before doing Echocardiogram.

physics.med-ph↗

arXiv2015

Analytical solution with tanh-method and fractional sub-equation method for non-linear partial differential equations and corresponding fractional differential equation composed with Jumarie fractional derivative

The solution of non-linear differential equation, non-linear partial differential equation and non-linear fractional differential equation is current research in Applied Science. Here tanh-method and Fractional Sub-Equation methods are used to solve three non-linear differential equations and the corresponding fractional differential equation. The fractional differential equations here are composed with Jumarie fractional derivative. Both the solution is obtained in analytical traveling wave solution form. We have not come across solutions of these equations reported anywhere earlier.

math.GM↗

arXiv2015

A subsampled double bootstrap for massive data

The bootstrap is a popular and powerful method for assessing precision of estimators and inferential methods. However, for massive datasets which are increasingly prevalent, the bootstrap becomes prohibitively costly in computation and its feasibility is questionable even with modern parallel computing platforms. Recently Kleiner, Talwalkar, Sarkar, and Jordan (2014) proposed a method called BLB (Bag of Little Bootstraps) for massive data which is more computationally scalable with little sacrifice of statistical accuracy. Building on BLB and the idea of fast double bootstrap, we propose a new resampling method, the subsampled double bootstrap, for both independent data and time series data. We establish consistency of the subsampled double bootstrap under mild conditions for both independent and dependent cases. Methodologically, the subsampled double bootstrap is superior to BLB in terms of running time, more sample coverage and automatic implementation with less tuning parameters for a given time budget. Its advantage relative to BLB and bootstrap is also demonstrated in numerical simulations and a data illustration.

stat.ME↗

arXiv2015

Analytic Solution of Linear Fractional Differential Equation with Jumarie Derivative in Term of Mittag-Leffler Function

There is no unified method to solve the fractional differential equation. The type of derivative here used in this paper is of Jumarie formulation, for the several differential equations studied. Here we develop an algorithm to solve the linear fractional differential equation composed via Jumarie fractional derivative in terms of Mittag-Leffler function; and show its conjugation with ordinary calculus. In these fractional differential equations the one parameter Mittag-Leffler function plays the role similar as exponential function used in ordinary differential equations.

math.CA↗

arXiv2015

Characterization of non-differentiable points in a function by Fractional derivative of Jumarrie type

There are many functions which are continuous everywhere but not differentiable at some points, like in physical systems of ECG, EEG plots, and cracks pattern and for several other phenomena. Using classical calculus those functions cannot be characterized-especially at the non-differentiable points. To characterize those functions the concept of Fractional Derivative is used. From the analysis it is established that though those functions are unreachable at the non-differentiable points, in classical sense but can be characterized using Fractional derivative. In this paper we demonstrate use of modified Riemann-Liouvelli derivative by Jumarrie to calculate the fractional derivatives of the non-differentiable points of a function, which may be one step to characterize and distinguish and compare several non-differentiable points in a system or across the systems. This method we are extending to differentiate various ECG graphs by quantification of non-differentiable points; is useful method in differential diagnostic. Each steps of calculating these fractional derivatives is elaborated.

math.CA↗