arXiv subjects

Sayan Mukherjee

Publications and source records attributed to Sayan Mukherjee.

At least 55 records · Page 3Linked to original sources

arXiv2021

A Methodology for Exploring Deep Convolutional Features in Relation to Hand-Crafted Features with an Application to Music Audio Modeling

Understanding the features learned by deep models is important from a model trust perspective, especially as deep systems are deployed in the real world. Most recent approaches for deep feature understanding or model explanation focus on highlighting input data features that are relevant for classification decisions. In this work, we instead take the perspective of relating deep features to well-studied, hand-crafted features that are meaningful for the application of interest. We propose a methodology and set of systematic experiments for exploring deep features in this setting, where input feature importance approaches for deep feature understanding do not apply. Our experiments focus on understanding which hand-crafted and deep features are useful for the classification task of interest, how robust these features are for related tasks and how similar the deep features are to the meaningful hand-crafted features. Our proposed method is general to many application areas and we demonstrate its utility on orchestral music audio data.

cs.SD↗

arXiv2021

How Many Directions Determine a Shape and other Sufficiency Results for Two Topological Transforms

In this paper we consider two topological transforms that are popular in applied topology: the Persistent Homology Transform (PHT) and the Euler Characteristic Transform (ECT). Both of these transforms are of interest for their mathematical properties as well as their applications to science and engineering, because they provide a way of summarizing shapes in a topological, yet quantitative, way. Both transforms take a shape, viewed as a tame subset $M$ of $\mathbb{R}^d$, and associates to each direction $v\in S^{d-1}$ a shape summary obtained by scanning $M$ in the direction $v$. These shape summaries are either persistence diagrams or piecewise constant integer-valued functions called Euler curves. By using an inversion theorem of Schapira, we show that both transforms are injective on the space of shapes, i.e.~each shape has a unique transform. Moreover, we prove that these transforms determine continuous maps from the sphere to the space of persistence diagrams, equipped with any Wasserstein $p$-distance, or the space of Euler curves, equipped with certain $L^p$ norms. By making use of a stratified space structure on the sphere, induced by hyperplane divisions, we prove additional uniqueness results in terms of distributions on the space of Euler curves. Finally, our main result proves that any shape in a certain uncountable space of PL embedded shapes with plausible geometric bounds can be uniquely determined using only finitely many directions.

math.AT↗

arXiv2021

Concerning three classes of non-Diophantine arithmetics

We present three classes of abstract prearithmetics, $\{\mathbf{A}_M\}_{M \geq 1}$, $\{\mathbf{A}_{-M,M}\}_{M \geq 1}$, and $\{\mathbf{B}_M\}_{M > 0}$. The first one is weakly projective with respect to the nonnegative real Diophantine arithmetic $\mathbf{R_+}=(\mathbb{R}_+,+,\times,\leq_{\mathbb{R}_+})$, the second one is weakly projective with respect to the real Diophantine arithmetic $\mathbf{R}=(\mathbb{R},+,\times,\leq_{\mathbb{R}})$, while the third one is projective with respect to the extended real Diophantine arithmetic $\overline{\mathbf{R}}=(\overline{\mathbb{R}},+,\times,\leq_{\overline{\mathbb{R}}})$. In addition, we have that every $\mathbf{A}_M$ and every $\mathbf{B}_M$ are a complete totally ordered semiring, while every $\mathbf{A}_{-M,M}$ is not. We show that the projection of any series of elements of $\mathbb{R}_+$ converges in $\mathbf{A}_M$, for any $M \geq 1$, and that the projection of any non-oscillating series series of elements of $\mathbb{R}$ converges in $\mathbf{A}_{-M,M}$, for any $M \geq 1$, and in $\mathbf{B}_M$, for all $M > 0$. We also prove that working in $\mathbf{A}_M$ and in $\mathbf{A}_{-M,M}$, for any $M \geq 1$, and in $\mathbf{B}_M$, for all $M>0$, allows to overcome a version of the paradox of the heap.

math.LO↗

arXiv2021

Accelerating Markov Random Field Inference with Uncertainty Quantification

Statistical machine learning has widespread application in various domains. These methods include probabilistic algorithms, such as Markov Chain Monte-Carlo (MCMC), which rely on generating random numbers from probability distributions. These algorithms are computationally expensive on conventional processors, yet their statistical properties, namely interpretability and uncertainty quantification (UQ) compared to deep learning, make them an attractive alternative approach. Therefore, hardware specialization can be adopted to address the shortcomings of conventional processors in running these applications. In this paper, we propose a high-throughput accelerator for Markov Random Field (MRF) inference, a powerful model for representing a wide range of applications, using MCMC with Gibbs sampling. We propose a tiled architecture which takes advantage of near-memory computing, and memory optimizations tailored to the semantics of MRF. Additionally, we propose a novel hybrid on-chip/off-chip memory system and logging scheme to efficiently support UQ. This memory system design is not specific to MRF models and is applicable to applications using probabilistic algorithms. In addition, it dramatically reduces off-chip memory bandwidth requirements. We implemented an FPGA prototype of our proposed architecture using high-level synthesis tools and achieved 146MHz frequency for an accelerator with 32 function units on an Intel Arria 10 FPGA. Compared to prior work on FPGA, our accelerator achieves 26X speedup. Furthermore, our proposed memory system and logging scheme to support UQ reduces off-chip bandwidth by 71% for two applications. ASIC analysis in 15nm shows our design with 2048 function units running at 3GHz outperforms GPU implementations of motion estimation and stereo vision on Nvidia RTX2080Ti by 120X-210X, occupying only 7.7% of the area.

cs.AR↗

arXiv2021

Stability theorems for some Kruskal-Katona type results

The classical Kruskal-Katona theorem gives a tight upper bound for the size of an $r$-uniform hypergraph $\mathcal{H}$ as a function of the size of its shadow. Its stability version was obtained by Keevash who proved that if the size of $\mathcal{H}$ is close to the maximum, then $\mathcal{H}$ is structurally close to a complete $r$-uniform hypergraph. We prove similar stability results for two classes of hypergraphs whose extremal properties have been investigated by many researchers: the cancellative hypergraphs and hypergraphs without expansion of cliques.

math.CO↗

arXiv2021

A study on dynamics and multiscale complexity of a neuro system

We explore the chaotic dynamics and complexity of a neuro-system with respect to variable synaptic weights in both noise free and noisy conditions. The chaotic dynamics of the system is investigated by bifurcation analysis and 0-1 test. A multiscale complexity of the system is proposed based on the notion of recurrence plot density entropy. Numerical results support the proposed analysis. Impact of music on the aforesaid neuro-system has also been studied. The analysis shows that inclusion of white noise even with a minimal strength makes the neuro dynamics more complex, where as music signal keeps the dynamics almost similar to that of the original system. This is properly interpreted by the proposed multiscale complexity measure.

nlin.CD↗

arXiv2021

Exploring noise-induced chaos and complexity in a red blood cell system

We investigate dynamical changes and its corresponding phase space complexity in a stochastic red blood cell system. The system is obtained by incorporating power noise with the associated sinusoidal flow. Both chaotic and non-chaotic dynamics of sinusoidal flow in red blood cell are identified by 0-1 test. Furthermore, dynamical complexity of the sinusoidal flow in the system is investigated by heterogeneous recurrence based entropy. The numerical simulation is performed to quantify the existence of chaotic dynamics and complexity for the sinusoidal blood flow.

nlin.CD↗

arXiv2020

Reachability for Updatable Timed Automata made faster and more effective

Updatable timed automata (UTA) are extensions of classic timed automata that allow special updates to clock variables, like x:= x - 1, x := y + 2, etc., on transitions. Reachability for UTA is undecidable in general. Various subclasses with decidable reachability have been studied. A generic approach to UTA reachability consists of two phases: first, a static analysis of the automaton is performed to compute a set of clock constraints at each state; in the second phase, reachable sets of configurations, called zones, are enumerated. In this work, we improve the algorithm for the static analysis. Compared to the existing algorithm, our method computes smaller sets of constraints and guarantees termination for more UTA, making reachability faster and more effective. As the main application, we get an alternate proof of decidability and a more efficient algorithm for timed automata with bounded subtraction, a class of UTA widely used for modelling scheduling problems. We have implemented our procedure in the tool TChecker and conducted experiments that validate the benefits of our approach.

cs.LO↗

arXiv2020

Random Lie Brackets that Induce Torsion: A Model for Noisy Vector Fields

We define and study a random Lie bracket that induces torsion in expectation. Almost all stochastic analysis on manifolds have assumed parallel transport. Mathematically this assumption is very reasonable. However, in many applied geometry and graphics problems parallel transport is not achieved, the "change in coordinates" are not exact due to noise. We formulate a stochastic model on a manifold for which parallel transport does not hold and analyze the consequences of this model with respect to classic quantities studied in Riemannian geometry. We first define a stochastic lie bracket that induces a stochastic covariant derivative. We then study the connection implied by the stochastic covariant derivative and note that the stochastic lie bracket induces torsion. We then state the induced stochastic geodesic equations and a stochastic differential equation for parallel transport. We also derive the curvature tensors for our construction and a stochastic Laplace-Beltrami operator. We close with a discussion of the motivation and relevance of our construction.

math.PR↗

arXiv2020

Subspace Clustering through Sub-Clusters

The problem of dimension reduction is of increasing importance in modern data analysis. In this paper, we consider modeling the collection of points in a high dimensional space as a union of low dimensional subspaces. In particular we propose a highly scalable sampling based algorithm that clusters the entire data via first spectral clustering of a small random sample followed by classifying or labeling the remaining out of sample points. The key idea is that this random subset borrows information across the entire data set and that the problem of clustering points can be replaced with the more efficient and robust problem of "clustering sub-clusters". We provide theoretical guarantees for our procedure. The numerical results indicate we outperform other state-of-the-art subspace clustering algorithms with respect to accuracy and speed.

stat.ML↗

arXiv2020

Stanza: A Nonlinear State Space Model for Probabilistic Inference in Non-Stationary Time Series

Time series with long-term structure arise in a variety of contexts and capturing this temporal structure is a critical challenge in time series analysis for both inference and forecasting settings. Traditionally, state space models have been successful in providing uncertainty estimates of trajectories in the latent space. More recently, deep learning, attention-based approaches have achieved state of the art performance for sequence modeling, though often require large amounts of data and parameters to do so. We propose Stanza, a nonlinear, non-stationary state space model as an intermediate approach to fill the gap between traditional models and modern deep learning approaches for complex time series. Stanza strikes a balance between competitive forecasting accuracy and probabilistic, interpretable inference for highly structured time series. In particular, Stanza achieves forecasting accuracy competitive with deep LSTMs on real-world datasets, especially for multi-step ahead forecasting.

stat.ML↗

arXiv2020

A Case for Quantifying Statistical Robustness of Specialized Probabilistic AI Accelerators

Statistical machine learning often uses probabilistic algorithms, such as Markov Chain Monte Carlo (MCMC), to solve a wide range of problems. Many accelerators are proposed using specialized hardware to address sampling inefficiency, the critical performance bottleneck of probabilistic algorithms. These accelerators usually improve the hardware efficiency by using some approximation techniques, such as reducing bit representation, truncating small values to zero, or simplifying the Random Number Generator (RNG). Understanding the influence of these approximations on result quality is crucial to meeting the quality requirements of real applications. Although a common approach is to compare the end-point result quality using community-standard benchmarks and metrics, we claim a probabilistic architecture should provide some measure (or guarantee) of statistical robustness. This work takes a first step towards quantifying the statistical robustness of specialized hardware MCMC accelerators by proposing three pillars of statistical robustness: sampling quality, convergence diagnostic, and goodness of fit. Each pillar has at least one quantitative metric without the need to know the ground truth data. We apply this method to analyze the statistical robustness of an MCMC accelerator proposed by previous work, with some modifications, as a case study. The method also applies to other probabilistic accelerators and can be used in design space exploration.

eess.SP↗

arXiv2020

Beyond Application End-Point Results: Quantifying Statistical Robustness of MCMC Accelerators

Statistical machine learning often uses probabilistic algorithms, such as Markov Chain Monte Carlo (MCMC), to solve a wide range of problems. Probabilistic computations, often considered too slow on conventional processors, can be accelerated with specialized hardware by exploiting parallelism and optimizing the design using various approximation techniques. Current methodologies for evaluating correctness of probabilistic accelerators are often incomplete, mostly focusing only on end-point result quality ("accuracy"). It is important for hardware designers and domain experts to look beyond end-point "accuracy" and be aware of the hardware optimizations impact on other statistical properties. This work takes a first step towards defining metrics and a methodology for quantitatively evaluating correctness of probabilistic accelerators beyond end-point result quality. We propose three pillars of statistical robustness: 1) sampling quality, 2) convergence diagnostic, and 3) goodness of fit. We apply our framework to a representative MCMC accelerator and surface design issues that cannot be exposed using only application end-point result quality. Applying the framework to guide design space exploration shows that statistical robustness comparable to floating-point software can be achieved by slightly increasing the bit representation, without floating-point hardware requirements.

eess.SP↗

arXiv2019

Bayesian Non-Parametric Factor Analysis for Longitudinal Spatial Surfaces

We introduce a Bayesian non-parametric spatial factor analysis model with spatial dependency induced through a prior on factor loadings. For each column of the loadings matrix, spatial dependency is encoded using a probit stick-breaking process (PSBP) and a multiplicative gamma process shrinkage prior is used across columns to adaptively determine the number of latent factors. By encoding spatial information into the loadings matrix, meaningful factors are learned that respect the observed neighborhood dependencies, making them useful for assessing rates over space. Furthermore, the spatial PSBP prior can be used for clustering temporal trends, allowing users to identify regions within the spatial domain with similar temporal trajectories, an important task in many applied settings. In the manuscript, we illustrate the model's performance in simulated data, but also in two real-world examples: longitudinal monitoring of glaucoma and malaria surveillance across the Peruvian Amazon. The R package spBFA, available on CRAN, implements the method.

stat.ME↗

arXiv2019

Predicting Clinical Outcomes in Glioblastoma: An Application of Topological and Functional Data Analysis

Glioblastoma multiforme (GBM) is an aggressive form of human brain cancer that is under active study in the field of cancer biology. Its rapid progression and the relative time cost of obtaining molecular data make other readily-available forms of data, such as images, an important resource for actionable measures in patients. Our goal is to utilize information given by medical images taken from GBM patients in statistical settings. To do this, we design a novel statistic---the smooth Euler characteristic transform (SECT)---that quantifies magnetic resonance images (MRIs) of tumors. Due to its well-defined inner product structure, the SECT can be used in a wider range of functional and nonparametric modeling approaches than other previously proposed topological summary statistics. When applied to a cohort of GBM patients, we find that the SECT is a better predictor of clinical outcomes than both existing tumor shape quantifications and common molecular assays. Specifically, we demonstrate that SECT features alone explain more of the variance in GBM patient survival than gene expression, volumetric features, and morphometric features. The main takeaways from our findings are thus twofold. First, they suggest that images contain valuable information that can play an important role in clinical prognosis and other medical decisions. Second, they show that the SECT is a viable tool for the broader study of medical imaging informatics.

stat.AP↗

arXiv2019

Scalable Modeling of Spatiotemporal Data using the Variational Autoencoder: an Application in Glaucoma

As big spatial data becomes increasingly prevalent, classical spatiotemporal (ST) methods often do not scale well. While methods have been developed to account for high-dimensional spatial objects, the setting where there are exceedingly large samples of spatial observations has had less attention. The variational autoencoder (VAE), an unsupervised generative model based on deep learning and approximate Bayesian inference, fills this void using a latent variable specification that is inferred jointly across the large number of samples. In this manuscript, we compare the performance of the VAE with a more classical ST method when analyzing longitudinal visual fields from a large cohort of patients in a prospective glaucoma study. Through simulation and a case study, we demonstrate that the VAE is a scalable method for analyzing ST data, when the goal is to obtain accurate predictions. R code to implement the VAE can be found on GitHub: https://github.com/berchuck/vaeST.

stat.AP↗

arXiv2019

Adaptive particle-based approximations of the Gibbs posterior for inverse problems

In this work, we adopt a general framework based on the Gibbs posterior to update belief distributions for inverse problems governed by partial differential equations (PDEs). The Gibbs posterior formulation is a generalization of standard Bayesian inference that only relies on a loss function connecting the unknown parameters to the data. It is particularly useful when the true data generating mechanism (or noise distribution) is unknown or difficult to specify. The Gibbs posterior coincides with Bayesian updating when a true likelihood function is known and the loss function corresponds to the negative log-likelihood, yet provides subjective inference in more general settings. We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner. We demonstrate the efficiency of our approach through several numerical examples.

stat.CO↗

arXiv2019

The Geometry of Synchronization Problems and Learning Group Actions

We develop a geometric framework that characterizes the synchronization problem --- the problem of consistently registering or aligning a collection of objects. The theory we formulate characterizes the cohomological nature of synchronization based on the classical theory of fibre bundles. We first establish the correspondence between synchronization problems in a topological group $G$ over a connected graph $Γ$ and the moduli space of flat principal $G$-bundles over $Γ$, and develop a discrete analogy of the renowned theorem of classifying flat principal bundles with fix base and structural group using the representation variety. In particular, we show that prescribing an edge potential on a graph is equivalent to specifying an equivalence class of flat principal bundles, of which the triviality of holonomy dictates the synchronizability of the edge potential. We then develop a twisted cohomology theory for associated vector bundles of the flat principal bundle arising from an edge potential, which is a discrete version of the twisted cohomology in differential geometry. This theory realizes the obstruction to synchronizability as a cohomology group of the twisted de Rham cochain complex. We then build a discrete twisted Hodge theory --- a fibre bundle analog of the discrete Hodge theory on graphs --- which geometrically realizes the graph connection Laplacian as a Hodge Laplacian of degree zero. Motivated by our geometric framework, we study the problem of learning group actions --- partitioning a collection of objects based on the local synchronizability of pairwise correspondence relations. A dual interpretation is to learn finitely generated subgroups of an ambient transformation group from noisy observed group elements. A synchronization-based algorithm is also provided, and we demonstrate its efficacy using simulations and real data.

math.ST↗