arXiv subjects

Saikat Chatterjee

Publications and source records attributed to Saikat Chatterjee.

At least 73 records · Page 4Linked to original sources

arXiv2018

Supervised Linear Regression for Graph Learning from Graph Signals

We propose a supervised learning approach for predicting an underlying graph from a set of graph signals. Our approach is based on linear regression. In the linear regression model, we predict edge-weights of a graph as the output, given a set of signal values on nodes of the graph as the input. We solve for the optimal regression coefficients using a relevant optimization problem that is convex and uses a graph-Laplacian based regularization. The regularization helps to promote a specific graph spectral profile of the graph signals. Simulation experiments demonstrate that our approach predicts well even in presence of outliers in input data.

cs.IT↗

arXiv2018

Locally Convex Sparse Learning over Networks

We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum solution for that node. The nodes exchange their signal estimates over the network in order to refine their local estimates. At a node, the optimization algorithm is based on an $\ell_1$-norm minimization with appropriate modifications to promote sparsity as well as to include influence of estimates from neighboring nodes. Our expectation is that local estimates in each node improve fast and converge, resulting in a limited demand for communication of estimates between nodes and reducing the processing time. We provide restricted-isometry-property (RIP)-based theoretical analysis on estimation quality. In the scenario of clean observation, it is shown that the local estimates converge to the exact sparse signal under certain technical conditions. Simulation results show that the proposed algorithms show competitive performance compared to a globally optimum distributed LASSO algorithm in the sense of convergence speed and estimation error.

stat.ML↗

arXiv2018

Gaussian Processes Over Graphs

We propose Gaussian processes for signals over graphs (GPG) using the apriori knowledge that the target vectors lie over a graph. We incorporate this information using a graph- Laplacian based regularization which enforces the target vectors to have a specific profile in terms of graph Fourier transform coeffcients, for example lowpass or bandpass graph signals. We discuss how the regularization affects the mean and the variance in the prediction output. In particular, we prove that the predictive variance of the GPG is strictly smaller than the conventional Gaussian process (GP) for any non-trivial graph. We validate our concepts by application to various real-world graph signals. Our experiments show that the performance of the GPG is superior to GP for small training data sizes and under noisy training.

stat.ML↗

arXiv2018

R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network

We consider a neural network architecture with randomized features, a sign-splitter, followed by rectified linear units (ReLU). We prove that our architecture exhibits robustness to the input perturbation: the output feature of the neural network exhibits a Lipschitz continuity in terms of the input perturbation. We further show that the network output exhibits a discrimination ability that inputs that are not arbitrarily close generate output vectors which maintain distance between each other obeying a certain lower bound. This ensures that two different inputs remain discriminable while contracting the distance in the output feature space.

stat.ML↗

arXiv2018

Extreme Learning Machine for Graph Signal Processing

In this article, we improve extreme learning machines for regression tasks using a graph signal processing based regularization. We assume that the target signal for prediction or regression is a graph signal. With this assumption, we use the regularization to enforce that the output of an extreme learning machine is smooth over a given graph. Simulation results with real data confirm that such regularization helps significantly when the available training data is limited in size and corrupted by noise.

stat.ML↗

arXiv2018

Multi-kernel Regression For Graph Signal Processing

We develop a multi-kernel based regression method for graph signal processing where the target signal is assumed to be smooth over a graph. In multi-kernel regression, an effective kernel function is expressed as a linear combination of many basis kernel functions. We estimate the linear weights to learn the effective kernel function by appropriate regularization based on graph smoothness. We show that the resulting optimization problem is shown to be convex and pro- pose an accelerated projected gradient descent based solution. Simulation results using real-world graph signals show efficiency of the multi-kernel based approach over a standard kernel based approach.

stat.ML↗

arXiv2018

Hilbert Transform, Analytic Signal, and Modulation Analysis for Graph Signal Processing

We propose Hilbert transform (HT) and analytic signal (AS) construction for signals over graphs. This is motivated by the popularity of HT, AS, and modulation analysis in conventional signal processing, and the observation that complementary insight is often obtained by viewing conventional signals in the graph setting. Our definitions of HT and AS use a conjugate-symmetry-like property exhibited by the graph Fourier transform (GFT). We show that a real graph signal (GS) can be represented using smaller number of GFT coefficients than the signal length. We show that the graph HT (GHT) and graph AS (GAS) operations are linear and shift-invariant over graphs. Using the GAS, we define the amplitude, phase, and frequency modulations for a graph signal (GS). Further, we use convex optimization to develop an alternative definition of envelope for a GS. We illustrate the proposed concepts by showing applications to synthesized and real-world signals. For example, we show that the GHT is suitable for anomaly detection/analysis over networks and that GAS reveals complementary information in speech signals.

cs.IT↗

arXiv2017

Large Neural Network Based Detection of Apnea, Bradycardia and Desaturation Events

Apnea, bradycardia and desaturation (ABD) events often precede life-threatening events including sepsis in newborn babies. Here, we explore machine learning for detection of ABD events as a binary classification problem. We investigate the use of a large neural network to achieve a good detection performance. To be user friendly, the chosen neural network does not require a high level of parameter tuning. Furthermore, a limited amount of training data is available and the training dataset is unbalanced. Comparing with two widely used state-of-the-art machine learning algorithms, the large neural network is found to be efficient. Even with a limited and unbalanced training data, the large neural network provides a detection performance level that is feasible to use in clinical care.

cs.LG↗

arXiv2017

Progressive Learning for Systematic Design of Large Neural Networks

We develop an algorithm for systematic design of a large artificial neural network using a progression property. We find that some non-linear functions, such as the rectifier linear unit and its derivatives, hold the property. The systematic design addresses the choice of network size and regularization of parameters. The number of nodes and layers in network increases in progression with the objective of consistently reducing an appropriate cost. Each layer is optimized at a time, where appropriate parameters are learned using convex optimization. Regularization parameters for convex optimization do not need a significant manual effort for tuning. We also use random instances for some weight matrices, and that helps to reduce the number of parameters we learn. The developed network is expected to show good generalization power due to appropriate regularization and use of random weights in the layers. This expectation is verified by extensive experiments for classification and regression problems, using standard databases.

cs.NE↗

arXiv2017

The inner mass power spectrum of galaxies using strong gravitational lensing: beyond linear approximation

In the last decade the detection of individual massive dark matter sub-halos has been possible using potential correction formalism in strong gravitational lens imaging. Here we propose a statistical formalism to relate strong gravitational lens surface brightness anomalies to the lens potential fluctuations arising from dark matter distribution in the lens galaxy. We consider these fluctuations as a Gaussian random field in addition to the unperturbed smooth lens model. This is very similar to weak lensing formalism and we show that in this way we can measure the power spectrum of these perturbations to the potential. We test the method by applying it to simulated mock lenses of different geometries and by performing an MCMC analysis of the theoretical power spectra. This method can measure density fluctuations in early type galaxies on scales of 1-10 kpc at typical rms-levels of a percent, using a single lens system observed with the Hubble Space Telescope with typical signal-to-noise ratios obtained in a single orbit.

astro-ph.CO↗

arXiv2017

Estimate Exchange over Network is Good for Distributed Hard Thresholding Pursuit

We investigate an existing distributed algorithm for learning sparse signals or data over networks. The algorithm is iterative and exchanges intermediate estimates of a sparse signal over a network. This learning strategy using exchange of intermediate estimates over the network requires a limited communication overhead for information transmission. Our objective in this article is to show that the strategy is good for learning in spite of limited communication. In pursuit of this objective, we first provide a restricted isometry property (RIP)-based theoretical analysis on convergence of the iterative algorithm. Then, using simulations, we show that the algorithm provides competitive performance in learning sparse signals vis-a-vis an existing alternate distributed algorithm. The alternate distributed algorithm exchanges more information including observations and system parameters.

stat.ML↗

arXiv2017

A Connectedness Constraint for Learning Sparse Graphs

Graphs are naturally sparse objects that are used to study many problems involving networks, for example, distributed learning and graph signal processing. In some cases, the graph is not given, but must be learned from the problem and available data. Often it is desirable to learn sparse graphs. However, making a graph highly sparse can split the graph into several disconnected components, leading to several separate networks. The main difficulty is that connectedness is often treated as a combinatorial property, making it hard to enforce in e.g. convex optimization problems. In this article, we show how connectedness of undirected graphs can be formulated as an analytical property and can be enforced as a convex constraint. We especially show how the constraint relates to the distributed consensus problem and graph Laplacian learning. Using simulated and real data, we perform experiments to learn sparse and connected graphs from data.

stat.ML↗

arXiv2016

Categorified presheaves and sieves

Let $\mathcal C$ be a category of a set of (small) categories. This paper concerns with the ${\mathbf {Cat}}$-valued presheaves and sieves over category $\mathcal C.$ Since ${\mathbf {Cat}}$ is not a concrete category, existing definition of presheaves can not deal with the situation. This paper proposes a new framework for the purpose. The main result is a version of Yoneda embedding for ${\mathbf {Cat}}$-valued presheaves, and construction of the ${\mathbf {Cat}}$-valued sieves over the category ${\mathcal O}(\mathbf B)$ of subcategories of a given topological category $\mathbf B.$

math.CT↗

arXiv2016

On $\mathbf {Cat}$-valued sheaves

Let ${\widetilde {\mathcal O}}(\mathbf B)$ be the category of (open) subcategories of a topological groupoid ${\mathbf B}.$ This paper concerns with the ${\mathbf {Cat}}$-valued sheaves over category ${\widetilde {\mathcal O}}(\mathbf B).$ Since ${\mathbf {Cat}}$ is not a concrete category, traditional definition of presheaf can not deal with the situation. [13] proposes a new framework for the purpose. Starting from the definition given in [13], we build-up the frame work for ${\mathbf {Cat}}$-valued sheaves. For that purpose we introduce a notion of categorical union, such that categorical union of subcategories is a subcategory, which is required for a meaningful definition of a categorical cover of a topological category. The main result is the following. For a fixed category $\mathbf C,$ the categories of local functorial sections from $\mathbf B$ to $\mathbf C$ define a ${\mathbf {Cat}}$-valued sheaf on ${\widetilde {\mathcal O}}(\mathbf B).$ Replacing $\mathbf C$ with a categorical group $\mathcal G,$ we find a ${\mathbf {CatGrp}}$-valued sheaf on ${\widetilde {\mathcal O}}(\mathbf B).$

math.CT↗

arXiv2016

Construction of Categorical Bundles from Local Data

A categorical principal bundle is a structure comprised of categories that is analogous to a classical principal bundle; examples arise from geometric contexts involving bundles over path spaces. We show how a categorical principal bundle can be constructed from local data specified through transition functors and natural transformations.

math.CT↗

arXiv2016

Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

Performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric and so-called T-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on l1-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to l2-norm and "zero-norm" regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns the T-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

cs.IT↗

arXiv2015

Geodesics on path spaces and double category

Let $M$ be a Riemannian manifold and ${\mathcal P}M$ be the space of all smooth paths on $M$. We describe geodesics on path space ${\mathcal P}M$. Normal neighbourhood structure on ${\mathcal P}M$ has been discussed. We identify paths on $M$ under "back-track" equivalence. Under this identification we show that if $M$ is complete, then geodesics on path space yield a double category.We gave a physical interpretation of this double category.

math.DG↗

arXiv2015

Twisted-Product Categorical Bundles

Categorical bundles provide a natural framework for gauge theories involving multiple gauge groups. Unlike the case of traditional bundles there are distinct notions of triviality, and hence also of local triviality, for categorical bundles. We study categorical principal bundles that are product bundles in the categorical sense, developing the relationship between functorial sections of such bundles and trivializations. We construct functorial cocycles with values in categorical groups using a suitable family of locally defined functions on the object space of the base category. Categorical product bundles being too rigid to give a widely applicable model for local triviality, we introduce the notion of a twisted-product categorical bundle. We relate such bundles to decorated categorical bundles that contain more information, specifically parallel transport data.

math.DG↗