arXiv subjects

Mark Coates

Publications and source records attributed to Mark Coates.

At least 91 records · Page 5Linked to original sources

arXiv2019

Bayesian Graph Convolutional Neural Networks Using Non-Parametric Graph Learning

Graph convolutional neural networks (GCNN) have been successfully applied to many different graph based learning tasks including node and graph classification, matrix completion, and learning of node embeddings. Despite their impressive performance, the techniques have a limited capability to incorporate the uncertainty in the underlined graph structure. In order to address this issue, a Bayesian GCNN (BGCN) framework was recently proposed. In this framework, the observed graph is considered to be a random realization from a parametric random graph model and the joint Bayesian inference of the graph and GCNN weights is performed. In this paper, we propose a non-parametric generative model for graphs and incorporate it within the BGCN framework. In addition to the observed graph, our approach effectively uses the node features and training labels in the posterior inference of graphs and attains superior or comparable performance in benchmark node classification tasks.

cs.LG↗

arXiv2019

Invertible Particle Flow-based Sequential MCMC with extension to Gaussian Mixture noise models

Sequential state estimation in non-linear and non-Gaussian state spaces has a wide range of applications in statistics and signal processing. One of the most effective non-linear filtering approaches, particle filtering, suffers from weight degeneracy in high-dimensional filtering scenarios. Several avenues have been pursued to address high-dimensionality. Among these, particle flow particle filters construct effective proposal distributions by using invertible flow to migrate particles continuously from the prior distribution to the posterior, and sequential Markov chain Monte Carlo (SMCMC) methods use a Metropolis-Hastings (MH) accept-reject approach to improve filtering performance. In this paper, we propose to combine the strengths of invertible particle flow and SMCMC by constructing a composite Metropolis-Hastings (MH) kernel within the SMCMC framework using invertible particle flow. In addition, we propose a Gaussian mixture model (GMM)-based particle flow algorithm to construct effective MH kernels for multi-modal distributions. Simulation results show that for high-dimensional state estimation example problems the proposed kernels significantly increase the acceptance rate with minimal additional computational overhead and improve estimation accuracy compared with state-of-the-art filtering algorithms.

stat.ME↗

arXiv2018

Bayesian graph convolutional neural networks for semi-supervised classification

Recently, techniques for applying convolutional neural networks to graph-structured data have emerged. Graph convolutional neural networks (GCNNs) have been used to address node and graph classification and matrix completion. Although the performance has been impressive, the current implementations have limited capability to incorporate uncertainty in the graph structure. Almost all GCNNs process a graph as though it is a ground-truth depiction of the relationship between nodes, but often the graphs employed in applications are themselves derived from noisy data or modelling assumptions. Spurious edges may be included; other edges may be missing between nodes that have very strong relationships. In this paper we adopt a Bayesian approach, viewing the observed graph as a realization from a parametric family of random graphs. We then target inference of the joint posterior of the random graph parameters and the node (or graph) labels. We present the Bayesian GCNN framework and develop an iterative learning procedure for the case of assortative mixed-membership stochastic block models. We present the results of experiments that demonstrate that the Bayesian formulation can provide better performance when there are very few labels available during the training process.

stat.ML↗

arXiv2018

Cost Adaptation for Robust Decentralized Swarm Behaviour

Decentralized receding horizon control (D-RHC) provides a mechanism for coordination in multi-agent settings without a centralized command center. However, combining a set of different goals, costs, and constraints to form an efficient optimization objective for D-RHC can be difficult. To allay this problem, we use a meta-learning process -- cost adaptation -- which generates the optimization objective for D-RHC to solve based on a set of human-generated priors (cost and constraint functions) and an auxiliary heuristic. We use this adaptive D-RHC method for control of mesh-networked swarm agents. This formulation allows a wide range of tasks to be encoded and can account for network delays, heterogeneous capabilities, and increasingly large swarms through the adaptation mechanism. We leverage the Unity3D game engine to build a simulator capable of introducing artificial networking failures and delays in the swarm. Using the simulator we validate our method on an example coordinated exploration task. We demonstrate that cost adaptation allows for more efficient and safer task completion under varying environment conditions and increasingly large swarm sizes. We release our simulator and code to the community for future work.

cs.AI↗

arXiv2017

A multi-sensor multi-Bernoulli filter

In this paper we derive a multi-sensor multi-Bernoulli (MS-MeMBer) filter for multi-target tracking. Measurements from multiple sensors are employed by the proposed filter to update a set of tracks modeled as a multi-Bernoulli random finite set. An exact implementation of the MS-MeMBer update procedure is computationally intractable. We propose an efficient approximate implementation by using a greedy measurement partitioning mechanism. The proposed filter allows for Gaussian mixture or particle filter implementations. Numerical simulations conducted for both linear-Gaussian and non-linear models highlight the improved accuracy of the MS-MeMBer filter and its reduced computational load with respect to the multi-sensor cardinalized probability hypothesis density filter and the iterated-corrector cardinality-balanced multi-Bernoulli filter especially for low probabilities of detection.

stat.ME↗

arXiv2017

Particle Filtering with Invertible Particle Flow

A key challenge when designing particle filters in high-dimensional state spaces is the construction of a proposal distribution that is close to the posterior distribution. Recent advances in particle flow filters provide a promising avenue to avoid weight degeneracy; particles drawn from the prior distribution are migrated in the state-space to the posterior distribution by solving partial differential equations. Numerous particle flow filters have been proposed based on different assumptions concerning the flow dynamics. Approximations are needed in the implementation of all of these filters; as a result the articles do not exactly match a sample drawn from the desired posterior distribution. Past efforts to correct the discrepancies involve expensive calculations of importance weights. In this paper, we present new filters which incorporate deterministic particle flows into an encompassing particle filter framework. The valuable theoretical guarantees concerning particle filter performance still apply, but we can exploit the attractive performance of the particle flow methods. The filters we describe involve a computationally efficient weight update step, arising because the embedded particle flows we design possess an invertible mapping property. We evaluate the proposed particle flow particle filters' performance through numerical simulations of a challenging multi-target multi-sensor tracking scenario and complex high-dimensional filtering examples.

stat.ME↗

arXiv2017

Online Bayesian Inference of Diffusion Networks

Understanding the process by which a contagion disseminates throughout a network is of great importance in many real world applications. The required sophistication of the inference approach depends on the type of information we want to extract as well as the number of observations that are available to us. We analyze scenarios in which not only the underlying network structure (parental relationships and link strengths) needs to be detected, but also the infection times must be estimated. We assume that our only observation of the diffusion process is a set of time series, one for each node of the network, which exhibit changepoints when an infection occurs. After formulating a model to describe the contagion, and selecting appropriate prior distributions, we seek to find the set of model parameters that best explains our observations. Modeling the problem in a Bayesian framework, we exploit Monte Carlo Markov Chain, Sequential Monte Carlo, and time series analysis techniques to develop batch and online inference algorithms. We evaluate the performance of our proposed algorithms via numerical simulations of synthetic network contagions and analysis of real-world datasets.

cs.SI↗

arXiv2017

Microwave breast cancer detection using Empirical Mode Decomposition features

Microwave-based breast cancer detection has been proposed as a complementary approach to compensate for some drawbacks of existing breast cancer detection techniques. Among the existing microwave breast cancer detection methods, machine learning-type algorithms have recently become more popular. These focus on detecting the existence of breast tumours rather than performing imaging to identify the exact tumour position. A key step of the machine learning approaches is feature extraction. One of the most widely used feature extraction method is principle component analysis (PCA). However, it can be sensitive to signal misalignment. This paper presents an empirical mode decomposition (EMD)-based feature extraction method, which is more robust to the misalignment. Experimental results involving clinical data sets combined with numerically simulated tumour responses show that combined features from EMD and PCA improve the detection performance with an ensemble selection-based classifier.

stat.ML↗

arXiv2016

Bayesian Inference of Diffusion Networks with Unknown Infection Times

The analysis of diffusion processes in real-world propagation scenarios often involves estimating variables that are not directly observed. These hidden variables include parental relationships, the strengths of connections between nodes, and the moments of time that infection occurs. In this paper, we propose a framework in which all three sets of parameters are assumed to be hidden and we develop a Bayesian approach to infer them. After justifying the model assumptions, we evaluate the performance efficiency of our proposed approach through numerical simulations on synthetic datasets and real-world diffusion processes.

cs.SI↗

arXiv2016

Multisensor CPHD filter

The single sensor probability hypothesis density (PHD) and cardinalized probability hypothesis density (CPHD) filters have been developed in the literature using the random finite set framework. The existing multisensor extensions of these filters have limitations such as sensor order dependence, numerical instability or high computational requirements. In this paper we derive update equations for the multisensor CPHD filter. The multisensor PHD filter is derived as a special case. Exact implementation of the multisensor CPHD involves sums over all partitions of the measurements from different sensors and is thus intractable. We propose a computationally tractable approximation which combines a greedy measurement partitioning algorithm with the Gaussian mixture representation of the PHD. Our greedy approximation method allows the user to control the tradeoff between computational overhead and approximation accuracy.

stat.ME↗

arXiv2016

Sparse Multivariate Factor Regression

We consider the problem of multivariate regression in a setting where the relevant predictors could be shared among different responses. We propose an algorithm which decomposes the coefficient matrix into the product of a long matrix and a wide matrix, with an elastic net penalty on the former and an $\ell_1$ penalty on the latter. The first matrix linearly transforms the predictors to a set of latent factors, and the second one regresses the responses on these factors. Our algorithm simultaneously performs dimension reduction and coefficient estimation and automatically estimates the number of latent factors from the data. Our formulation results in a non-convex optimization problem, which despite its flexibility to impose effective low-dimensional structure, is difficult, or even impossible, to solve exactly in a reasonable time. We specify an optimization algorithm based on alternating minimization with three different sets of updates to solve this non-convex problem and provide theoretical results on its convergence and optimality. Finally, we demonstrate the effectiveness of our algorithm via experiments on simulated and real data.

stat.ML↗

arXiv2016

Semi-parametric Order-based Generalized Multivariate Regression

In this paper, we consider a generalized multivariate regression problem where the responses are monotonic functions of linear transformations of predictors. We propose a semi-parametric algorithm based on the ordering of the responses which is invariant to the functional form of the transformation function. We prove that our algorithm, which maximizes the rank correlation of responses and linear transformations of predictors, is a consistent estimator of the true coefficient matrix. We also identify the rate of convergence and show that the squared estimation error decays with a rate of $o(1/\sqrt{n})$. We then propose a greedy algorithm to maximize the highly non-smooth objective function of our model and examine its performance through extensive simulations. Finally, we compare our algorithm with traditional multivariate regression algorithms over synthetic and real data.

stat.ML↗

arXiv2015

Optimal Forwarding in Opportunistic Delay Tolerant Networks with Meeting Rate Estimations

Data transfer in opportunistic Delay Tolerant Networks (DTNs) must rely on unscheduled sporadic meetings between nodes. The main challenge in these networks is to develop a mechanism based on which nodes can learn to make nearly optimal forwarding decision rules despite having no a-priori knowledge of the network topology. The forwarding mechanism should ideally result in a high delivery probability, low average latency and efficient usage of the network resources. In this paper, we propose both centralized and decentralized single-copy message forwarding algorithms that, under relatively strong assumptions about the networks behaviour, minimize the expected latencies from any node in the network to a particular destination. After proving the optimality of our proposed algorithms, we develop a decentralized algorithm that involves a recursive maximum likelihood procedure to estimate the meeting rates. We confirm the improvement that our proposed algorithms make in the system performance through numerical simulations on datasets from synthetic and real-world opportunistic networks.

cs.NI↗

arXiv2013

Content Distribution Strategies in Opportunistic Networks

This paper describes a mechanism for content distribution through opportunistic contacts between subscribers. A subset of subscribers in the network are seeded with the content. The remaining subscribers obtain the information through opportunistic contact with a user carrying the updated content. We study how the rate of content retrieval by subscribers is affected by the number of initial seeders in the network. We also study the rate of content retrieval by the subscribers under coding strategies (Network Coding, Erasure Coding) and under Flooding, Epidemic Routing.

cs.SI↗

arXiv2011

GANC: Greedy Agglomerative Normalized Cut

This paper describes a graph clustering algorithm that aims to minimize the normalized cut criterion and has a model order selection procedure. The performance of the proposed algorithm is comparable to spectral approaches in terms of minimizing normalized cut. However, unlike spectral approaches, the proposed algorithm scales to graphs with millions of nodes and edges. The algorithm consists of three components that are processed sequentially: a greedy agglomerative hierarchical clustering procedure, model order selection, and a local refinement. For a graph of n nodes and O(n) edges, the computational complexity of the algorithm is O(n log^2 n), a major improvement over the O(n^3) complexity of spectral methods. Experiments are performed on real and synthetic networks to demonstrate the scalability of the proposed approach, the effectiveness of the model order selection procedure, and the performance of the proposed algorithm in terms of minimizing the normalized cut metric.

cs.AI↗

arXiv2010

Real-Time Multi-path Tracking of Probabilistic Available Bandwidth

Applications such as traffic engineering and network provisioning can greatly benefit from knowing, in real time, what is the largest input rate at which it is possible to transmit on a given path without causing congestion. We consider a probabilistic formulation for available bandwidth where the user specifies the probability of achieving an output rate almost as large as the input rate. We are interested in estimating and tracking the network-wide probabilistic available bandwidth (PAB) on multiple paths simultaneously with minimal overhead on the network. We propose a novel framework based on chirps, Bayesian inference, belief propagation and active sampling to estimate the PAB. We also consider the time evolution of the PAB by forming a dynamic model and designing a tracking algorithm based on particle filters. We implement our method in a lightweight and practical tool that has been deployed on the PlanetLab network to do online experiments. We show through these experiments and simulations that our approach outperforms block-based algorithms in terms of input rate cost and probability of successful transmission.

cs.NI↗

arXiv2010

Large scale probabilistic available bandwidth estimation

The common utilization-based definition of available bandwidth and many of the existing tools to estimate it suffer from several important weaknesses: i) most tools report a point estimate of average available bandwidth over a measurement interval and do not provide a confidence interval; ii) the commonly adopted models used to relate the available bandwidth metric to the measured data are invalid in almost all practical scenarios; iii) existing tools do not scale well and are not suited to the task of multi-path estimation in large-scale networks; iv) almost all tools use ad-hoc techniques to address measurement noise; and v) tools do not provide enough flexibility in terms of accuracy, overhead, latency and reliability to adapt to the requirements of various applications. In this paper we propose a new definition for available bandwidth and a novel framework that addresses these issues. We define probabilistic available bandwidth (PAB) as the largest input rate at which we can send a traffic flow along a path while achieving, with specified probability, an output rate that is almost as large as the input rate. PAB is expressed directly in terms of the measurable output rate and includes adjustable parameters that allow the user to adapt to different application requirements. Our probabilistic framework to estimate network-wide probabilistic available bandwidth is based on packet trains, Bayesian inference, factor graphs and active sampling. We deploy our tool on the PlanetLab network and our results show that we can obtain accurate estimates with a much smaller measurement overhead compared to existing approaches.

cs.NI↗

arXiv2010

Multi-path Probabilistic Available Bandwidth Estimation through Bayesian Active Learning

Knowing the largest rate at which data can be sent on an end-to-end path such that the egress rate is equal to the ingress rate with high probability can be very practical when choosing transmission rates in video streaming or selecting peers in peer-to-peer applications. We introduce probabilistic available bandwidth, which is defined in terms of ingress rates and egress rates of traffic on a path, rather than in terms of capacity and utilization of the constituent links of the path like the standard available bandwidth metric. In this paper, we describe a distributed algorithm, based on a probabilistic graphical model and Bayesian active learning, for simultaneously estimating the probabilistic available bandwidth of multiple paths through a network. Our procedure exploits the fact that each packet train provides information not only about the path it traverses, but also about any path that shares a link with the monitored path. Simulations and PlanetLab experiments indicate that this process can dramatically reduce the number of probes required to generate accurate estimates.

cs.NI↗