arXiv subjects

John Evans

Publications and source records attributed to John Evans.

15 recordsLinked to original sources

arXiv2026

Graphical Causal Reasoning for Root Cause Analysis in Cloud Networks

Cloud-computing relies on large-scale networks which are inherently complex systems. In this paper, we present a novel approach to root cause analysis (RCA) of cloud network incidents, leveraging graph-based causal discovery techniques. Our method addresses the limitations of rule-based automation by introducing a spatiotemporal grouping strategy and an automation ontology to reduce the dimensionality of the problem. We construct a causal graph from binary time series data using bivariate Granger causality and conditional independence tests. For inference, we introduce a probabilistic method that assigns edge-specific conditional probabilities as a function of time lag, allowing for interpretable, time-aware root cause scoring via causal graph traversal. We evaluated the system using a labeled dataset of 35 production incidents from a major cloud provider. The model successfully recalled the correct root cause in 85.7% of incidents and produced an exact match in 74.3%. In production, the deployed system has been used in over 800 real-world incidents, with positive qualitative feedback from network engineers. These results highlight the practicality of a data-driven, causal approach to RCA in dynamic and large-scale operational environments.

cs.NI↗

arXiv2026

NetCause: Counterfactual Learning for Root Cause Analysis in Large-Scale Networks

Can a learned model capture how faults propagate through a large-scale network and use this knowledge to causally attribute customer impact to its underlying root cause? Existing root cause analysis techniques often rely on static rules, correlation heuristics, or topology-local reasoning, which struggle to generalize in dynamic environments where faults propagate across complex physical and logical dependencies. We present NetCause, a self-supervised learning-based framework that models network incidents as graph-temporal processes and uses counterfactual simulation to rank candidate root causes. This approach produces an interpretable ranking of root cause hypotheses and integrates naturally with operator-defined mitigation and remediation actions. We train the model on over 1,500 incidents collected over six months from a leading cloud provider's production network and evaluate it on 31 expert-labeled incidents. NetCause consistently improves root cause ranking quality in the regime most relevant to operational decision-making, achieving a 16.1% accuracy improvement over a rule-based heuristic baseline. While training is computationally intensive, inference is lightweight, requiring only seconds of GPU runtime per incident (well below typical telemetry collection latencies).

cs.NI↗

arXiv2025

Solving engineering eigenvalue problems with neural networks using the Rayleigh quotient

From characterizing the speed of a thermal system's response to computing natural modes of vibration, eigenvalue analysis is ubiquitous in engineering. In spite of this, eigenvalue problems have received relatively little treatment compared to standard forward and inverse problems in the physics-informed machine learning literature. In particular, neural network discretizations of solutions to eigenvalue problems have seen only a handful of studies. Owing to their nonlinearity, neural network discretizations prevent the conversion of the continuous eigenvalue differential equation into a standard discrete eigenvalue problem. In this setting, eigenvalue analysis requires more specialized techniques. Using a neural network discretization of the eigenfunction, we show that a variational form of the eigenvalue problem called the "Rayleigh quotient" in tandem with a Gram-Schmidt orthogonalization procedure is a particularly simple and robust approach to find the eigenvalues and their corresponding eigenfunctions. This method is shown to be useful for finding sets of harmonic functions on irregular domains, parametric and nonlinear eigenproblems, and high-dimensional eigenanalysis. We also discuss the utility of harmonic functions as a spectral basis for approximating solutions to partial differential equations. Through various examples from engineering mechanics, the combination of the Rayleigh quotient objective, Gram-Schmidt procedure, and the neural network discretization of the eigenfunction is shown to offer unique advantages for handling continuous eigenvalue problems.

math.NA↗

arXiv2024

Corn Ear Detection and Orientation Estimation Using Deep Learning

Monitoring growth behavior of maize plants such as the development of ears can give key insights into the plant's health and development. Traditionally, the measurement of the angle of ears is performed manually, which can be time-consuming and prone to human error. To address these challenges, this paper presents a computer vision-based system for detecting and tracking ears of corn in an image sequence. The proposed system could accurately detect, track, and predict the ear's orientation, which can be useful in monitoring their growth behavior. This can significantly save time compared to manual measurement and enables additional areas of ear orientation research and potential increase in efficiencies for maize production. Using an object detector with keypoint detection, the algorithm proposed could detect 90 percent of all ears. The cardinal estimation had a mean absolute error (MAE) of 18 degrees, compared to a mean 15 degree difference between two people measuring by hand. These results demonstrate the feasibility of using computer vision techniques for monitoring maize growth and can lead to further research in this area.

cs.CV↗

arXiv2021

Analysis Framework for Multi-messenger Astronomy with IceCube

Combining observational data from multiple instruments for multi-messenger astronomy can be challenging due to the complexity of the instrument response functions and likelihood calculation. We introduce a python-based unbinned-likelihood analysis package called i3mla (IceCube Maximum Likelihood Analysis). i3mla is designed to be compatible with the Multi-Mission Maximum Likelihood (3ML) framework, which enables multi-messenger astronomy analyses by combining the likelihood across different instruments. By making it possible to use IceCube data in the 3ML framework, we aim to facilitate the use of neutrino data in multi-messenger astronomy. In this work we illustrate how to use the i3mla package with 3ML and present preliminary sensitivities using the i3mla package and 3ML through a joint-fit with HAWC Public dataset.

astro-ph.IM↗

arXiv2020

Artificial Intelligence for Digital Agriculture at Scale: Techniques, Policies, and Challenges

Digital agriculture has the promise to transform agricultural throughput. It can do this by applying data science and engineering for mapping input factors to crop throughput, while bounding the available resources. In addition, as the data volumes and varieties increase with the increase in sensor deployment in agricultural fields, data engineering techniques will also be instrumental in collection of distributed data as well as distributed processing of the data. These have to be done such that the latency requirements of the end users and applications are satisfied. Understanding how farm technology and big data can improve farm productivity can significantly increase the world's food production by 2050 in the face of constrained arable land and with the water levels receding. While much has been written about digital agriculture's potential, little is known about the economic costs and benefits of these emergent systems. In particular, the on-farm decision making processes, both in terms of adoption and optimal implementation, have not been adequately addressed. For example, if some algorithm needs data from multiple data owners to be pooled together, that raises the question of data ownership. This paper is the first one to bring together the important questions that will guide the end-to-end pipeline for the evolution of a new generation of digital agricultural solutions, driving the next revolution in agriculture and sustainability under one umbrella.

cs.CY↗

arXiv2019

The BIDS Toolbox: A web service to manage brain imaging datasets

Data sharing is a key factor for ensuring reproducibility and transparency of scientific experiments, and neuroimaging is no exception. The vast heterogeneity of data formats and imaging modalities utilised in the field makes it a very challenging problem. In this context, the Brain Imaging Data Structure (BIDS) appears as a solution for organising and describing neuroimaging datasets. Since its publication in 2015, BIDS has gained widespread attention in the field, as it provides a common way to arrange and share multimodal brain images. Although the evident benefits it presents, BIDS has not been widely adopted in the field of MRI yet and we believe that this is due to the lack of a go-to tool to create and managed BIDS datasets. Motivated by this, we present the BIDS Toolbox, a web service to manage brain imaging datasets in BIDS format. Different from other tools, the BIDS Toolbox allows the creation and modification of BIDS-compliant datasets based on MRI data. It provides both a web interface and REST endpoints for its use. In this paper we describe its design and early prototype, and provide a link to the public source code repository.

cs.DL↗

arXiv2017

Workload Engineering: Optimising WAN and DC Resources Through RL-Based Workload Placement

With the rise in data centre virtualization, there are increasing choices as to where to place workloads, be it in web applications, Enterprise IT or in Network Function Virtualisation. Workload placement approaches available today primarily focus on optimising the use of data centre resources. Given the significant forecasts for network traffic growth to/from data centres, effective management of both data centre resources and of the wide area networks resources that provide access to those data centres will become increasingly important. In this paper, we present an architecture for workload placement, which uniquely employs a logically centralised controller that is both network and data centre aware, which aims to place workloads to optimise the use of both data centre and wide area network resources. We call this approach workload engineering. We present the results of a simulation study, where we use a reinforcement-learning based placement algorithm on the controller. Results of the study show this algorithm was able to place workloads to make more efficient use of network and data centre resources and placed ~5-8% more workloads than other heuristic placement algorithms considered, for the same installed capacity.

cs.NI↗

arXiv2016

A Comparison of Approaches for Traffic Engineering in IP and MPLS Networks

As IP / MPLS network service providers drive for improved network efficiencies, traffic engineering is becoming increasingly important. In this article we review the approaches for traffic engineering in IP / MPLS networks and present the result of a study, which compares their performance in operational networks

cs.NI↗

arXiv2016

Demand Engineering: IP Network Optimisation Through Intelligent Demand Placement

Traffic engineering has been used in IP and MPLS networks for a number of years as a tool for making more efficient use of capacity by explicitly routing traffic demands where there is available network capacity that would otherwise be unused. Deployment of traffic engineering imposes an additional layer of complexity to network design and operations, however, which has constrained its adoption for capacity optimisation. The rise of Software Defined Networks has renewed interest in the use of traffic engineering approaches leveraging centralised network controllers for capacity optimisation. We argue that future networks can realise the network optimisation benefits of traffic engineering without incurring additional network complexity through closer coupling between the network and the applications and services using the network. This can be achieved through leveraging a network- and traffic-aware controller to directly influence where applications and services site or locate service instances, i.e. which implicitly impacts the paths that the applications or services traffic demands take through the network. We call this technique Demand Engineering. Demand Engineering has the additional benefit of providing an admission control capability, i.e. which can provide an assurance that network SLAs can be met. In this paper we describe the concept of Demand Engineering, give examples of its use and present simulation results indicating its potential benefits. We also compare demand engineering to traffic engineering.

cs.NI↗

arXiv2016

Knowledge-Defined Networking

The research community has considered in the past the application of Artificial Intelligence (AI) techniques to control and operate networks. A notable example is the Knowledge Plane proposed by D.Clark et al. However, such techniques have not been extensively prototyped or deployed in the field yet. In this paper, we explore the reasons for the lack of adoption and posit that the rise of two recent paradigms: Software-Defined Networking (SDN) and Network Analytics (NA), will facilitate the adoption of AI techniques in the context of network operation and control. We describe a new paradigm that accommodates and exploits SDN, NA and AI, and provide use cases that illustrate its applicability and benefits. We also present simple experimental results that support its feasibility. We refer to this new paradigm as Knowledge-Defined Networking (KDN).

cs.NI↗

arXiv2015

SunPy - Python for Solar Physics

This paper presents SunPy (version 0.5), a community-developed Python package for solar physics. Python, a free, cross-platform, general-purpose, high-level programming language, has seen widespread adoption among the scientific community, resulting in the availability of a large number of software packages, from numerical computation (NumPy, SciPy) and machine learning (scikit-learn) to visualisation and plotting (matplotlib). SunPy is a data-analysis environment specialising in providing the software necessary to analyse solar and heliospheric data in Python. SunPy is open-source software (BSD licence) and has an open and transparent development workflow that anyone can contribute to. SunPy provides access to solar data through integration with the Virtual Solar Observatory (VSO), the Heliophysics Event Knowledgebase (HEK), and the HELiophysics Integrated Observatory (HELIO) webservices. It currently supports image data from major solar missions (e.g., SDO, SOHO, STEREO, and IRIS), time-series data from missions such as GOES, SDO/EVE, and PROBA2/LYRA, and radio spectra from e-Callisto and STEREO/SWAVES. We describe SunPy's functionality, provide examples of solar data analysis in SunPy, and show how Python-based solar data-analysis can leverage the many existing tools already available in Python. We discuss the future goals of the project and encourage interested users to become involved in the planning and development of SunPy.

astro-ph.IM↗

arXiv2002

A New Window Onto Quantum Chaos

In this article the statistical properties of symmetrical random matrices whose elements are drawn from a q-parameterized non-extensive statistics power-law distribution are investigated. In the limit as q->1 the well known Gaussian orthogonal ensemble (GOE) results are recovered. The relevant level spacing distribution is derived and one obtains a suitably generalized non-extensive Wigner distribution which depends on the value of the tunable non-extensivity parameter q. This non-extensive Wigner distribution can be seen to be a one-parameter level-spacing distribution that allows one to interpolate between chaotic and nearly integrable regimes.

cond-mat.stat-mech↗

arXiv2002

Towards a Non-extensive Random Matrix Theory

In this article the statistical properties of symmetrical random matrices whose elements are drawn from a q-parametrized non-extensive statistics power-law distribution are investigated. In the limit as q->1 the well known Gaussian orthogonal ensemble (GOE) results are recovered. The relevant level spacing distribution is derived and one obtains a suitably generalized nonextensive Wigner distribution which depends on the value of the tunable non-extensivity parameter q. This non-extensive Wigner distribution can be seen to be a one-parameter level-spacing distribution that allows one to interpolate between chaotic and nearly integrable regimes.

cond-mat.stat-mech↗

arXiv2002

Excess Demand Financial Market Model

Recently we reported on an application of the Tsallis non-extensive statistics to the S&P500 stock index. There we argued that the statistics are applicable to a broad range of markets and exchanges where anamolous (super) diffusion and 'heavy' tails of the distribution are present, as they are in the S&P500. We have characterized the statistics of the underlying security as non-extensive, and now we seek to generalize to the non-extensive statistics the excess demand models of investors that drive the price formation in a market.

cond-mat.stat-mech↗