arXiv subjects

Richard McClatchey

Publications and source records attributed to Richard McClatchey.

At least 19 recordsLinked to original sources

arXiv2018

Data provenance tracking as the basis for a biomedical virtual research environment

In complex data analyses it is increasingly important to capture information about the usage of data sets in addition to their preservation over time to ensure reproducibility of results, to verify the work of others and to ensure appropriate conditions data have been used for specific analyses. Scientific workflow based studies are beginning to realize the benefit of capturing this provenance of data and the activities used to process, transform and carry out studies on those data. One way to support the development of workflows and their use in (collaborative) biomedical analyses is through the use of a Virtual Research Environment. The dynamic and distributed nature of Grid/Cloud computing, however, makes the capture and processing of provenance information a major research challenge. Furthermore most workflow provenance management services are designed only for data-flow oriented workflows and researchers are now realising that tracking data or workflows alone or separately is insufficient to support the scientific process. What is required for collaborative research is traceable and reproducible provenance support in a full orchestrated Virtual Research Environment (VRE) that enables researchers to define their studies in terms of the datasets and processes used, to monitor and visualize the outcome of their analyses and to log their results so that others users can call upon that acquired knowledge to support subsequent studies. We have extended the work carried out in the neuGRID and N4U projects in providing a so-called Virtual Laboratory to provide the foundation for a generic VRE in which sets of biomedical data (images, laboratory test results, patient records, epidemiological analyses etc.) and the workflows (pipelines) used to process those data, together with their provenance data and results sets are captured in the CRISTAL software.

cs.SE↗

arXiv2018

The Deployment of an Enhanced Model-Driven Architecture for Business Process Management

Business systems these days need to be agile to address the needs of a changing world. Business modelling requires business process management to be highly adaptable with the ability to support dynamic workflows, inter-application integration (potentially between businesses) and process reconfiguration. Designing systems with the in-built ability to cater for evolution is also becoming critical to their success. To handle change, systems need the capability to adapt as and when necessary to changes in users requirements. Allowing systems to be self-describing is one way to facilitate this. Using our implementation of a self-describing system, a so-called description-driven approach, new versions of data structures or processes can be created alongside older versions providing a log of changes to the underlying data schema and enabling the gathering of traceable (provenance) data. The CRISTAL software, which originated at CERN for handling physics data, uses versions of stored descriptions to define versions of data and workflows which can be evolved over time and thereby to handle evolving system needs. It has been customised for use in business applications as the Agilium-NG product. This paper reports on how the Agilium-NG software has enabled the deployment of an unique business process management solution that can be dynamically evolved to cater for changing user requirement.

cs.SE↗

arXiv2018

Cloud Infrastructure Provenance Collection and Management to Reproduce Scientific Workflow Execution

The emergence of Cloud computing provides a new computing paradigm for scientific workflow execution. It provides dynamic, on-demand and scalable resources that enable the processing of complex workflow-based experiments. With the ever growing size of the experimental data and increasingly complex processing workflows, the need for reproducibility has also become essential. Provenance has been thought of a mechanism to verify a workflow and to provide workflow reproducibility. One of the obstacles in reproducing an experiment execution is the lack of information about the execution infrastructure in the collected provenance. This information becomes critical in the context of Cloud in which resources are provisioned on-demand and by specifying resource configurations. Therefore, a mechanism is required that enables capturing of infrastructure information along with the provenance of workflows executing on the Cloud to facilitate the re-creation of execution environment on the Cloud. This paper presents a framework, ReCAP, along with the proposed mapping approaches that aid in capturing the Cloud-aware provenance information and help in re-provisioning the execution resource on the Cloud with similar configurations. Experimental evaluation has shown the impact of different resource configurations on the workflow execution performance, therefore justifies the need for collecting such provenance information in the context of Cloud. The evaluation has also demonstrated that the proposed mapping approaches can capture Cloud information in various Cloud usage scenarios without causing performance overhead and can also enable the re-provisioning of resources on Cloud. Experiments were conducted using workflows from different scientific domains such as astronomy and neuroscience to demonstrate the applicability of this research for different workflows.

cs.DC↗

arXiv2018

Cloud Provider Capacity Augmentation Through Automated Resource Bartering

Growing interest in Cloud Computing places a heavy workload on cloud providers which is becoming increasingly difficult for them to manage with their primary datacenter infrastructures. Resource limitations can make providers vulnerable to significant reputational damage and it often forces customers to select services from the larger, more established companies, sometimes at a higher price. Funding limitations, however, commonly prevent emerging and even established providers from making continual investment in hardware speculatively assuming a certain level of growth in demand. As an alternative, they may strive to use the current inter-cloud resource sharing platforms which mainly rely on monetary payments and thus putting pressure on already stretched cash flows. To address such issues, we have designed and implemented a new multi-agent based Cloud Resource Bartering System (CRBS) that fosters the management and bartering of pooled resources without requiring costly financial transactions between providers. Agents in CRBS not only strengthen the trading relationship among providers but also enable them to handle surges in demand with their primary setup. Unlike existing systems, CRBS assigns resources by considering resource urgency which comparatively improves customers satisfaction and the resource utilization rate by more than 50%.The evaluation results provide evidence that our system assists providers to timely acquire the additional resources and to maintain sustainable service delivery. We conclude that the existence of such a system is economically beneficial for cloud providers and enables them to adapt to fluctuating workloads.

cs.SE↗

arXiv2018

Using a Model-driven Approach in Building a Provenance Framework for Tracking Policy-making Processes in Smart Cities

The significance of provenance in various settings has emphasised its potential in the policy-making process for analytics in Smart Cities. At present, there exists no framework that can capture the provenance in a policy-making setting. This research therefore aims at defining a novel framework, namely, the Policy Cycle Provenance (PCP) Framework, to capture the provenance of the policy-making process. However, it is not straightforward to design the provenance framework due to a number of associated policy design challenges. The design challenges revealed the need for an adaptive system for tracking policies therefore a model-driven approach has been considered in designing the PCP framework. Also, suitability of a networking approach is proposed for designing workflows for tracking the policy-making process.

cs.SE↗

arXiv2018

An Adaptable System to Support Provenance Management for the Public Policy-Making Process in Smart Cities

Government policies aim to address public issues and problems and therefore play a pivotal role in peoples lives. The creation of public policies, however, is complex given the perspective of large and diverse stakeholders involvement, considerable human participation, lengthy processes, complex task specification and the non-deterministic nature of the process. The inherent complexities of the policy process impart challenges for designing a computing system that assists in supporting and automating the business process pertaining to policy setup, which also raises concerns for setting up a tracking service in the policy-making environment. A tracking service informs how decisions have been taken during policy creation and can provide useful and intrinsic information regarding the policy process. At present, there exists no computing system that assists in tracking the complete process that has been employed for policy creation. To design such a system, it is important to consider the policy environment challenges; for this a novel network and goal based approach has been framed and is covered in detail in this paper. Furthermore, smart governance objectives that include stakeholders participation and citizens involvement have been considered. Thus, the proposed approach has been devised by considering smart governance principles and the knowledge environment of policy making where tasks are largely dependent on policy makers decisions and on individual policy objectives. Our approach reckons the human dimension for deciding and defining autonomous process activities at run time. Furthermore, with the network-based approach, so-called provenance data tracking is employed which enables the capture of policy process.

cs.DB↗

arXiv2018

A Provenance Framework for Policy Analytics in Smart Cities

Sustainable urban environments based on Internet of Things (IoT) technologies require appropriate policy management. However, such policies are established as a result of underlying, potentially complex and long-term policy making processes. Consequently, better policies require improved and verifiable planning processes. In order to assess and evaluate the planning process, transparency of the system is pivotal which can be achieved by tracking the provenance of policy making process. However, at present no system is available that can track the complete cycle of urban planning and decision making. We propose to capture the complete process of policy making and to investigate the role of IoT provenance to support design-making for policy analytics and implementation. The environment in which this research will be demonstrated is that of Smart Cities whose requirements will drive the research process.

cs.CY↗

arXiv2018

A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment

The concept of coupling geographically distributed resources for solving large scale problems is becoming increasingly popular forming what is popularly called grid computing. Management of resources in the Grid environment becomes complex as the resources are geographically distributed, heterogeneous in nature and owned by different individuals and organizations each having their own resource management policies and different access and cost models. There have been many projects that have designed and implemented the resource management systems with a variety of architectures and services. In this paper we have presented the general requirements that a Resource Management system should satisfy. The taxonomy has also been defined based on which survey of resource management systems in different existing Grid projects has been conducted to identify the key areas where these systems lack the desired functionality.

cs.DC↗

arXiv2015

Development of a Large-scale Neuroimages and Clinical Variables Data Atlas in the neuGRID4You (N4U) project

Exceptional growth in the availability of large-scale clinical imaging datasets has led to the development of computational infrastructures offering scientists access to image repositories and associated clinical variables data. The EU FP7 neuGRID and its follow on neuGRID4You (N4U) project is a leading e-Infrastructure where neuroscientists can find core services and resources for brain image analysis. The core component of this e-Infrastructure is the N4U Virtual Laboratory, which offers an easy access for neuroscientists to a wide range of datasets and algorithms, pipelines, computational resources, services, and associated support services. The foundation of this virtual laboratory is a massive data store plus information services called the Data Atlas that stores datasets, clinical study data, data dictionaries, algorithm/pipeline definitions, and provides interfaces for parameterised querying so that neuroscientists can perform analyses on required datasets. This paper presents the overall design and development of the Data Atlas, its associated datasets and indexing and a set of retrieval services that originated from the development of the N4U Virtual Laboratory in the EU FP7 N4U project in the light of user requirements.

cs.DB↗

arXiv2015

Analysis Traceability and Provenance for HEP

This paper presents the use of the CRISTAL software in the N4U project. CRISTAL was used to create a set of provenance aware analysis tools for the Neuroscience domain. This paper advocates that the approach taken in N4U to build the analysis suite is sufficiently generic to be able to be applied to the HEP domain. A mapping to the PROV model for provenance interoperability is also presented and how this can be applied to the HEP domain for the interoperability of HEP analyses.

cs.DB↗

arXiv2015

Traceability and Provenance in Big Data Medical Systems

Providing an appropriate level of accessibility to and tracking of data or process elements in large volumes of medical data, is an essential requirement in the Big Data era. Researchers require systems that provide traceability of information through provenance data capture and management to support their clinical analyses. We present an approach that has been adopted in the neuGRID and N4U projects, which aimed to provide detailed traceability to support research analysis processes in the study of biomarkers for Alzheimers disease, but is generically applicable across medical systems. To facilitate the orchestration of complex, large-scale analyses in these projects we have adapted CRISTAL, a workflow and provenance tracking solution. The use of CRISTAL has provided a rich environment for neuroscientists to track and manage the evolution of data and workflow usage over time in neuGRID and N4U.

cs.DB↗

arXiv2015

Using Cloud-Aware Provenance to Reproduce Scientific Workflow Execution on Cloud

Provenance has been thought of a mechanism to verify a workflow and to provide workflow reproducibility. This provenance of scientific workflows has been effectively carried out in Grid based scientific workflow systems. However, recent adoption of Cloud-based scientific workflows present an opportunity to investigate the suitability of existing approaches or propose new approaches to collect provenance information from the Cloud and to utilize it for workflow repeatability in the Cloud infrastructure. This paper presents a novel approach that can assist in mitigating this challenge. This approach can collect Cloud infrastructure information from an outside Cloud client along with workflow provenance and can establish a mapping between them. This mapping is later used to re-provision resources on the Cloud for workflow execution. The reproducibility of the workflow execution is performed by: (a) capturing the Cloud infrastructure information (virtual machine configuration) along with the workflow provenance, (b) re-provisioning the similar resources on the Cloud and re-executing the workflow on them and (c) by comparing the outputs of workflows. The evaluation of the prototype suggests that the proposed approach is feasible and can be investigated further. Moreover, there is no reference reproducibility model exists in literature that can provide guidelines to achieve this goal in Cloud. This paper also attempts to present a model that is used in the proposed design to achieve workflow reproducibility in the Cloud environment.

cs.DB↗

arXiv2015

Designing Traceability into Big Data Systems

Providing an appropriate level of accessibility and traceability to data or process elements (so-called Items) in large volumes of data, often Cloud-resident, is an essential requirement in the Big Data era. Enterprise-wide data systems need to be designed from the outset to support usage of such Items across the spectrum of business use rather than from any specific application view. The design philosophy advocated in this paper is to drive the design process using a so-called description-driven approach which enriches models with meta-data and description and focuses the design process on Item re-use, thereby promoting traceability. Details are given of the description-driven design of big data systems at CERN, in health informatics and in business process management. Evidence is presented that the approach leads to design simplicity and consequent ease of management thanks to loose typing and the adoption of a unified approach to Item management and usage.

cs.DB↗

arXiv2015

Position Paper: Provenance Data Visualisation for Neuroimaging Analysis

Visualisation facilitates the understanding of scientific data both through exploration and explanation of visualised data. Provenance contributes to the understanding of data by containing the contributing factors behind a result. With the significant increase in data volumes and algorithm complexity, clinical researchers are struggling with information tracking, analysis reproducibility and the verification of scientific output. Data coming from various heterogeneous sources (multiple sources with varying level of trust) in a collaborative environment adds to the uncertainty of the scientific output. Systems are required that offer provenance data capture and visualisation support for analyses. We present an account for the need to visualise provenance information in order to aid the process of verification of scientific outputs, comparison of analyses,progression and evolution of results for neuroimaging analysis.

cs.DB↗

arXiv2015

Scientific Workflow Repeatability through Cloud-Aware Provenance

The transformations, analyses and interpretations of data in scientific workflows are vital for the repeatability and reliability of scientific workflows. This provenance of scientific workflows has been effectively carried out in Grid based scientific workflow systems. However, recent adoption of Cloud-based scientific workflows present an opportunity to investigate the suitability of existing approaches or propose new approaches to collect provenance information from the Cloud and to utilize it for workflow repeatability in the Cloud infrastructure. The dynamic nature of the Cloud in comparison to the Grid makes it difficult because resources are provisioned on-demand unlike the Grid. This paper presents a novel approach that can assist in mitigating this challenge. This approach can collect Cloud infrastructure information along with workflow provenance and can establish a mapping between them. This mapping is later used to re-provision resources on the Cloud. The repeatability of the workflow execution is performed by: (a) capturing the Cloud infrastructure information (virtual machine configuration) along with the workflow provenance, and (b) re-provisioning the similar resources on the Cloud and re-executing the workflow on them. The evaluation of an initial prototype suggests that the proposed approach is feasible and can be investigated further.

cs.DB↗

arXiv2014

Research Traceability using Provenance Services for Biomedical Analysis

We outline the approach being developed in the neuGRID project to use provenance management techniques for the purposes of capturing and preserving the provenance data that emerges in the specification and execution of workflows in biomedical analyses. In the neuGRID project a provenance service has been designed and implemented that is intended to capture, store, retrieve and reconstruct the workflow information needed to facilitate users in conducting user analyses. We describe the architecture of the neuGRID provenance service and discuss how the CRISTAL system from CERN is being adapted to address the requirements of the project and then consider how a generalised approach for provenance management could emerge for more generic application to the (Health)Grid community.

cs.DB↗

arXiv2014

CRISTAL-ISE : Provenance Applied in Industry

This paper presents the CRISTAL-iSE project as a framework for the management of provenance information in industry. The project itself is a research collaboration between academia and industry. A key factor in the project is the use of a system known as CRISTAL which is a mature system based on proven description driven principles. A crucial element in the description driven approach is that the fact that objects (Items) are described at runtime enabling managed systems to be both dynamic and flexible. Another factor is the notion that all Items in CRISTAL are stored and versioned, therefore enabling a provenance collection system. In this paper a concrete application, called Agilium, is briefly described and a future application CIMAG-RA is presented which will harness the power of both CRISTAL and Agilium.

cs.DB↗

arXiv2014

Designing Reusable Systems that Can Handle Change - Description-Driven Systems : Revisiting Object-Oriented Principles

In the age of the Cloud and so-called Big Data systems must be increasingly flexible, reconfigurable and adaptable to change in addition to being developed rapidly. As a consequence, designing systems to cater for evolution is becoming critical to their success. To be able to cope with change, systems must have the capability of reuse and the ability to adapt as and when necessary to changes in requirements. Allowing systems to be self-describing is one way to facilitate this. To address the issues of reuse in designing evolvable systems, this paper proposes a so-called description-driven approach to systems design. This approach enables new versions of data structures and processes to be created alongside the old, thereby providing a history of changes to the underlying data models and enabling the capture of provenance data. The efficacy of the description-driven approach is exemplified by the CRISTAL project. CRISTAL is based on description-driven design principles; it uses versions of stored descriptions to define various versions of data which can be stored in diverse forms. This paper discusses the need for capturing holistic system description when modelling large-scale distributed systems.

cs.SE↗