arXiv subjects

Rajkumar Buyya

Publications and source records attributed to Rajkumar Buyya.

283 records · Page 16Linked to original sources

arXiv2003

Guided Google: A Meta Search Engine and its Implementation using the Google Distributed Web Services

With the advent of the Internet, search engines have begun sprouting like mushrooms after a rainfall. Only in recent years, have developers become more innovative, and came up with guided searching facilities online. The goals of these applications are to help ease and guide the searching efforts of a novice web user toward their desired objectives. A number of implementations of such services are emerging. This paper proposes a guided meta-search engine, called "Guided Google", as it serves as an interface to the actual Google.com search engine, using the Google Web Services.

cs.DC↗

arXiv2003

Grid Market Directory: A Web Services based Grid Service Publication Directory

As Grids are emerging as the next generation service-oriented computing platforms, they need to support Grid economy that helps in the management of supply and demand for resources and offers an economic incentive for Grid resource providers. To enable this Grid economy, a market-like Grid environment including an infrastructure that supports the publication of services and their discovery is needed. As part of the Gridbus project, we proposed and have developed a Grid Market Directory (GMD) that serves as a registry for high-level service publication and discovery in Virtual Organisations.

cs.DC↗

arXiv2003

G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids

Grids are experiencing a rapid growth in their application and along with this there is a requirement for a portal which is easy to use and scalable. We have responded to this requirement by developing an easy to use, scalable, web-based portal called G-Monitor. This paper proposes a generic architecture for a web portal into a grid environment and discusses our implementation and its application.

cs.DC↗

arXiv2003

Visual Environment for Rapid Composition of Parameter-Sweep Applications for Distributed Processing on Global Grids

Computational Grids are emerging as a platform for next-generation parallel and distributed computing. Large-scale parametric studies and parameter sweep applications find a natural place in the Grid?s distribution model. There is little or no communication between jobs. The task of parallelizing and distributing existing applications is conceptually trivial. These properties of parametric studies make it an ideal place to start developing integrated development environments (IDEs) for rapidly Grid-enabling applications. However, the availability of IDEs for scientists to Grid-enable their applications, without the need of developing them as parallel applications explicitly, is still lacking. This paper presents a Java based IDE called Visual Parametric Tool (VPT), developed as part of the Gridbus project, for rapid creation of parameter sweep (data parallel/SPMD) applications. It supports automatic creation of parameter script and parameterisation of the input data files, which is compatible with the Nimrod-G parameter specification language. The usefulness of VPT is demonstrated by a case study on composition of molecular docking application as a parameter sweep application. Such applications can be deployed on clusters using the Nimrod/enFuzion system and on global Grids using the Nimrod-G grid resource broker.

cs.DC↗

arXiv2002

Libra: An Economy driven Job Scheduling System for Clusters

Clusters of computers have emerged as mainstream parallel and distributed platforms for high-performance, high-throughput and high-availability computing. To enable effective resource management on clusters, numerous cluster managements systems and schedulers have been designed. However, their focus has essentially been on maximizing CPU performance, but not on improving the value of utility delivered to the user and quality of services. This paper presents a new computational economy driven scheduling system called Libra, which has been designed to support allocation of resources based on the users? quality of service (QoS) requirements. It is intended to work as an add-on to the existing queuing and resource management system. The first version has been implemented as a plugin scheduler to the PBS (Portable Batch System) system. The scheduler offers market-based economy driven service for managing batch jobs on clusters by scheduling CPU time according to user utility as determined by their budget and deadline rather than system performance considerations. The Libra scheduler ensures that both these constraints are met within an O(n) run-time. The Libra scheduler has been simulated using the GridSim toolkit to carry out a detailed performance analysis. Results show that the deadline and budget based proportional resource allocation strategy improves the utility of the system and user satisfaction as compared to system-centric scheduling strategies.

cs.DC↗

arXiv2002

Economic-based Distributed Resource Management and Scheduling for Grid Computing

Computational Grids, emerging as an infrastructure for next generation computing, enable the sharing, selection, and aggregation of geographically distributed resources for solving large-scale problems in science, engineering, and commerce. As the resources in the Grid are heterogeneous and geographically distributed with varying availability and a variety of usage and cost policies for diverse users at different times and, priorities as well as goals that vary with time. The management of resources and application scheduling in such a large and distributed environment is a complex task. This thesis proposes a distributed computational economy as an effective metaphor for the management of resources and application scheduling. It proposes an architectural framework that supports resource trading and quality of services based scheduling. It enables the regulation of supply and demand for resources and provides an incentive for resource owners for participating in the Grid and motives the users to trade-off between the deadline, budget, and the required level of quality of service. The thesis demonstrates the capability of economic-based systems for peer-to-peer distributed computing by developing users' quality-of-service requirements driven scheduling strategies and algorithms. It demonstrates their effectiveness by performing scheduling experiments on the World-Wide Grid for solving parameter sweep applications.

cs.DC↗

arXiv2002

GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing

Clusters, grids, and peer-to-peer (P2P) networks have emerged as popular paradigms for next generation parallel and distributed computing. The management of resources and scheduling of applications in such large-scale distributed systems is a complex undertaking. In order to prove the effectiveness of resource brokers and associated scheduling algorithms, their performance needs to be evaluated under different scenarios such as varying number of resources and users with different requirements. In a grid environment, it is hard and even impossible to perform scheduler performance evaluation in a repeatable and controllable manner as resources and users are distributed across multiple organizations with their own policies. To overcome this limitation, we have developed a Java-based discrete-event grid simulation toolkit called GridSim. The toolkit supports modeling and simulation of heterogeneous grid resources (both time- and space-shared), users and application models. It provides primitives for creation of application tasks, mapping of tasks to resources, and their management. To demonstrate suitability of the GridSim toolkit, we have simulated a Nimrod-G like grid resource broker and evaluated the performance of deadline and budget constrained cost- and time-minimization scheduling algorithms.

cs.DC↗

arXiv2002

A Deadline and Budget Constrained Cost-Time Optimisation Algorithm for Scheduling Task Farming Applications on Global Grids

Computational Grids and peer-to-peer (P2P) networks enable the sharing, selection, and aggregation of geographically distributed resources for solving large-scale problems in science, engineering, and commerce. The management and composition of resources and services for scheduling applications, however, becomes a complex undertaking. We have proposed a computational economy framework for regulating the supply and demand for resources and allocating them for applications based on the users quality of services requirements. The framework requires economy driven deadline and budget constrained (DBC) scheduling algorithms for allocating resources to application jobs in such a way that the users requirements are met. In this paper, we propose a new scheduling algorithm, called DBC cost-time optimisation, which extends the DBC cost-optimisation algorithm to optimise for time, keeping the cost of computation at the minimum. The superiority of this new scheduling algorithm, in achieving lower job completion time, is demonstrated by simulating the World-Wide Grid and scheduling task-farming applications for different deadline and budget scenarios using both this new and the cost optimisation scheduling algorithms.

cs.DC↗

arXiv2001

Virtual Laboratory: Enabling On-Demand Drug Design with the World Wide Grid

Computational Grids are emerging as a popular paradigm for solving large-scale compute and data intensive problems in science, engineering, and commerce. However, application composition, resource management and scheduling in these environments is a complex undertaking. In this paper, we illustrate the creation of a virtual laboratory environment by leveraging existing Grid technologies to enable molecular modeling for drug design on distributed resources. It involves screening millions of molecules of chemical compounds against a protein target, chemical database (CDB) to identify those with potential use for drug design. We have grid-enabled the molecular docking process by composing it as a parameter sweep application using the Nimrod-G tools. We then developed new tools for remote access to molecules in CDB small molecule database. The Nimrod-G resource broker along with molecule CDB data broker is used for scheduling and on-demand processing of jobs on distributed grid resources. The results demonstrate the ease of use and suitability of the Nimrod-G and virtual laboratory tools.

cs.DC↗

arXiv2001

A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Brok

Computational Grids, coupling geographically distributed resources such as PCs, workstations, clusters, and scientific instruments, have emerged as a next generation computing platform for solving large-scale problems in science, engineering, and commerce. However, application development, resource management, and scheduling in these environments continue to be a complex undertaking. In this article, we discuss our efforts in developing a resource management system for scheduling computations on resources distributed across the world with varying quality of service. Our service-oriented grid computing system called Nimrod-G manages all operations associated with remote execution including resource discovery, trading, scheduling based on economic principles and a user defined quality of service requirement. The Nimrod-G resource broker is implemented by leveraging existing technologies such as Globus, and provides new services that are essential for constructing industrial-strength Grids. We discuss results of preliminary experiments on scheduling some parametric computations using the Nimrod-G resource broker on a world-wide grid testbed that spans five continents.

cs.DC↗

arXiv2001

Economic Models for Management of Resources in Grid Computing

The accelerated development in Grid and peer-to-peer computing has positioned them as promising next generation computing platforms. They enable the creation of Virtual Enterprises (VE) for sharing resources distributed across the world. However, resource management, application development and usage models in these environments is a complex undertaking. This is due to the geographic distribution of resources that are owned by different organizations. The resource owners of each of these resources have different usage or access policies and cost models, and varying loads and availability. In order to address complex resource management issues, we have proposed a computational economy framework for resource allocation and for regulating supply and demand in Grid computing environments. The framework provides mechanisms for optimizing resource provider and consumer objective functions through trading and brokering services. In a real world market, there exist various economic models for setting the price for goods based on supply-and-demand and their value to the user. They include commodity market, posted price, tenders and auctions. In this paper, we discuss the use of these models for interaction between Grid components in deciding resource value and the necessary infrastructure to realize them. In addition to normal services offered by Grid computing systems, we need an infrastructure to support interaction protocols, allocation mechanisms, currency, secure banking, and enforcement services. Furthermore, we demonstrate the usage of some of these economic models in resource brokering through Nimrod/G deadline and cost-based scheduling for two different optimization strategies on the World Wide Grid (WWG) testbed.

cs.DC↗

arXiv2000

Cluster Computing: A High-Performance Contender

When you first heard people speak of Piles of PCs, the first thing that came to mind may have been a cluttered computer room with processors, monitors, and snarls of cables all around. Collections of computers have undoubtedly become more sophisticated than in the early days of shared drives and modem connections. No matter what you call them, Clusters of Workstations (COW), Networks of Workstations (NOW), Workstation Clusters (WCs), Clusters of PCs (CoPs), clusters of computers are now filling the processing niche once occupied by more powerful stand-alone machines. This article discusses the need for cluster computing technology, Technologies, Components, and Applications, Supercluster Systems and Issues, The Need for a New Task Force, and Cluster Computing Educational Resources.

cs.DC↗

arXiv2000

Nimrod/G: An Architecture of a Resource Management and Scheduling System in a Global Computational Grid

The availability of powerful microprocessors and high-speed networks as commodity components has enabled high performance computing on distributed systems (wide-area cluster computing). In this environment, as the resources are usually distributed geographically at various levels (department, enterprise, or worldwide) there is a great challenge in integrating, coordinating and presenting them as a single resource to the user; thus forming a computational grid. Another challenge comes from the distributed ownership of resources with each resource having its own access policy, cost, and mechanism. The proposed Nimrod/G grid-enabled resource management and scheduling system builds on our earlier work on Nimrod and follows a modular and component-based architecture enabling extensibility, portability, ease of development, and interoperability of independently developed components. It uses the Globus toolkit services and can be easily extended to operate with any other emerging grid middleware services. It focuses on the management and scheduling of computations over dynamic resources scattered geographically across the Internet at department, enterprise, or global level with particular emphasis on developing scheduling schemes based on the concept of computational economy for a real test bed, namely, the Globus testbed (GUSTO).

cs.DC↗