Narrow Results

Estimates of task runtime, disk space usage, and memory consumption, are commonly used by scheduling and resource provisioning algorithms to support efficient and reliable workflow executions. Such algorithms often assume that accurate estimates are available, but such estimates are difficult to generate in practice. In this work, we first profile five real scientific workflows, collecting fine-grained information such as process I/O, runtime, memory usage, and CPU utilization. We then propose a method to automatically characterize workflow task requirements based on these profiles. Our method estimates task runtime, disk space, and peak memory consumption based on the size of the tasks’ input data. It looks for correlations between the parameters of a dataset, and if no correlation is found, the dataset is divided into smaller subsets using a clustering technique. Task estimates are generated based on the ratio parameter/input data size if they are correlated, or based on the probability distribution function of the parameter. We then propose an online estimation process based on the MAPE-K loop, where task executions are monitored and estimates are updated as more information becomes available. Experimental results show that our online estimation process results in much more accurate predictions than an offline approach, where all task requirements are estimated prior to workflow execution.

Scientific WorkFlows (SWFs) play significant roles in scientific research and engineering simulation, which are often data intensive and has complex data dependencies. The storage of massive intermediate datasets has great impacts on the performance and the quality of service of a SWF system, which has become a difficult and complex task. Through analyzing the cost transitive tournament shortest path (CTT-SP)-based algorithm proposed for the intermediate data storage in SWF systems, the disadvantage of the CTT-SP algorithm over its sensitivity to the main branches can be found. We proposed an improved CTT-SP algorithm based on the critical path method (CPM), aiming at reducing the sensitivity of the main branch and improving the performance of the algorithm. By adding the generation information of the datasets on the arrows, the data dependency graph of a SWF can be converted to an activity on arrow (AOA) net. Then, the critical path of the AOA net is used to be the main branch of the CTT-SP algorithm, which can reduce the impacts of the main branch and keep the quality of service of the SWF systems. Experiments are designed to test the impacts of the main branches and the performance of the improved CTT-SP algorithms. The results show the significant impacts of the main branches on the performance of the CTT-SP algorithm and the effectiveness of the CPM on improving the performance of the CTT-SP algorithm. Comparison results demonstrate the positiveness and effectiveness of the improved CTT-SP algorithm.

Scientific workflow is a common model to organize large scientific computations. It borrows the concept of workflow in business activities to manage the complicated processes in scientific computing automatically or semi-automatically. The workflow scheduling, which maps tasks in workflows to parallel computing resources, has been extensively studied over years. In recent years, with the rise of cloud computing as a new large-scale distributed computing model, it is of great significance to study workflow scheduling problem in the cloud. Compared with traditional distributed computing platforms, cloud platforms have unique characteristics such as the self-service resource management model and the pay-as-you-go billing model. Therefore, the workflow scheduling in cloud needs to be reconsidered. When scheduling workflows in clouds, the monetary cost and the makespan of the workflow executions are concerned with both the cloud service providers (CSPs) and the customers. In this paper, we study a series of cost-and-time-aware workflow scheduling algorithms in cloud environments, which aims to provide researchers with a choice of appropriate cloud workflow scheduling approaches in various scenarios. We conducted a broad review of different cloud workflow scheduling algorithms and categorized them based on their optimization objectives and constraints. Also, we discuss the possible future research direction of the clouds workflow scheduling.

Cloud services are used to achieve diverse computing needs such as cost, security, scalability, and availability. Acceleration evolution in the distributed and cloud domains is common for large and dynamic workflows deployment. Resources and task mapping depend on the user’s objectives such as reduction in cost or execution completion within the stipulated time in consideration with certain quality of services. Multiple virtual machine instances can be launched by defining different configurations such as operating system, server types, and applications. Though workflow scheduling is an NP-Hard problem, variety of decision-making techniques are available for optimum resource allocation. In this research paper, different algorithms are studied and compared with evolutionary approaches. Workflow scheduling using genetic algorithm is implemented and discussed. This paper aims to design a decision-making technique to optimize resources of cloud. It is an adaptive scheduling to maximize profit by reducing execution time. The approach implemented is useful to cloud service providers to maximize profit and resource efficiency in their services.