Parallel Processing Letters

Many HPC systems run workloads comprising scientific simulations where the problem size is often fixed while model parameters are changed for each run. The speedup profile of such applications can often be determined using data from previous runs. Many applications exhibit sublinear speedups and thus only incremental improvements in execution time when larger numbers of processors are used. In this paper, we consider a workload of applications queued for execution and co-schedules that can exploit speedup profiles. More specifically, we seek co-schedules to reduce workload completion time (i.e., makespan), and total energy consumed by the system. Additionally, we explore how co-schedules may reduce average application turnaround time and thus benefit the user. We propose speedup-aware processor partitioning (SAPP) co-scheduling schemes, including optimal and greedy variants. We show that our greedy co-schedules reduce system energy, workload makespan and average application turnaround time when compared to a base scheme that runs each application on all available processors. Our experiments demonstrate that on a system with 128 processors, on average, our SAPP-greedy co-schedules can reduce makespan and turnaround time by 20% and decrease total energy consumed by 40%.

Rapid advances in cloud computing have made the vision of utility computing a near-reality, but only in certain domains. For science and engineering parallel or distributed applications, on-demand access to resources within grids and clouds is hampered by two major factors: communication performance and paradigm mismatch issues. We propose a framework for addressing the latter aspect via software adaptations that attempt to reconcile model and interface differences between application needs and resource platforms. Such matching can greatly enhance flexibility in choice of execution platforms — a key characteristic of utility computing — even though they may not be a natural fit or may incur some performance loss. Our design philosophy, middleware components, and experiences from a cross-paradigm experiment are described.

We aim at mapping streaming applications that can be modeled by a series-parallel graph onto a 2-dimensional tiled chip multiprocessor (CMP) architecture. The objective of the mapping is to minimize the energy consumption, using dynamic voltage and frequency scaling (DVFS) techniques, while maintaining a given level of performance, reflected by the rate of processing the data streams. This mapping problem turns out to be NP-hard, and several heuristics are proposed. We assess their performance through comprehensive simulations using the StreamIt workflow suite and randomly generated series-parallel graphs, and various CMP grid sizes.

Scientists today are exploring the use of new tools and computing platforms to do their science. They are using workflow management tools to describe and manage complex applications and are evaluating the features and performance of clouds to see if they meet their computational needs. Although today, hosting is limited to providing virtual resources and simple services, one can imagine that in the future entire scientific analyses will be hosted for the user. The latter would specify the desired analysis, the timeframe of the computation, and the available budget. Hosted services would then deliver the desired results within the provided constraints. This paper describes current work on managing scientific applications on the cloud, focusing on workflow management and related data management issues. Frequently, applications are not represented by single workflows but rather as sets of related workflowsworkflow ensembles. Thus, hosted services need to be able to manage entire workflow ensembles, evaluating tradeoffs between completing as many high-value ensemble members as possible and delivering results within a certain time and budget. This paper gives an overview of existing hosted science issues, presents the current state of the art on resource provisioning that can support it, as well as outlines future research directions in this field.

Federated, secure, standardized, scalable, and transparent mechanism to access and share resources, particularly data resources, across organizational boundaries that does not require application modification and does not disrupt existing data access patterns has been needed for some time in the computational science community. The Global Federated File System (GFFS) addresses this need and is a foundational component of the NSF-funded eXtreme Science and Engineering Discovery Environment (XSEDE) program. The GFFS allows user applications to access (create, read, update, delete) remote resources in a location-transparent fashion. Existing applications, whether they are statically linked binaries, dynamically linked binaries, or scripts (shell, PERL, Python), can access resources anywhere in the GFFS without modification (subject to access control). In this paper we present an overview of the GFFS and its most common use cases: accessing data at an NSF center from a home or campus, accessing data on a campus machine from an NSF center, directly sharing data with a collaborator at another institution, accessing remote computing resources, and interacting with remote running jobs. We present these uses cases and how they are realized using the GFFS.

We describe an approach to data analytics on large systems using a suite of robust parallel algorithms running on both clouds and HPC systems. We apply this to cases where the data is defined in a vector space and when only pairwise distances between points are defined. We introduce improvements to known algorithms for functionality, features and performance but review state of the art as this is not broadly familiar. Visualization is valuable for steering complex analytics and we discuss it for both the non vector semi-metric case and for clustering high dimension vector spaces. We exploit deterministic annealing which is heuristic but has clear general principles that can give reasonably fast robust algorithms. We apply methods to several life sciences applications.