Kertész, Attila and Maros, Gergő and Dombi, József Dániel (2014) Multi-job Meta-Brokering in Distributed Computing Infrastructures using Pliant Logic. In: 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing - PDP 2014. IEEE Computer Society, Piscataway, pp. 138-145. ISBN 978-1-4799-2728-9 10.1109/PDP.2014.33

Text Kertesz_138_2541552_z.pdf Restricted to Registered users only Download (166kB) | Request a copy

Abstract

The ever growing number of computation- intensive applications calls for the interoperation of distributed infrastructures such as Clouds, Grids and private clusters. The European SHIWA and ER-flow projects have been initiated to enable the combination of heterogeneous scientific work- flows, and to execute them in a large-scale system consisting of multiple Distributed Computing Infrastructures including Grids and Clouds. In this paper we focus on one of the resource management challenges of these projects called multi- job scheduling. A parameter study job of a workflow having a large number of input files to be consumed by independent job instances is called a multi-job. In order to cope with the high uncertainty and unpredictable load of these infrastructures and with the simultaneous submissions of multi-job instances, we use statistical historical job allocation data gathered from real-world workflow archives and propose an adaptive meta- brokering approach for the management of this unified system based on the Pliant logic concept, which is a specific part of fuzzy logic theory. We argue that this novel scheduling technique produce better performance scores, hence the overall load of the system can be more balanced.

Update Item