Service Level Agreements for Job Submission and Scheduling in High Performance Computing - PDF

Please download to get full document.

View again

of 25
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Internet & Technology

Published:

Views: 7 | Pages: 25

Extension: PDF | Download: 0

Share
Related documents
Description
Höchstleistungsrechenzentrum Universität Stuttgart Prof. Dr.-Ing. Dr.h.c. Dr.h.c. M. Resch Nobelstraße Stuttgart Institut für Höchstleistungsrechnen Service Level Agreements for Job Submission
Transcript
Höchstleistungsrechenzentrum Universität Stuttgart Prof. Dr.-Ing. Dr.h.c. Dr.h.c. M. Resch Nobelstraße Stuttgart Institut für Höchstleistungsrechnen Service Level Agreements for Job Submission and Scheduling in High Performance Computing Von der Fakultät Energie-, Verfahrens- und Biotechnik der Universität Stuttgart zur Erlangung der Würde eines Doktor-Ingenieurs (Dr.-Ing.) genehmigte Abhandlung vorgelegt von Roland Kübert aus Aschaffenburg Hauptberichter: Prof. Dr.-Ing. Dr.h.c. Dr.h.c. Michael M. Resch Mitberichter: Prof. Dr.-Ing. Stefan Wesner Tag der Einreichung: 23. Januar 2013 Tag der mündlichen Prüfung: 04. August 2014 Höchstleistungsrechenzentrum Stuttgart Universität Stuttgart August 2014 Acknowledgements No man is an island, entire of itself, as John Donne so pointedly stated in Meditation XVII. When writing a thesis, it sometimes feels as one has taken an endeavour where this is untrue, but in fact it is not. There are many people who provide help and support and I am very glad to acknowledge the help I received in the course of this work. First and foremost, all the thanks in the world go to Lutz Schubert, a long-time colleague at HLRS without whom this thesis would not have been started in the first place. His continued interest in hosting the PhD seminar then helped to keep me going and to bring the thesis to a fruitful completion. Stefan Wesner, who was my superior during most of my time at HLRS and as well my second advisor, helped me a lot especially during the end of the thesis and contributed to a number of publications. Thanks to all participants at the regular HLRS PhD day, which offered the possibility to present my work to a wider audience than the small group of the PhD seminar, and to Michael Resch, the director of HLRS and my advisor. Special thanks go to my parents, who have supported me throughout my life and whose support enabled me to study an pursue a PhD in the first place. Last but not least I also thank my friends for providing support and friendship you know who you are. Aschaffenburg, August 2014 Roland Kübert iii iv Abstract High performance computing (HPC) providers mostly offer their services on a best effort basis. The operational model for HPC resources is at its heart still the same as in its infancy when users placed sets of punch card at a desk for a job to be run. Nowadays, users can verify programs beforehand on their local computers before submitting online. They do not need to interact with human operators but can instead submit jobs themselves to various queues with different parameters. Still, though, jobs are scheduled in a batch system with the provider s goal of reaching a high utilisation. Scheduling is thus oriented only towards a global optimisation rather than an individual service offer. Having only a single global optimisation point prevents providers from pursuing differing goals. There is no way to offer enhanced services with complex QoS parameters. Even if complex scenarios can be realised, they require manual intervention, which cannot scale. Urgent computing, for example, requires immediate execution of jobs, for example for weather predictions or a medical use case. The problem is therefore double sided, with different characteristics and consequences for clients and providers. This work is based on an analysis of scenarios that cannot easily be performed today but are in demand. At first, requirements specific to these scenarios as well as generic requirements were elicited. Then, analysis of the state of the art in relevant areas was undertaken, namely in the fields of job submission and scheduling, simulation thereof, service level agreements (SLAs) and SLA management. The current models of SLA usage and SLA management, relying on a correspondence of one SLA per job to be computed, was identified as a hindrance to the uptake of service based offerings in the HPC domain. A different SLA model is proposed in this work: using SLAs as long-term contracts. In contrast to the usage of SLAs as describing quality of service requirements of a single job, long-term SLAs describes classes of service levels which can be referenced by individual jobs. Derived from the previously identified requirements, an SLA management framework for the HPC environment using long-term SLAs was proposed and its feasibility demonstrated by its implementation and ability to realise a scenario. In order to facilitate the simulation of service levels and scheduling, an existing simulation software tool was enhanced and modified to support SLAs in order to be able to simulate service levels offline without having to use actual physical infrastructure. The work performed in this thesis introduced long-term SLAs as a tool for offering service level in HPC environments and confirmed their profitability. Implementing them can be done with a straightforward SLA management framework that can be easily integrated v into existing setups, from application layer to infrastructure layer. Scheduling according to service levels cannot be solved in a generic way. Nonetheless, for specific setups - for example, multiple service levels differing in priority relative to each other - some principles can be stated that generally hold true. In most cases, however, the scheduler implementation needs to be tailored to the offers an HPC provider chooses to tender. This is simplified by the simulation tool adapted to SLA based scheduling, which allows the simulation of different workloads, physical resources and scheduling algorithms. In conclusion, the combination of long-term SLAs and an SLA management framework reduced to the bare necessities allows providers the implementation of service level based offerings. Work to be done includes among other things, the usage of a provider s real workload traces and infrastructures and the implementation of SLA management in the client domain. vi Zusammenfassung Dienste im Bereich Höchstleistungsrechnen (HPC) werden heutzutage hauptsächlich auf einer best effort -Basis angeboten. Das bedeutet, dass ein HPC-Anbieter sich bemüht, die Dienste bestmöglich zu erfüllen aber keinerlei Garantien hinsichtlich Ausführungszeitrahmen, Qualität oder ähnlichem gibt. Das Betriebsmodell für HPC-Ressourcen ist im Kern noch das gleiche wie zu Beginn des Aufkommens von HPC, als Benutzer Lochkarten persönlich abgaben um die darauf aufgeführten Berechnungen durchführen zu lassen. Heutzutage können Benutzer ihre Programme natürlich auf ihrem lokalen Rechner testen bevor sie diese übermitteln und anstatt einer Interaktion mit menschlichem Bedienpersonal werden die Jobs an verschiedene Warteschlangen mit unterschiedlichen Parametern übermittelt. Nach wie vor werden die Jobs allerdings in einer Stapelverarbeitung abgearbeitet und das Ziel des Anbieters ist es, eine hohe Auslastung des Systems zu gewährleisten. Das Scheduling zielt demnach auf eine globale Optimierung und nicht auf die Erfüllung individueller Dienstgüten. Die Tatsache, dass nur ein einziger globaler Optimierungspunkt existiert verhindert dass ein Anbieter verschiedene Ziele verfolgt. Es gibt keine Möglichkeit, erweiterte Dienste mit komplexen Dienstgüteparametern anzubieten. Wenn doch komplexe Szenarien realisiert werden, bedarf es des manuellen Eingriffs in das System, was in keiner Weise skalierbar ist. Urgent computing verlangt zum Beispiel nach sofortiger Ausführung von Jobs, zum Beispiel für dringende Wettervorhersagen oder medizinische Anwendungsfälle. Das Problem ist daher zweiseitig, mit unterschiedlichen Ausprägungen und Konsequenzen für Kunden und Anbieter. Diese Arbeit basiert auf der Analyse von Szenarien, die in der heutigen HPC-Landschaft nicht durchgeführt werden können, die aber von Kunden nachgefragt werden. Grundlage der Untersuchungen war die Analyse der Szenarien und die Identifikation von szenariospezifischen als auch von generellen Anforderungen. Danach wurde der Stand der Technik in den relevanten Feldern Job-Übermittlung und Job-Scheduling, Simulation desselben, Dienstgütevereinbarungen (SLAs) und der Verwaltung von SLAs untersucht. Das heute verwendete Modell der Nutzung und Verwaltung von SLAs, das eine eins-zueins-entsprechung von Job zu SLA vorsieht, wurde als hinderlich für die Nutzung von SLAs im HPC-Bereich identifiziert. Diese Arbeit stellt deshalb ein anderes SLA-Modell vor: die Verwendung von SLAs als stabile Langzeitverträge. Im Unterschied zu SLAs wie sie momentan verwendet werden zur Beschreibung von Qualitätsanforderungen eines einzelnen Jobs können mit Langzeit-SLAs Klassen von Qualitätsmerkmalen abgebildet werden, die dann von einzelnen Jobs referenziert werden können. Basierend auf den einvii gangs erwähnten Anforderungen wurde ein SLA-Verwaltungs-Framework für Langzeit- SLAs entworfen und implementiert und seine praktische Einsatzfähigkeit an Hand eines Szenarios demonstriert. Zusätzlich wurde, um die Analyse von SLA-basiertem Scheduling zu erleichtern, ein existierendes Simulationswerkzeug um Unterstützung für SLAs erweitert. Mit diesem Werkzeug wurden verschiedene Untersuchungen über den Einsatz von SLAs im HPC-Umfeld durchgeführt. Diese Arbeit hat Langzeit-SLAs als ein Werkzeug für das Anbieten von Dienstgüten im HPC-Bereich eingeführt und deren Ertragswert bestätigt. Langzeit-SLAs können in einem existierenden HPC-Umfeld mit Hilfe eines unkomplizierten SLA-Verwaltungs- Frameworks eingeführt werden. Dieses erlaubt eine einfache Integration in die verschieden Ebenen, von der Infrastruktur bis zu den Anwendungen. Die Problematik des Schedulings von Langzeit-SLAs kann hingegen nicht im Allgemeinen gelöst werden. Nichtsdestoweniger können für bestimme Situation, zum Beispiel verschiedene Dienstgüten die sich nur relativ in ihrer Priorität unterscheiden, allgemeine Richtlinien empfohlen werden. Im Wesentlichen müssen Implementierungen aber auf die von einem Anbieter angebotenen Dienstgüten hin angepasst werden. Dies wird durch die Entwicklung des oben angesprochenen Simulationswerkzeugs erleichtert, das die Simulation von verschiedenen Workloads, Infrastrukturen und Scheduling-Algorithmen ermöglicht. Zusammenfassend lässt sich sagen, dass eine Kombination aus Langzeit-SLAs und einem auf das nötigste reduzierten SLA-Verwaltungs-Framework HPC-Anbietern eine einfache Möglichkeit bietet, Dienstgüten anzubieten. Zukünftige Arbeiten können sich zum Beispiel mit der Verwendung von Anbieter-spezifischen Workloads und der Implementierung der SLA-Verwaltung auch auf Kundenseite beschäftigen. viii Contents List of Figures List of Tables Abbreviations xiii xv xvii 1 Introduction Objectives Chosen Approach Research Contribution Background A brief introduction to high performance computing and service level agreements High performance computing Job submission and scheduling Service Level Agreements Use Cases Scientific use case Virtual turbine use case Requirements Generic requirements Scientific use case Virtual turbine use case State of the Art High performance computing Job scheduling Service level agreements Specifying SLAs for job submission Using SLAs for job scheduling Managing SLAs Resource managers and job schedulers Portable Batch System derivatives Condor Load Sharing Facility LoadLeveler ix Contents Maui Cluster Scheduler and Moab Cluster Suite Oracle Grid Engine SLURM Grid MP Application Level Placement Scheduler Summary Simulation of HPC resources SimJava GridSim MicroGrid Bricks SimGrid Simbatch MONARC Grid Scheduling Simulator Alea Pyss - the Python Scheduler Simulator Summary Summary Service level agreements for HPC job submission and scheduling Service levels for high performance computing providers Long-term SLAs for HPC service levels Service levels for HPC Service level provisioning benefits HPC job submission and scheduling with SLAs Submission of HPC jobs related to SLAs Scheduling of jobs according to service levels An SLA management framework for HPC providers Evaluation Generic requirements Use case specific requirements Summary Implementation of SLA-enhanced job control Scenario Implementation architecture Job submission and scheduling with the Globus Toolkit and TORQUE Adding SLA-based job submission and scheduling Prototypical implementation Submitting a job related to an SLA Receiving SLA information on the provider side Securing the ManagedJobFactory service Queuing jobs in reference to an SLA Running the job x Contents Accounting and billing Evaluation Generic requirements Scenario specific requirements Summary Simulating SLA usage in an HPC environment The Alea scheduling simulator Features Shortcomings SLA-based scheduling with Alea Service level distributions for workloads Implementing scheduling with SLAs Experimental results Prioritised and best effort access Cancellation and resubmission of best effort jobs Influence of different cancellation policies Summary Conclusions and future work Conclusions Service level agreements for the HPC domain Service level agreement management for HPC environments Simulating SLA usage in an HPC environment Future work Refinement of simulation through feedback of real data SLA management in the client domain Analysing revenue changes for HPC providers xi Contents xii List of Figures 1.1 Layered architecture for HPC services Brokered SLA architecture [49] SLA management as seen by the BEinGRID project [2] SLA management as seen by the BREIN project [54] SLA management architecture proposed by the FinGrid project [55] SLAs in the IRMOS architecture [56] IRMOS service management [56] The SLA layer developed by SLA4D-Grid in the D-Grid SLA4D-Grid s SLA layer architecture PBS/TORQUE architecture The architecture of Bricks [66] SimGrid component overview [67] The architecture of GSSIM [71] High-level SLA management components WS-GRAM components [88] WS-GRAM protocol sequence [88] SLA selection and job submission using a GUI client Architecture of Alea The Alea architecture with the added SLA loader Histogram showing the percentage of jobs for different job sizes Average wait time increase for bronze jobs Average wait time for silver jobs Average wait time for bronze jobs Average wait time for silver jobs Wasted bronze core hours due to cancellations Silver SLA price to make up for revenue loss due to cancellations Wasted core hours versus job size for least percentage completed cancellation Wasted core hours versus job size for shortest job by core hours cancellation Wasted core hours versus job size for shortest job by time cancellation Silver SLA price versus job size for least percentage completed cancellation Silver SLA price versus job size for shortest job by core hours cancellation Silver SLA price versus job size for shortest job by time cancellation xiii List of Figures 6.16 Bronze average wait time versus job size for least percentage completed cancellation Bronze average wait time versus job size for shortest job by core hours cancellation Bronze average wait time versus job size for shortest job by time cancellation Silver average wait time versus job size for least percentage completed cancellation Silver average wait time versus job size for shortest job by core hours cancellation Silver average wait time versus job size for shortest job by time cancellation Feedback between simulation, contractual offerings and analysis of workload traces xiv List of Tables 2.1 Examples of current HPC computers as of November Fees and charges at the High Performance Computing Center Stuttgart [12] Batch queues at HLRS NEC SX-9 [15] Batch queues at TU Dresden s cluster Deimos [26] Batch queues at HLRS Nehalem cluster [27] Batch queues at FZJ s Blue Gene/L [28] Comparison of SLA management approaches Comparison of resource managers and job schedulers Comparison of simulation software for HPC Details for experiment Details for experiment Sample jobs showing the influence of different cancellation policies Details for experiment xv List of Tables xvi Abbreviations ALPS Application Level Placement Scheduler, page 38 API Application programming interface, page 40 ASCI Advanced Strategic Computing Initiative, page 19 BASIL Batch and Application Scheduler Interface Layer, page 38 BEinGRID Business Experiments in Grids, page 24 BLAST Basic local alignment search tool, page 23 BMBF German Federal Ministry of Education and Research, page 29 BMP Bitmap file format, page 80 BOINC Berkeley Open Infrastructure for Network Computing, page 36 BREIN Business objective driven reliable and intelligent grids for real business, page 17 CAD Computer-aided design, page 14 CAVE Cave virtual automatic environment, page 14 CERN European Organization for Nuclear Research, page 44 CIT California Institute of Technology, page 44 CPU Central processing unit, page 8 CSV Comma-separated values, page 80 DES Discrete event simulation, page 40 EDF Earliest deadline first, page 20 EPR Endpoint reference, page 66 EU European Union, page 45 FCFS First come, first served, page 20 xvii Abbreviations FinGrid Financial Business Grid, page 4 FLOPS Floating point operations per second, page 8 FLOSS Free/libre/open source software, page 106 FTP File transfer protocol, page 65 FZJ Forschungszentrum Jülich, page 18 GIF Graphics Interchange Format, page 80 GPL GNU General Public License, page 46 GPU Graphics processing unit, page 8 GRAM Grid Resource Allocation Manager, page 33 GSSIM Grid Scheduling Simulator, page 44 GT Globus Toolkit, page 18 GUI Graphical user interface, page 44 GWF Grid Workload Format, page 80 HLRS High performance computing center Stuttgart, page 9 HPC High performance computing, page 1 HPCWSAG High performance computing WS-Agreement, page 22 HTC High throughput computing, page 35 ICT Information and communications technology, page 24 IRMOS Interactive Realtime Multimedia Applications on Service Oriented Infrastructures, page 4 ISONI Intelligent service-oriented network infrastructure, page 28 JPEG Joint Photographic Experts Group, page 80 LAN Local area network, page 44 LGPL GNU Lesser General Public License, page 46 LHC Large Hadron Collider, page 44 LSF Load Sharing Facility, page 36 MONARC Models of Networked Analysis at Regional Centers, page 44 xviii Abbreviations MWF MetaCentrum Workload Format, page 80 NextGRID The Next Generation Grid, page 4 NQS Network Queuing System, page 19 OGE Oracle Grid Engine, page 37 PBS Portable Batch System, page 33 PDP Policy decision point, page 59 PEP Policy enforcement point, page 59 PKI Public key infrastructure, page 55 PNG Portable Network Graphics, page 80 PWF Pisa Workload Format, page 80 QoS Quality of service, page 1 RAM Random access memory, page 8 RMS Resource management system, page 10 RR Round-robin, page 20 RSL Resource specification language, page 69 SDM Service Domain Manager, page 37 SJF Shortest job first, page 20 SLA Service level agreement, page 1 SLA4D-Grid Service Level Agreements For D-Grid, page 4 SLAMF Service level agreement management framework, page 1 SLURM Simple Linux Utility for Resource Management, page 37 SOA Service-oriented architecture, page 65 SWF Standard workload format, page 46 TMF TeleManagement Forum, page 57 TORQUE Terascale Open-Source Resource and Queue Manager, page 10 TU Technical university, page 18 UNICORE Uniform Access over the Internet to Computing Resources, page 35 xix Abbreviations URL Uniform resource locator, page 69 VO Virtual organisation, page 23 VT Virtual turbine, page 14 WAN Wide area network, page 44 WS-Ag WS-Agreement, page 22 WSLA Web Service Level Agreements, page 22 WSRF Web Services Resource Framework, page 65 xx Acknowledgements No man is an island, entire of itself, as John Donne so pointedly stated in his Meditation XVII. When writing a thesis, it sometimes feels as one has taken an endeavour where this is untrue, but in fact it is not. There are many people that provide help and support and I am very glad to acknowledge the helped I received and that helped me to be abl
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks