GABB: the plan-based job scheduling optimized by genetic algorithm for HPC systems with shared burst buffers

Shuaizhe Lv; Xuchao Xie; Zhenlong Song

doi:10.1117/12.3034965

11 July 2024 GABB: the plan-based job scheduling optimized by genetic algorithm for HPC systems with shared burst buffers

Shuaizhe Lv, Xuchao Xie, Zhenlong Song

Author Affiliations +

Proceedings Volume 13210, Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024); 132101Y (2024) https://doi.org/10.1117/12.3034965
Event: Third International Symposium on Computer Applications and Information Systems (ISCAIS 2023), 2024, Wuhan, China

Abstract

Burst Buffer was proposed to address the growing performance gap between computation and I/O in high-performance computing systems (HPC). However, the introduction of Burst Buffer has brought new issues in system resource management and job scheduling. Resource management systems on HPC platforms lack efficient management of multiple types of resources, and the allocation of computing resources and Burst Buffer resources is independent of each other. This can lead to underutilization of system resources and job blocking. To address these issues, this paper proposes GABB, a plan-based job scheduling strategy for shared Burst Buffer, and utilizes genetic algorithms for optimization. GABB employs a plan-based job scheduling strategy to uniformly schedule all jobs in the waiting queue to generate an execution plan. Furthermore, GABB comprehensively considers changes in system resources and the job's demand for computing and Burst Buffer resources during the job scheduling process. Finally, GABB utilizes an improved genetic algorithm to optimize the job scheduling scheme. We conducted experimental simulations of a shared Burst Buffer system and implemented a plan-based job scheduling algorithm. The experimental results indicate that BBGA significantly reduces the mean waiting time of jobs by over 20% compared to the Shortest Job First (SJF) algorithm, and reduces the mean bounded slowdown of jobs by over 25%.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Shuaizhe Lv, Xuchao Xie, and Zhenlong Song "GABB: the plan-based job scheduling optimized by genetic algorithm for HPC systems with shared burst buffers", Proc. SPIE 13210, Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024), 132101Y (11 July 2024); https://doi.org/10.1117/12.3034965

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Computing systems

Genetic algorithms

Data storage

Mathematical optimization

Computer simulations

Genetics

Systems modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years