학술논문

Seamless management of ensemble climate prediction experiments on HPC platforms
Document Type
Conference
Source
2016 International Conference on High Performance Computing & Simulation (HPCS) High Performance Computing & Simulation (HPCS), 2016 International Conference on. :895-900 Jul, 2016
Subject
Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Computational modeling
Atmospheric modeling
Monitoring
Earth
Data models
Weather forecasting
Earth System Modelling
Climate
Weather
High Performance Computing
Workflow manager
Language
Abstract
One of the main challenges for the weather, air quality and climate science is how to efficiently perform large numbers of simulations of the Earth system on a variety of supercomputers. In particular, the climate community has developed complex computational systems to obtain climate projections and predictions. A huge amount of computational resources are needed to produce these simulations, as well as to deal with the data coming in and out from the models. Regardless of the platform, climate simulations typically consist of hundreds of programs or scripts whose workflow can be complex. In this paper, Autosubmit, a Python-based tool that allows creating, launching and monitoring climate experiments, is introduced. The experiment is defined as a sequence of jobs that Autosubmit remotely submits and manages in a transparent way to the user. The same experiment can run in more than one supercomputing platform and for different workflow configurations. Autosubmit could be expanded to perform any weather, air quality and climate experiment on any computing platform to ensure the efficient handling of highly-dependent jobs, an optimum use of available computing resources, and a user-friendly management of the experiments, including creation, documentation, start, stop, restart, live monitoring and reproduction.