학술논문
StreamFlow: Cross-Breeding Cloud With HPC
Document Type
Periodical
Author
Source
IEEE Transactions on Emerging Topics in Computing IEEE Trans. Emerg. Topics Comput. Emerging Topics in Computing, IEEE Transactions on. 9(4):1723-1737 Jan, 2021
Subject
Language
ISSN
2168-6750
2376-4562
2376-4562
Abstract
Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g., Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow.