학술논문

Understanding the Performance of a Prototype of a WLCG Data Lake for HL-LHC
Document Type
Conference
Source
2018 IEEE 14th International Conference on e-Science (e-Science) E-SCIENCE e-Science (e-Science), 2018 IEEE 14th International Conference on. :332-333 Oct, 2018
Subject
Computing and Processing
Engineered Materials, Dielectrics and Plasmas
Fields, Waves and Electromagnetics
General Topics for Engineers
Geoscience
Signal Processing and Analysis
Lakes
Prototypes
Distributed databases
Europe
Organizations
Benchmark testing
Earth Observing System
HL-LHC
WLCG
storage
performance
distributed computing
Language
Abstract
Storage is identified as one of the main challenges for WLCG in the next decade in the computing strategy document for HL-LHC [1]. Extrapolating todays computing models, the ATLAS and CMS experiments alone would need one order of magnitude more storage resources than what could be provided by the funding agencies. Organization and consolidation of storage and evolution of the compute facilities will be central in addressing the possible resources shortage. In this contribution we describe the architecture of a prototype of a WLCG data lake for HL-LHC. A WLCG data lake aims to provide a geographically distributed storage service, distributed across large data centres interconnected by fast networks with low latency. We present methodology used to measure and understand the performance of a WLCG data lake prototype, in order to compare event throughput at the same cost at the same compute facilities backed by the traditional storage services and backed by the WLCG data lake. We will discuss various possible data processing models w.r.t. network latency, available storage media, and data caching approaches. We will present benchmarks for storage service and compute performance.