학술논문

A Decoupled Data Pipeline and its Reliability Assessment: Case Study in Extreme Climatic Humans Studies
Document Type
Conference
Source
2021 IEEE International Conference on Big Data (Big Data) Big Data (Big Data), 2021 IEEE International Conference on. :3075-3084 Dec, 2021
Subject
Communication, Networking and Broadcast Technologies
Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Performance evaluation
Cloud computing
Soft sensors
Scalability
Pipelines
Big Data
Real-time systems
Big Data Pipeline
Decoupled Big Data Services
Edge and Cloud Computing Services
Reliable Computing Pipeline
Language
Abstract
Data platforms with an ability to capture, process and analyze high frequency streaming data from vast complex systems reliably with high scalability is the central problem in this paper. Particularly when developed data platforms are to be deployed within extreme climatic conditions. We have developed a decoupled data pipeline by fusing multiple services within edge and cloud computing paradigms following design principles in software reliability to provide optimal services during vast human studies in extreme climatic simulations. The data pipeline is demonstrated through a pre-deployment acclimation case study in climatic chambers. Performance evaluation of each service within the developed pipeline shows high levels of reliability and availability across multiple testing and study simulations. As a result, real-time data from multiple complex data sources are made available for on time personalized analytics for analysis of physiological responses to austere environmental conditions.