학술논문

Key lessons in the efficient archive of small files to the CCLRC MSS using SRB
Document Type
Conference
Source
2005 IEEE International Symposium on Mass Storage Systems and Technology Local to Global Data Interoperability Local to Global Data Interoperability - Challenges and Technologies, 2005. :83-87 2005
Subject
Computing and Processing
Communication, Networking and Broadcast Technologies
Intersymbol interference
Throughput
Collision mitigation
Writing
Containers
Performance analysis
History
Robots
Software development management
Computer architecture
Language
Abstract
High volume data projects within the CCLRC and the wider UK academic community (ISIS, BADC, and BBSRC) are increasingly looking to implement access to a "limitless" data archive through an SRB infrastructure. This paper describes the recent development of SRB containers as an efficient solution to the "small file problem", and the benefits this has brought to these scientific communities as it opens up high volume archives and mass storage systems as vital components of SRB-based data management infrastructures. By tracing the development of container implementation into the Atlas petabyte data store (based at CCLRC in the UK), across three specific projects (CMS, BBSRC and especially ISIS), the paper identifies and describes key lessons learned, both for CCLRC's particular projects and also for archival systems in general.