부산대학교 도서관

Large-scale science experimental services based on high performance infrastructure, supporting multiple users’ request for simulation jobs in diverse sciences such as a large ion collider experiment (ALICE), compact muon solenoid (CMS), and next-generation sequencing (NGS), requires extremely high performance and scalability to store and manipulate a set of scientific experimental data from advanced research equipment. The current a large unified storage is composed of multiple storage type (e.g. TAPE, HDD, SSD, Cloud), as data farms have grown in terms of both their size and the number of file they contain. However, previously reported file systems which use a union mount such as UnionFS, AuFS, unifies several storages and provides a merged large single storage to handle a massive amount of files, have shown a lack of efficient write policy with regard to the variant of storage device type. It raising the strong needs for alternative policy to write files in large-scale unification file system with heterogeneous storage device. In this work, we propose an efficient file management framework for large-scale data farm. The suggested framework provides scalable storage and rapid I/O performance through a new write policy on unification file system. It can be efficiently used in science experimental data farm that is particularly consists of heterogeneous storage devices. Applying the proposed write policy to AuFS which has presented one of the multi-layered unification file systems, we show its strength in terms of data write performance for High Performance Computer (HPC) based science experimental services.