학술논문

Trellis for efficient data and task management in the VA Million Veteran Program
Document Type
article
Source
Scientific Reports, Vol 11, Iss 1, Pp 1-12 (2021)
Subject
Medicine
Science
Language
English
ISSN
2045-2322
Abstract
Abstract Biomedical studies have become larger in size and yielded large quantities of data, yet efficient data processing remains a challenge. Here we present Trellis, a cloud-based data and task management framework that completely automates the process from data ingestion to result presentation, while tracking data lineage, facilitating information query, and supporting fault-tolerance and scalability. Using a graph database to coordinate the state of the data processing workflows and a scalable microservice architecture to perform bioinformatics tasks, Trellis has enabled efficient variant calling on 100,000 human genomes collected in the VA Million Veteran Program.