학술논문

JanusAQP: Efficient Partition Tree Maintenance for Dynamic Approximate Query Processing
Document Type
Conference
Source
2023 IEEE 39th International Conference on Data Engineering (ICDE) ICDE Data Engineering (ICDE), 2023 IEEE 39th International Conference on. :572-584 Apr, 2023
Subject
Computing and Processing
Costs
Databases
Query processing
Maintenance engineering
Data engineering
Approximation error
Distance measurement
approximate query processing
partition tree
dynamic data structure
stratified sampling
partitioning
Language
ISSN
2375-026X
Abstract
Approximate query processing over dynamic databases, i.e., under insertions/deletions, has applications ranging from high-frequency trading to internet-of-things analytics. We present JanusAQP, a new dynamic AQP system, which supports SUM, COUNT, AVG, MIN, and MAX queries under insertions and deletions to the dataset. JanusAQP extends static partition tree synopses, which are hierarchical aggregations of datasets, into the dynamic setting. This paper contributes new methods for: (1) efficient initialization of the data synopsis in the presence of incoming data, (2) maintenance of the data synopsis under insertions/deletions, and (3) re-optimization of the partitioning to reduce the approximation error. JanusAQP reduces the error of a state-of-the-art baseline by more than 60% using only 10% storage cost. JanusAQP can process more than 100K updates per second in a single node setting and keep the query latency at a millisecond level.