Electrical and Computer Engineering
42nd International Conference on Very Large Data Bases
Link to Published Version
Journal or Book Title
Proceedings of the VLDB Endowment
September 5-9, 2016
New Delhi, India
Modern enterprises generate huge amounts of streaming data, for example, micro-blog feeds, financial data, network monitoring and industrial application monitoring. While Data Stream Management Systems have proven successful in providing support for real-time alerting, many applications, such as network monitoring for intrusion detection and real-time bidding, require complex analytics over historical and real-time data over the data streams. We present a new method to process one of the most fundamental analytical primitives, quantile queries, on the union of historical and streaming data. Our method combines an index on historical data with a memory-efficient sketch on streaming data to answer quantile queries with accuracy-resource tradeoffs that are significantly better than current solutions that are based solely on disk-resident indexes or solely on streaming algorithms.
© ACM, 2016 This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Proceedings of the VLDB Endowment 10, no. 4 (2016): 433-444. https://doi.org/10.14778/3025111.3025124
Singh, Sneha Aman; AT&T Labs; and Tirthapura, Srikanta, "Estimating quantiles from the union of historical and streaming data" (2016). Electrical and Computer Engineering Conference Papers, Posters and Presentations. 44.