Campus Units
Electrical and Computer Engineering
Document Type
Conference Proceeding
Conference
7th International Conference on Information Science and Applications (ICISA 2016)
Publication Version
Accepted Manuscript
Link to Published Version
https://doi.org/10.1007/978-981-10-0557-2_25
Publication Date
2-16-2016
Journal or Book Title
Information Science and Applications (ICISA) 2016
Volume
376
First Page
247
Last Page
257
DOI
10.1007/978-981-10-0557-2_25
Conference Title
7th International Conference on Information Science and Applications (ICISA 2016)
Conference Date
February 15-18, 2016
City
Ho Chi Minh City, Vietnam
Abstract
In this paper we propose and prove that cyclic quorum sets can efficiently manage all-pairs computations and data replication. The quorums are O(N/√P) in size, up to 50% smaller than the dual N/√P array implementations, and significantly smaller than solutions requiring all data. Implementation evaluation demonstrated scalability on real datasets with a 7x speed up on 8 nodes with 1/3rd the memory usage per process.
The all-pairs problem requires all data elements to be paired with all other data elements. These all-pair problems occur in many science fields, which has led to their continued interest. Additionally, as datasets grow in size, new methods like these that can reduce memory footprints and distribute work equally across compute nodes will be demanded.
Copyright Owner
Springer Science+Business Media Singapore
Copyright Date
2016
Language
en
File Format
application/pdf
Recommended Citation
Kleinheksel, Cory J. and Somani, Arun K., "Scaling Distributed All-Pairs Algorithms" (2016). Electrical and Computer Engineering Conference Papers, Posters and Presentations. 112.
https://lib.dr.iastate.edu/ece_conf/112
Comments
This is a post-peer-review, pre-copyedit version of a proceeding published as Kleinheksel, Cory J., and Arun K. Somani. "Scaling Distributed All-Pairs Algorithms." In: Kuinam J. Kim and Nikolai Joukov (eds.) Information Science and Applications (ICISA) 2016. Lecture Notes in Electrical Engineering 376 (2016): 247-257. The final authenticated version is available online at DOI: 10.1007/978-981-10-0557-2_25. Posted with permission.