Campus Units

Electrical and Computer Engineering

Document Type

Conference Proceeding

Conference

7th International Conference on Information Science and Applications (ICISA 2016)

Publication Version

Accepted Manuscript

Link to Published Version

https://doi.org/10.1007/978-981-10-0557-2_25

Publication Date

2-16-2016

Journal or Book Title

Information Science and Applications (ICISA) 2016

Volume

376

First Page

247

Last Page

257

DOI

10.1007/978-981-10-0557-2_25

Conference Title

7th International Conference on Information Science and Applications (ICISA 2016)

Conference Date

February 15-18, 2016

City

Ho Chi Minh City, Vietnam

Abstract

In this paper we propose and prove that cyclic quorum sets can efficiently manage all-pairs computations and data replication. The quorums are O(N/√P) in size, up to 50% smaller than the dual N/√P array implementations, and significantly smaller than solutions requiring all data. Implementation evaluation demonstrated scalability on real datasets with a 7x speed up on 8 nodes with 1/3rd the memory usage per process.

The all-pairs problem requires all data elements to be paired with all other data elements. These all-pair problems occur in many science fields, which has led to their continued interest. Additionally, as datasets grow in size, new methods like these that can reduce memory footprints and distribute work equally across compute nodes will be demanded.

Comments

This is a post-peer-review, pre-copyedit version of a proceeding published as Kleinheksel, Cory J., and Arun K. Somani. "Scaling Distributed All-Pairs Algorithms." In: Kuinam J. Kim and Nikolai Joukov (eds.) Information Science and Applications (ICISA) 2016. Lecture Notes in Electrical Engineering 376 (2016): 247-257. The final authenticated version is available online at DOI: 10.1007/978-981-10-0557-2_25. Posted with permission.

Copyright Owner

Springer Science+Business Media Singapore

Language

en

File Format

application/pdf

Published Version

Share

Article Location

 
COinS