Efficient Parallel All-Pairs Computation Framework: using Computation - Communication Overlap
Date
Authors
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Authors
Research Projects
Organizational Units
Journal Issue
Is Version Of
Versions
Series
Department
Abstract
The advent of parallel computing systems enabled the users with huge computation power to efficiently process huge work loads. Most of the recent applications, which are data intensive, require parallel computing power to complete the job efficiently. To facilitate efficient computing there is a necessity for simplified abstraction of the parallel computing systems.
We propose one such parallel computation abstraction, designed to solve All-Pairs problems which fit the needs of several data intensive applications. All-Pairs problems require each data element to be paired with every other data element. This framework aims to address recurring problems of scalability, distributing equal workload to all nodes and reducing memory footprint. Our framework reduces memory footprint of All-Pairs problems, by reducing memory requirement from N/ sqrt(P) to 3N/P. A bioinformatics application is implemented to demonstrate the scalability (ranging up to 512 cores), redundancy management and speed up the performance of the framework(superlinear
speed up).