Campus Units

Mechanical Engineering, Electrical and Computer Engineering

Document Type

Article

Publication Version

Published Version

Publication Date

2015

Journal or Book Title

Scientific Programming

Volume

2015

First Page

180214

Abstract

Dimensionality reduction refers to a set of mathematical techniques used to reduce complexity of the original high-dimensional data, while preserving its selected properties. Improvements in simulation strategies and experimental data collection methods are resulting in a deluge of heterogeneous and high-dimensional data, which often makes dimensionality reduction the only viable way to gain qualitative and quantitative understanding of the data. However, existing dimensionality reduction software often does not scale to datasets arising in real-life applications, which may consist of thousands of points with millions of dimensions. In this paper, we propose a parallel framework for dimensionality reduction of large-scale data. We identify key components underlying the spectral dimensionality reduction techniques, and propose their efficient parallel implementation. We show that the resulting framework can be used to process datasets consisting of millions of points when executed on a 16,000-core cluster, which is beyond the reach of currently available methods. To further demonstrate applicability of our framework we perform dimensionality reduction of 75,000 images representing morphology evolution during manufacturing of organic solar cells in order to identify how processing parameters affect morphology evolution.

Comments

This is an article published as Samudrala, Sai Kiranmayee, Jaroslaw Zola, Srinivas Aluru, and Baskar Ganapathysubramanian. "Parallel framework for dimensionality reduction of large-scale datasets." Scientific Programming 2015 (2015). Posted with permission.

Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.

Copyright Owner

Sai Kiranmayee Samudrala et al.

Language

en

File Format

application/pdf

Share

COinS