| |
|
|
| |
Session: |
Tools and Services for Grids |
| |
Title: |
Optimizing Reduction Computations In a Distributed Environment |
| |
Chair: |
Roland Wismüller (LRR-TUM, Technische Universität München) |
| |
Time: |
Tuesday, November 18, 4:00PM - 4:30PM |
| |
Rm #: |
40-41 |
| |
Speaker(s)/Author(s): |
|
| |
Tahsin Kurc (Ohio State University), Feng Lee (Ohio State University), Gagan Agrawal (Ohio State University), Umit Catalyurek (Ohio State University), Renato Ferreira (Ohio State University), Joel Saltz (Ohio State University) |
| |
|
| |
Description: |
| |
We investigate runtime strategies for data-intensive applications that involve generalized reductions on large, distributed datasets. Our set of strategies includes replicated filter state, partitioned filter state, and hybrid options between these two extremes. We evaluate these strategies using emulators of three real applications, different query and output sizes, and a number of configurations. We consider execution in a homogenous cluster and in a distributed environment where only a subset of nodes host the data. Our results show replicating the filter state scales well and outperforms other schemes, if sufficient memory is available and sufficient computation is involved to offset the cost of global merge step. In other cases, hybrid is usually the best. Moreover, in almost all cases, the performance of the hybrid strategy is quite close to the best strategy. Thus, we believe that hybrid is an attractive approach when the relative performance of different schemes cannot be predicted.
This paper has been nominated for the Best Paper of SC2003 award. |
| |
Link: |
Download PDF |
| |
|
|