Report

Roter Faden

• We try to solve a distributed outer product
• First attempt:
  • Naive allgather
  • Naive allreduce
  • -> Works decently well
• Second attempt
  • Look at various implementations of the decision tree
  • Optimize them, specifically targeted at our initial problem
  • -> Successful mainly for both ring (attempt to beat native-ring) and g-rabenseifner
• Future work
  • Push to upstream MPI implementation (if possible)
  • Take network topology into consideration

Table of Content

Abstract

• What do we want to solve
• Why is it a problem
• How do we solve it

Introduction

• Start from birds-eye view (gradient descent in Neural Networks)
• Zoom into the detailed problem (send around chunks of data)

Background

• Concept of {gather, reduce} (not the MPI function but instead the concept)
• MPI Library
  • allgather
  • allreduce
  • decision tree (explanation for the functions?)

Related Work

• Rabenseifner?

Implementation

• Basic implementation
  • Allgather
  • Allreduce
• Ring
• g-rabenseifner-allgather

Experimental Results

• Listing results
• reasoning and interpretation of results (in detail)

Conclusion

• some general conclusions of results in context of the whole problem statement and project.
  • what are some possible problems with current implementation? how could they be rectified?

Future Work

References