A FRAMEWORK ARCHITECTURE FOR SHARED FILE POINTER OPERATIONS IN OPEN MPI

Date

2013-05

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

MPI is a message passing interface that provides a distributed memory programming model for parallel computation. MPI-I/O is a parallel I/O library that is part of the MPI-2 specification. As an intermediate layer between the application and the file system, MPI-I/O is able to support features not directly enabled in the underlying file system. One such feature is a shared file pointer. A shared file pointer is a file pointer to an open file that is shared between the processes that opened the file.

The objective of this thesis is to develop a framework for shared file pointer operations in OMPIO for MPI-I/O, implement and evaluate various existing and new algorithms for shared file pointers, and to develop and evaluate a selection logic to decide which algorithm to use. Four algorithms are implemented in this thesis: the locked file, the shared memory, the individual file, and the additional process algorithms.

The results show that the shared memory algorithm is the fastest. Unfortunately, it can only be used if all of the processes are executing on a single node. The individual algorithm provides a good option when running on multiple nodes, but only supports write operations. The additional process algorithm can be run from multiple nodes and can be used on all file systems, but may not be supported on all environments due to the requirement of spawning an additional process.

Description

Keywords

MPI, MPI-I/O, Open MPI, OMPIO, Shared file pointers, Shared file pointer framework, Parallel I/O

Citation