{"id":1350,"date":"2013-11-03T10:59:26","date_gmt":"2013-11-03T09:59:26","guid":{"rendered":"http:\/\/htor.inf.ethz.ch\/blog\/?p=1350"},"modified":"2013-11-03T11:05:37","modified_gmt":"2013-11-03T10:05:37","slug":"advanced-mpi-programming-tutorial-at-supercomputing-2013","status":"publish","type":"post","link":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/2013\/11\/03\/advanced-mpi-programming-tutorial-at-supercomputing-2013\/","title":{"rendered":"Advanced MPI Programming Tutorial at Supercomputing 2013"},"content":{"rendered":"

Pavan Balaji, Jim Dinan, Rajeev Thakur and I are giving our Advanced MPI Programming tutorial<\/a> at Supercomputing 2013 on Sunday November 17th.<\/p>\n

Are you wondering about the new MPI-3 standard? How it affects you as a scientific or HPC programmer and what nice new features you can use to make your life easier and your application faster? Then you should not miss our tutorial.<\/p>\n

<\/center><\/p>\n

Our abstract summarizes the main topics:<\/p>\n

\nThe vast majority of production parallel scientific applications today use MPI and run successfully on the largest systems in the world. For example, several MPI applications are running at full scale on the Sequoia system (on ?1.6 million cores) and achieving 12 to 14 petaflops\/s of sustained performance. At the same time, the MPI standard itself is evolving (MPI-3 was released late last year) to address the needs and challenges of future extreme-scale platforms as well as applications. This tutorial will cover several advanced features of MPI, including new MPI-3 features, that can help users program modern systems effectively. Using code examples based on scenarios found in real applications, we will cover several topics including efficient ways of doing 2D and 3D stencil computation, derived datatypes, one-sided communication, hybrid (MPI + shared memory) programming, topologies and topology mapping, and neighborhood and nonblocking collectives. Attendees will leave the tutorial with an understanding of how to use these advanced features of MPI and guidelines on how they might perform on different
\nplatforms and architectures.\n<\/p><\/blockquote>\n

This tutorial is about advanced use of MPI. It will cover several advanced features that are part of
\nMPI-1 and MPI-2 (derived datatypes, one-sided communication, thread support, topologies and topology
\nmapping) as well as new features that were recently added to MPI as part of MPI-3 (substantial additions
\nto the one-sided communication interface, neighborhood collectives, nonblocking collectives, support for
\nshared-memory programming).<\/p>\n

Implementations of MPI-2 are widely available both from vendors and open-source projects. In addition,
\nthe latest release of the MPICH implementation of MPI supports all of MPI-3. Vendor implementations
\nderived from MPICH will soon support these new features. As a result, users will be able to use in practice
\nwhat they learn in this tutorial.<\/p>\n

The tutorial will be example driven, reflecting scenarios found in real applications. We will begin with
\na 2D stencil computation with a 1D decomposition to illustrate simple Isend\/Irecv based communication.<\/p>\n

We will then use a 2D decomposition to illustrate the need for MPI derived datatypes. We will introduce
\na simple performance model to demonstrate what performance can be expected and compare it with actual
\nperformance measured on real systems. This model will be used to discuss, evaluate, and motivate the rest
\nof the tutorial.
\nWe will use the same 2D stencil example to illustrate various ways of doing one-sided communication in
\nMPI and discuss the pros and cons of the different approaches as well as regular point-to-point communica-
\ntion. We will then discuss a 3D stencil without getting into complicated code details.
\nWe will use examples of distributed linked lists and distributed locks to illustrate some of the new ad-
\nvanced one-sided communication features, such as the atomic read-modify-write operations.
\nWe will discuss the support for threads and hybrid programming in MPI and provide two hybrid ver-
\nsions of the stencil example: MPI+OpenMP and MPI+MPI. The latter uses the new features in MPI-3 for
\nshared-memory programming. We will also discuss performance and correctness guidelines for hybrid pro-
\ngramming.<\/p>\n

We will introduce process topologies, topology mapping, and the new \u201cneighborhood\u201d collective func-
\ntions added in MPI-3. These collectives are particularly intended to support stencil computations in a scalable
\nmanner, both in terms of memory consumption and performance.
\nWe will conclude with a discussion of other features in MPI-3 not explicitly covered in this tutorial
\n(interface for tools, Fortran 2008 bindings, etc.) as well as a summary of recent activities of the MPI Forum
\nbeyond MPI-3.<\/p>\n

Our planned agenda for the day is<\/p>\n

    \n
  1. Introduction (8.30\u201310.00)\n