MPI Basics

I read the article "The Basics of MPI for Dummies" and I realized that a novice article could scare away.

Theory

Start over

At first, there was no single standard (API) for parallel computing, and programmers had to write architectural-specific code for each cluster. But, as you know, programmers are rational people and it was quickly decided to organize standards (the most famous are MPI , OpenMP).

MPI - Message Passing Interface. This is a specific API that cluster manufacturers implement in order to easily transfer programs from cluster to cluster without changing a single byte of the source code (!).
A parallel program should efficiently utilize computing power and communication environment. In MPI, all work on the distribution of the load on the nodes and the network falls on the programmer, and for maximum performance it is necessary to know the features of a particular cluster. MPI very elegantly solves the problem of network topology: there are communicator concepts - groups of processes that can be numbered according to the network topology (this is done using the function MPI_Cart_create , which allows you to specify any topology from the grid to the hypercube).

The expediency of parallelization

Some examples in the textbooks are very synthetic - they consider any series within the standard type (for example, double), which in practice is calculated in a time much less than that spent on initialization and transferring something over the network (calculating the number pi in double on two computers with Gigabit Ethernet is about two times slower than computing on one computer). However, MPI allows you to use multi-core processors (which for some reason many people forget), and between the cores the transfer speed is of a completely different order, so you always need to know the architecture and topology of the system .
')

Practice

You can write a lot about theory, but it is better to comprehend the theory in proportion to practice.
To get started, install some kind of MPI implementation on your computer. One of the most common implementations of MPI is MPICH ( MPI Chameleon ).

Installation

In ubunt it is installed in one line:

sudo apt-get install mpich2

Let's write a simple little program that doesn't do anything useful:

#include <stdio.h>
#include <mpi.h>

int main ( int argc, char* argv[])
{
int errCode;

if ((errCode = MPI_Init(&argc, &argv)) != 0)
{
return errCode;
}

int myRank;

MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

if (myRank == 0)
{
printf( "It works!\n" );
}

MPI_Finalize();
return 0;
}

* This source code was highlighted with Source Code Highlighter .

Compile this program:

mpicc -o test.bin ./test.c

Let's try to run:

mpirun ./test.bin

And (if you haven’t yet configured the mpd daemon), you’ll receive a message stating that the mpd daemon is not running.

mpiexec: cannot connect to local mpd (/tmp/mpd2.console_valery); possible causes:
1. no mpd is running on this host
2. an mpd is running but was started without a "console" (-n option)
In case 1, you can start an mpd on this host with:
mpd &
and you will be able to run jobs just on this host.
For more details on starting mpds on a set of hosts, see
the MPICH2 Installation Guide.

If you try to start mpd, it will say that there are no settings (why, in fact, the daemon does not start)

configuration file /home/valery/.mpd.conf not found
A file named .mpd.conf file must be present in the user's home
directory (/etc/mpd.conf if root) with read and write access
only for the user, and must contain at least a line with:
MPD_SECRETWORD=<secretword>
One way to safely create this file is to do the following:
cd $HOME
touch .mpd.conf
chmod 600 .mpd.conf
and then use an editor to insert a line like
MPD_SECRETWORD=mr45-j9z
into the file. (Of course use some other secret word than mr45-j9z.)


The secret word is needed only for connecting nodes. If we connect more computers, then it will be necessary to put MPICH on them and it will be necessary to add the node to the list of nodes, and also it will not be superfluous to configure the ssh connection using keys (to communicate with the nodes).
If everything is done correctly, we get something like this:

$ mpirun ./test.bin
It works!


MPI_Init is required for the call, since it initializes the MPI library.
MPI_COMM_WORLD - the identifier of the global communicator containing all processes.
MPI_Comm_rank - returns the identifier (number, rank) of the process within the specified communicator.

Why do we display only at the rank equal to 0? Simply, this process just corresponds to the default of the one that has access to the console of the terminal from which it was launched. We can use any other, but it’s just so convenient.

Instead of output

You can write a parallel program without having almost any knowledge of parallel programming, but writing effective programs is a time consuming process of choosing an algorithm and implementing it, fitting it to the system, and so on. But if you start writing simple little programs and at the same time read specifications and literature about computing systems (about their architecture, communication environments and so on), then with time,% username%, you will be able to subdue even such scary machines as those listed in top 500