next previous contents
Next: Profile Visualization with Upshot Previous: Timing Programs

MPE Logging Routines

The logging routines in MPE are used to create logfiles of events that occur during the execution of a parallel program. These files can be studied after the program has ended successfully.

There are three ways to perform logging: automatic, manual, or a combination of the two.

In all three cases, the code must be compiled/linked with "mpicc -mpilog". If this doesn't produce the output you think it should, try omitting the "-mpilog" and compiling with "-lmpe" at the end of the mpicc command line instead (to manually link the MPE library). If upshot still doesn't give you the type of output you were expecting, eg only events on process 0 are shown, or only communication is logged and not the states of the processes (indicated by arrows between processors on the upshot gantt chart but an absence of coloured rectangles on the processor schedules), then it is probably something to do with your code. Go back and have a look at the cpilog.c example.

Automatic Logging

Automatic logging will log ALL MPI calls. To achieve this, the code should contain no calls to any MPE logging routines, but should just be compiled and linked with "mpicc -mpilog"

Manual Logging

The following routines allow the user to log events that are meaningful for specific applications, rather than relying on automatic logging of all MPI library calls. The basic routines are MPE_Init_log, MPE_Log_event, and MPE_Finish_log.

MPE_Init_log must be called by all processes to initialize MPE logging data structures. MPE_Finish_log collects the log data from all the processes, merges it, and aligns the timestamps with respect to the times at which MPE_Init_log and MPE_Finish_log were called. Then, the process with rank 0 in MPI_Comm_world writes the log into the file whose name is given as an argument to MPE_Finish_log.

A single event is logged with the MPE_Log_event routine. The routines MPE_Describe_event and MPE_Describe_state allow one to add event and state descriptions and to define states by specifying a starting and ending event for each state. For the specific syntax of these routines, you should consult the man pages.

Note: it would appear that calling MPE_Stop_log() causes problems. Your program may crash, spewing out lots of error information. Do not call this function in your code.

The following sample program (which is cpilog.c I think) demonstrates some of these logging routines. To compile it you will need to include the "-lmpe" flag on the mpicc command line (so that the mpe library is linked into the executable)

/* Sample Program with Logging Commands */
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "mpi.h"
#include "mpe.h"

double f(a)
double a;
{
    return (4.0 / (1.0 + a*a));
}

int main(argc,argv)
int argc;
char *argv[];
{
  int done = 0, n, myid, numprocs, i, rc, repeat;
  double PI25DT = 3.141592653589793238462643;
  double mypi, pi, h, sum, x, a;
  double startwtime, endwtime;

  MPI_Init(&argc,&argv);
  MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
  MPI_Comm_rank(MPI_COMM_WORLD,&myid);

  MPE_Init_log();

  /*  Get event ID from MPE, user should NOT assign event ID  */
  event1a = MPE_Log_get_event_number(); 
  event1b = MPE_Log_get_event_number(); 
  event2a = MPE_Log_get_event_number(); 
  event2b = MPE_Log_get_event_number(); 
  event3a = MPE_Log_get_event_number(); 
  event3b = MPE_Log_get_event_number(); 
  event4a = MPE_Log_get_event_number(); 
  event4b = MPE_Log_get_event_number(); 

  if (myid == 0)
  {
    MPE_Describe_state(event1a, event1b, "Broadcast", "red");
    MPE_Describe_state(event2a, event2b, "Compute",   "blue");
    MPE_Describe_state(event3a, event3b, "Reduce",    "green");
    MPE_Describe_state(event4a, event4b, "Sync",      "orange");
  }

  while (!done)
  {
    if (myid == 0) 
    {
      printf("Enter the number of intervals: (0 quits) ");
      scanf("0",&n);
      startwtime = MPI_Wtime();
    }
    MPI_Barrier(MPI_COMM_WORLD);
    MPE_Start_log();

    MPE_Log_event(event1a, 0, "start broadcast");
    MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);
    MPE_Log_event(event1b, 0, "end broadcast");
    
    if (n == 0)
      done = 1;
    else
    {
      for (repeat=5; repeat; repeat--)
      {
        MPE_Log_event(event4a,0,"Start Sync");
        MPI_Barrier(MPI_COMM_WORLD);
        MPE_Log_event(event4b,0,"End Sync");
        MPE_Log_event(event2a, 0, "start compute");
        h = 1.0 / (double) n;
        sum = 0.0;
        for (i = myid + 1; i <= n; i += numprocs)
        {
          x = h * ((double)i - 0.5);
          sum += f(x);
        }
        mypi = h * sum;
        MPE_Log_event(event2b, 0, "end compute");
        fprintf( stderr, "[0] mypi = 0.000000\n", myid, mypi );

        MPE_Log_event(event3a, 0, "start reduce");
	    MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);
        MPE_Log_event(event3b, 0, "end reduce");
        if (myid == 0)
        {
          printf("pi is approximately 0.0000000000000000, Error is 0.0000000000000000\n",pi,abs(pi - PI25DT));
          endwtime = MPI_Wtime();
          printf("wall clock time = 0.000000\n", endwtime-startwtime);	       
        }
      }
    }
  }
  MPE_Finish_log("cpilog");
  MPI_Finalize();
}

Combined Manual/Automatic Logging

It is possible to combine automatic logging with manual logging. The user must NOT call MPE_Init_log or MPE_Finish_log in the user program, but can call other MPE routines to define events, states, etc.