Introduction to Parallel Debugging Tools

1. Introduction

Why do we need parallel debugging tools?

• The size and complexity have increased in proportion to the speed and architectures of parallel computers.
  • the mass of information in execution
  • the complexity of interactions
• Adding debugging sentences inside the code itself is tedious and may cause new bugs in the process of detecting the old ones.
• Deadlock problem.
• Non-determinism, parallel environment errors.

Limitations of parallel debugging tools

• Debugging is only within the range covered by the specialized commands designed in a debugging tool.
• The commands may be different in different debugging tools.

Commonly encountered problems when porting parallel code

• Parallel environment variables set-up
• Use of existing parallel libraries
  • Version
  • Syntax difference
  • Configuration
• Difference of Compiler options
• Platform-dependent implementations
  • Be cautious when using the MPI_Barrier on SP
  • smp mode (e.g., xlf_r -qsmp)
• Messed up of data-types
• Deadlock

On the IBM machines

Tools	Description	Object
TotalView	GUI-based debugging tool	MPI, OpenMP, FORTRAN, C, C++ programs
pdbx	command-line debugger	MPI, FORTRAN, C programs
pedb	X-window interface	MPI, FORTRAN, C programs
xprofiler	GUI-based performance profiling tool	MPI, FORTRAN, C programs

On the Origins

Tools	Description	Object
TotalView	GUI-based debugging tool	MPI, OpenMP, FORTRAN, C, C++ programs
SpeedShop	an integrated package of performance tools	FORTRAN, C programs
CVD	SGI ProDev WorkShop debugger	MPI, OpenMp, SGI directives

On the Linux Cluster

Tools	Description	Object
TotalView	GUI-based debugging tool	MPI, OpenMP, FORTRAN, C, C++ programs

TotalView has advanced support for C/C++ and F90, and understands such constructs as F90 modules and nested C++ templates. It has help for those who use complex objects like non-native types: through the new type mapping facility one can display these complex objects.

TotalView is a good choice for those working with parallelism or large amounts of data because it scales transparently to support the big code and data sets running with a large number of processes or processors. It's available on a wide variety of UNIX and Linux platforms (SP, Origin and Netfinity Cluster in the Supercomputing Institute).

To use pdbx for interactive debugging you first need to compile the C or Fortran 77 program and set up the execution environment as you would to invoke a parallel program with the poe command. Your program should be compiled with the -g flag in order to produce an object file with symbol table references. It is also advisable to not use the optimization option, -O. Using the debugger on optimized code may produce inconsistent and erroneous results. For more information on the -g and -O compiler options, refer to their use on other compiler commands such as cc and xlf.

Xprofiler is a GUI based performance profiling tool distributed as part of the IBM Parallel Environment for AIX. It can be used to graphically identify which functions are the most CPU intensive in your code. It provides a graphical function call tree as well as a text profile pertaining to your code. Xprofiler can be used to profile sequential and parallel C, C++, Fortran 90, Fortran77 and HPF programs.

To use Xprofiler, you first compile and link your program to ensure that profiling is enabled, then run the program to produce gmon.out file(s) (one for each processor involved in the execution) and finally invoke the xprofiler utility to display the profiling information.

Xprofiler provides CPU (busy) data only. It cannot be used to provide information such as I/O or communication data.