Installation

NOTE: This installation help is out of date, and will not apply to the alpha test release of the MPIBLACS. You should examine the paper:
Installing and Testing the BLACS,
for more up-to-date information

Directory Structure

By default, the the following structure is assumed for the BLACS and their tester:

If you have downloaded only the tester, you will only receive the BMAKES and TESTING directories. If you download only the BLACS, you will receive the BMAKES, SRC and LIB directories. If both are downloaded, the top level directory tree will be complete.

The BMAKES directory contains several example Bmake.inc files, which may be of help configuring the makefiles to your system. Bmake.inc is included by all BLACS makefiles; more details about this file is given below Having an example Bmake.inc does not mean you can avoid examining the Bmake.inc. For instance, if you are using PVM on a HP machine, you can copy BMAKES/Bmake.hppa and have most of the work done for you. You may still need to modify the Bmake.inc to adapt it to your system and needs; for instance you may have compiled your PVM at a different location than is standard. Conversely, the lack of an example Bmake.inc in no way implies that the BLACS will not run on your system.

The SRC directory contains the BLACS source codes. The subdirectories under SRC indicate the various message passing libraries the BLACS are presently supported on. CMMD is the message passing library for Thinking Machine's CM-5, MPL runs on IBM's SP series, and NX runs on the Intel family. MPI runs across most UNIX platforms. PVM runs across most UNIX platforms.

The BLACS libraries will be built into LIB.

The TESTING directory contains the tester and its related files. On all systems except PVM, the executable will be compiled into the BLACS/TESTING/EXE directory. On PVM, the executable defaults to $(HOME)/pvm3/bin/<ARCH>. The BLACS/TESTING/EXE directory also includes sample input files for the tester.

Unpacking

The name of the tar file you receive will depend on what version of the BLACS you have downloaded. The naming structure is : <communication library>blacs.tar.gz. The communication library may be cmmd, mpl, nx, pvm or mpi. We will use pvm as an example in the following instructions.

The unpacking can be accomplished by typing gunzip -c pvmblacs.tar.gz | tar xvf - .. This will create the BLACS directory and the previously mentioned directory tree if does not already exist. If you have already installed the tester or another version of the BLACS, make sure you are in the directory containing the BLACS directory you wish to update.

Customizing the BLACS

The BLACS have several options which allow them to be customized. The easiest way to vary these options is by using section 2 of the Bmake.inc file.

The most important option is the BLACS debug level (BlacsDebugLvl). This pre-compiler option indicates how much error checking the BLACS should do. The default is to compile with BlacsDebugLvl=0, i.e., no error checking. This produces the fastest code. When a code is being developed, users are strongly encouraged to compile a BLACS with BlacsDebugLvl=1, however. This level of debugging checks input arguments for validity, does extra checking when a context is formed, etc. It should not result in code that it too much slower, and it is much more helpful when problems occur.

Most other parameters are system dependent.

Customizing the CMMD BLACS

You need to specify where your CMMD header files are stored, using the SYSINC macro. The BLACS perform some internal timings, and thus they need a CMMD node timer which they can use. You may vary this by changing TIMERNUM.

Customizing the NX BLACS

The user may select the range of force type message IDs the BLACS use. This is done with the FORCETYPE macro. By default the BLACS use the entire force type message ID range. This range can safely be restricted to only two message IDs. If BeginForceType is defined to be greater than or equal to EndForceType, force type messaged will not be used by the BLACS.

Customizing the PVM BLACS

There are several option available to the PVM user. The location of the pvm.h is specified by the SYSINC macro. If an example Bmake.inc does not exist for the system the BLACS are being compiled for, the user must indicate how a fortran 77 routine calls a C routine on that platform. The BLACS provide for three different protocols, which are selected with the makefile macro INTFACE. The values of INTFACE, and their affect on how fortran 77 calls C are shown in the following table. 
INTFACE          FORTRAN CALL        C DECLARATION
----------       ------------        ------------------
-DAdd_           CALL DGESD2D        void dgesd2d_(...)
-DNoChange       CALL DGESD2D        void dgesd2d(...)
-DUpCase         CALL DGESD2D        void DGESD2D(...)
The UT BLACS provide two undocumented timer routines which return CPU and wall time (dcputime00 and dwalltime00). The CPU timer defaults to using the UNIX timer getrusage. Since the UNIX on the HP machines does not possess this function, the BLACS also support the use of the UNIX function times. The selection of which timer to use is accomplished by varying the WHICHTIMER make macro. If it is empty, then the default getrusage will be used. If the user wished to use times instead, he should set WHICHTIMER = -DUseTIMES.

The final PVM customization varies the way the BLACS handle output. By default, the BLACS use the routine pvm_catchout, which causes all output to be printed to the screen. This requires PVM to generate more messages, and if the spawning process dies output may be lost. If the user desires, he may turn off this feature so that output is dumped to the pvm log files /tmp/pvml.<uid> on each machine. This is done by defining the makefile macro CATCHOUT to -DBLACSNoCatchout.

Compiling

The first step is to modify the BLACS make include file Bmake.inc to match your system. This file sets up various macros needed for make and the BLACS. Bmake.inc is roughly split into three sections. Section 1 defines the macros necessary to find/name the various libraries. Section 2 sets up internals in the BLACS (as discussed above). Section 3 defines macros dealing with compilers, linker/loaders, etc. The directory BLACS/BMAKES contains examples of Bmake.inc's for various systems. This, together with the internal comments, should enable the user to adapt one of the Bmake's to his system.

Compilation may be accomplished at the BLACS level by typing make pvm. By default both the C and fortran callable libraries will be built. If the user knows he will be calling the BLACS strictly from fortran, he can build the fortran BLACS only by make pvm what=f77lib. To build only the C interface BLACS, the user would type make pvm what=Clib. Removal of object files can be accomplished by make pvm what=clean. The makefile itself will give further help if make is typed with no arguments.

NOTE: when the BLACS are are archived into library format, the archiver will probably report that it is truncating some long file names. This should not cause an error, and is normal behavior.