Department of Electrical and Computer Engineering
Information Infrastructure Institute

Example session

Here is an example of a typical session for a user on the Blue Gene. First we have a typical MPI application, in this case a program for computing π, called 'cpi'.

Source: cpi.c
Makefile: Makefile

First, we compile our application using a Makefile.

bgluser@cyblue:/gpfs01/users/bgluser> ls
cpi.c  Makefile

bgluser@cyblue:/gpfs01/users/bgluser> make
blrts_xlc -O2 -I. -I/bgl/BlueLight/ppcfloor/bglsys/include -o cpi cpi.c \
   -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts

Run on one of the on 128 nodes.

bgluser@cyblue:/gpfs01/users/bgluser> submit -n128 /gpfs01/users/bgluser/cpi

Our job ID is 6000. Let's look in the queue.

bgluser@cyblue:~ # status
JOBID   USER    STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
6000    bgluser RUN   normal     cyblue.ece. 128*bgsn.ec cpi        Jan 31 01:27
6001    sjnystr PEND   normal    cyblue.ece.             *asic.2.ll Jan 31 06:31

Note it takes about 60 seconds for a program to start executing after reaching the 'running' state in the queue. This is because the nodes need to boot first.

When the job is finished, program stdout goes to out.6000.  It will supply you with information about your job.

If you want to run a job with input, you need to use the bsub command.

bgluser@cyblue:/# source /bgl/lsf/conf/profile.lsf

For questions on different LSF commands refer to

http://home.eng.iastate.edu/~sjnystro/LSF%207.0.1/print/lsf_command_ref.pdf

bsub -n32 -i /gpfs02/users/bgluser/input -o /gpfs02/users/bgluser/out.%J mpirun /gpfs02/users/bgluser/cyblue-04

Here we can see that -i is for input and the -o is for output.  Be sure to use absolute paths, or else you will see errors in your output.