BluegeneBEAR

We want to explore what it will require to get a copy of AstroBEAR to compile and run on a BG/P system. The two issues are that 1) the machine architecture is not the same (e.g., they're 32-bit not 64-bit), and 2) the compiler versions are not (at present) the same.

We may find that the only way to maintain a stable, but viable, code base will be to split the SVN repository and pepper the code with # if BLUEGENE C preprocessor directives.

This page is intended to be a starting point for this discussion.

KY NOTE: As of 4/9/10, it looks like the code can be brought back to bluehive successfully. I would like to do more testing just to be sure, but I think I've seen it compile, run parallel, and restart OK.

Notes on code changes required for compiling and running on BG/P (SVN revision 510)

• Anything that uses LOC() to identify the start and end of it in memory requires declaration as a SEQUENCE.
• The compiler is very picky about the difference between ==/.eq. and .eqv.. .EQV. is for logicals, .EQ. for numerals.
• It doesn't want the pointer addresses to be declared on their own, only in the POINTER (pAddr,buf) declaration.
  • This might not be true. Seemed to be fixed after changing INT_PTR_KIND to 32-bit version.
• It doesn't like commas in read statements like READ(60),. The same goes for write statements.
• Doesn't know about ISNAN, replace with x .ne x or create a new PURE ELEMENTAL FUNCTION ISNAN(x) in globaldeclarations.f90.
• netlib.f requires explicit compilation with F77 compiler, a change to the makefile.
• Doesn't want qPrec to be redefined in cool.f90.
• Complains about using NOT() where .NOT.() should be.
• Picky about KINDs being the same for all arguments to MAX().
• In OPEN statements, position replaces access.
• Is less forgiving when declaring a (3,2) array; does not allow e.g. X = (/ 1,2,3,4,5,6 /), but requires X = RESHAPE( (/ 1,2,3,4,5,6 /), (/ 3,2 /) )
• Doesn't like iEntries = (/ 0:6 /), workaround is iEntries = (/ 0,1,2,3,4,5,6 /).
• Still doesn't like something in FixupParent. (See below)
• Requires COMMAND_ARGUMENT_COUNT in place of IARGC.
• Wants the C calls in hypreBEAR to have "_" appended.
  • Fixed by adding -qextname in the compilation instructions for hypreBEAR.f90 in Makefile.
• Doesn't like scalars that are part of integer arrays to be passed as the dimension argument to SUM.
  • Fixed by changing SUM(q(:,:,:,:,,dir(i)) to SUM(q(:,:,:,:,, DIM=dir(i))
• Picky about NAMELISTS being declared after execution statments. Just need to declare NAMELISTS in the specification part of subroutines/modules

FixupParent

The BG/P compiler had trouble for some reason with the following FORALL statement, and the other like it in InfoFieldUtils.f90::FixUpParent,

FORALL(ix=mT(1,1):mT(1,2),iy=mT(2,1):mT(2,2),iz=mT(3,1):mT(3,2))
        Parent%fixupflux4(as,ix,iy,iz,edge,1:nq) = &
        Parent%fixupflux4(as,ix,iy,iz,edge,1:nq) * &
        fact2*REAL(count(ix,iy,iz,1),KIND=qprec)
END FORALL    

The workaround was to deshinify this back into nested DO loops.