Posts for the month of May 2011

Played around with blog plugin

So this new blog plugin is pretty fun… I don't like that one has to create a blog shortname for each entry but I suggest that we all use our login name followed by the date in mmddyy format and then attach letters b, c, d if we make more than one post per day it should (for example this post is johannjc053111b since I made an entry earlier today. We could use more descriptive names, but since these are all in the same blog folder and since we will probably each make several entries each week - it is probably better to use the title to describe what the blog is about and keep the shortnames (URLs) simple. I don't like that we can't currently attach files to the blogs… But if we are working on a ticket, project page, or general documentation we can attach the images there and then link to them from the blog like so….

Discussion attachments

[attachment:AdvanceTimes_1.jpg:discussion:topic/23 SampleImage]

SampleImage

Or within the Image macro as so:

[[Image(discussion:topic/23:AdvanceTimes_1.jpg, 50%)]]

Wiki page attachments

[attachment:StrongScalingEfficiencies.png:wiki:OptimizingScrambler SampleImage]

SampleImage

Or within the Image macro as so:

[[Image(wiki:OptimizingScrambler:StrongScalingEfficiencies.png, 50%)]]

Results of strong scaling test on bluehive showing efficiencies

Ticket Attachments

[attachment:bug.png:ticket:126 TicketAttachment]

TicketAttachment

Or within the image macro as

[[Image(ticket:126:bug.png, width=50%)]]

field amr

Worked on Scrambler Paper

I've updated the scrambler paper with several new figures and attached it to the wiki page here ms.pdf

Worked on getting bear2fix on clover to read hdf5 files from bluegene

So I've been able to implement a hack in bear2fix on clover to access the runs from bluegene. This basically involved manipulating the byte ordering of the integers stored within the boxdata that was being read in. After considering the bit representation of 119 and 29 as well as the values it was reading from the chombo file (1996488704 and 486539264) it looked like the order of the bytes was actually reversed - not the order of the bits…

119 TTTFTTTF FFFFFFFF FFFFFFFF FFFFFFFF
1996488704 FFFFFFFF FFFFFFFF FFFFFFFF TTTFTTTF
29 TFTTTFFF FFFFFFFF FFFFFFFF FFFFFFFF
486539264 FFFFFFFF FFFFFFFF FFFFFFFF TFTTTFFF

I implemented a function that converts from bluegenes big endian byte order to clover's little endian byte order.

FUNCTION convertfrombluegene(a)
  INTEGER :: a, convertfrombluegene,i,j
  convertfrombluegene=0
  DO i=0, 3
     DO j=0,7
        if (btest(a,8*i+j))  convertfrombluegene=ibset(convertfrombluegene, 8*(3-i)+j)
     END DO
  END DO
END FUNCTION convertfrombluegene

Why is hdf5 having difficulty with these data types? The idea of using hdf5 is to avoid these type of conversion issues… In fact if you run h5dump on the file you can see that the fact that these integers are big endian is stored in the hdf5 file and that h5dump correctly reads there value. However since these integers are stored within a box container data type (which contains 6 integers because of chombo requirements) there values have to be read as a whole. (This is technically only true for attributes and not for datasets but there are box type attributes for which this is still a problem).

Possible solutions:

  • It looks like it may be possible to have hdf5 always convert values to little-endian before storing them - however this would cause problems if we ever wanted to do post processing of those data files on a big endian machine.
  • One could have bear2fix ask the user if the byte ordering of the box data needs to be reversed but this might be confusing to the naive user.
  • Perhaps the best solution would be to see if there is a way bear2fix to determine whether or not these box sub types are little endian or big endian and then check to see if it is running on a big endian or little endian machine.
    • (the latter is easy to do by just looking at the first bit of the integer 1 as it will be true for little endian machines)
    • The former could be :
      • guessed by looking at the last 16 bits to see if there are any 1's… But this would only work for datasets with dimensions less than 216 which is the current case - but won't be for much longer…
      • determined by having bear2fix probe the global.data file to get the global mX which it could then compare with the root level prob_domain attribute - if they didn't match it would know to convert the box data…
      • finding an hdf5 command that returns the data type of the box data types component fields… Since visit is able to properly read these data files on clover I assume that this is possible… although I'm not sure if visit needs any of the box data attributes or just the box data datasets.

So - after fixing this and looking at the output chombo file it was apparent that something was still wrong… A little more digging showed that the output of the box data was being byte reversed when creating new fixed grid chombos - when the data type was being recorded in the chombo file as little endian integers… So I had to add an lConvert global logical variable that was only turned on when reading in chombo files - and was turned off when outputting them. After fixing this I was still seeing crazy values for dx and xupper… It appears that the float attributes were also being read in reverse order. I wrote another routine for converting floats from big endian to little endian order:

FUNCTION convertfrombluegenefloat(a)
  REAL(8) :: a, convertfrombluegenefloat
  INTEGER(8) :: ai, converti
  INTEGER :: i,j
  converti=0
  ai=transfer(a,ai)   !Cast the 8 byte real as an 8 byte integer since btest and ibset require integers
  DO i=0, 8
     DO j=0,7
        if (btest(ai,8*i+j))  converti=ibset(converti, 8*(7-i)+j)
     END DO
  END DO
  convertfrombluegenefloat=transfer(converti,convertfrombluegenefloat)
END FUNCTION convertfrombluegenefloat