Introduction

• Distributed Tree/Control Algorithm
  • Many if not all current AMR codes opt to store the entire AMR structure on each processor. This however requires additional memory as well as communication. (7 floats per patch compared to 256 floats for the data (for 2D 4x4 hydro with 2 ghost zones). While this additional memory requirement is negligible for problems run on typical cpus on 100's of processors, it can be become considerable on 1000's or 10000's of processors. Since it is expected that efficient memory use and management will be required for HPC down the road, we implemented a distributed tree algorithm in which each processor is only aware of the local tree. Each processor after creating its children, decides which surrounding processors needs to be notified of it's new children. This sharing of children happens between grids that are physically neighboring in space, as well as previous or future grids that overlap in time. Each grid then receives new neighboring/overlapping children and then checks to see which of them neighbor/overlap its own children. Then when distributed its children to children processors it includes the child's neighbors/overlaps so that the child processor is aware of its surrounding tree.

• Threaded MultiLevelAdvance
  • Many if not all current AMR codes tend to perform grid updates across all levels in a prescribed order (0, 1, 2, 2, 1, 2, 2, 0…) Good performance then requires each level update to be balanced across all processors. Load balancing each level however, often leads to artificial fragmentation of grids and requires each processor to have grids on every level. This however, limits the depth of the AMR tree and the resulting simulations are often performed with large base grids with only 1 or 2 levels of AMR. The benefit of AMR however, is best utilized for scenarios with large dynamic ranges - and having only 2 levels of AMR severely limits the dynamic range that can be followed. In AstroBEAR we allow each levels grid advances to happen independently so that processors with coarse grids can update those grids while other processors with ine grids update theirs…

• Load Balancing

• Sweep method

Performance