Changes between Version 23 and Version 24 of Scrambler

Timestamp:

05/16/11 14:42:46 (14 years ago)

Author:

Brandon Shroyer

Comment:

—

Scrambler

@@ -1 +1 @@
 = Scrambler =
-[[CollapsibleStart(Justification)]]
+[[CollapsibleStart(AstroBEAR 2.0 In Brief)]]
 
 The growing size of scientific simulations can no longer be accommodated simply by increasing the number of nodes in a cluster.  Completing larger jobs without increasing the wall time requires a decrease in the workload per processor (i.e., increased parallelism).  Unfortunately, increased parallelism often leads to increased communication time.  Minimizing the cost of this communication requires efficient parallel algorithms to manage the distributed AMR structure and calculations.
 
-AstroBEAR's strength lies in its distributed tree structure.  Many AMR codes replicate the entire AMR tree on each computational node.  This approach incurs a heavy communication cost as the tree continuously broadcasts structural changes to all processors.  AstroBEAR, on the other hand, only keeps as much tree information as the local grids need to communicate with their neighbors.  This approach saves us memory usage as well as communication time, leaving us well-positioned to take advantage of low-memory architectures such as !BlueGene systems and GPUs.
+AstroBEAR's strength lies in its distributed tree structure.  Many AMR codes replicate the entire AMR tree on each computational node.  This approach incurs a heavy communication cost as the tree continuously broadcasts structural changes to all processors.  AstroBEAR, on the other hand, only keeps as much tree information as the local grids need to communicate with processors containing nearby grid regions.  This approach saves us memory usage as well as communication time, leaving us well-positioned to take advantage of low-memory architectures such as !BlueGene systems and GPUs.
 
+{{{
+#!comment
 Additionally Scrambler uses a distributed control structure that mirrors the nested AMR grid hierarchy.  Processors have child processors just as AMR grids have child grids.  These child processors receive new grids, their own new child processors, and all necessary tree information from their parent.  This eliminates the need for ANY global communication.  Processors only need to communicate with parent processors(processors containing parent grids), neighbor processors (processors containing adjacent grids), overlapping processors (processors containing previous AMR grids that overlap with the processors current grids), and child processors (processors assigned to child grids).  This does present a challenge for load balancing – since the total current workload for any given level can only be determined through collective communications after every grid has created children.  However, since regions of refinement in AMR simulations typically change slowly – it is possible to use the previous regions of refinement to predict the future regions of refinement and the amount of resources that should be allocated for any given region.  This allows the distribution to be parallelized as well.  Additionally the allocation of resources among child grids is done using a Hilbert space filling curve.  This allows neighboring processors to be physically close on the network (or on the same core) and allows Scrambler to take advantage of the network topology.
 
  In addition to being completely parallelized between processors, Scrambler makes use of threading to allow for parallel advancing of multiple AMR levels.  This allows for total load balancing instead of balancing each level of the AMR hierarchy.  This becomes especially important for deep simulations (simulations with low filling fractions but many levels of AMR) as opposed to shallow simulations (high filling fractions and only a few levels of AMR).  Processors with coarse grids can advance their grids simultaneously while processors with finer grids advance theirs.  Without this capability, base grids would need to be large enough to be able to be distributed across all of the processors.  For simulations with large base grids to be able to finish in a reasonable wall time, only a few levels of AMR can be used.  With threading this restriction is lifted.
 [[Image(threading2.png)]]
-
+}}}
 [[CollapsibleEnd()]]