| 328 | | == Data == |
| 329 | | There are essentially four basic data routines that involve sharing of data between grids |
| 330 | | * !ProlongateParentsData - Parent to Child (Inter-Level) |
| 331 | | * !ApplyChildrenData - Child to Parent (Inter-Level) |
| 332 | | * !ApplyOverlaps - Old Grids to Current Grids (Intra-Level) |
| 333 | | * !SyncFluxes - Current Grids to Current Grids (Intra-Level) |
| 334 | | For parallel applications this requires some degree of communication. In order to overlap computation with communication, it is good to post the sends as soon as the data is available - and to do as much computation as possible until having to wait for the receives to complete. When the sends are checked for completion and when the receives are first posted is somewhat arbitrary. It is reasonable to post the receivites before you expect the sends to post and to complete the sends sometime after you expect the receives to have finished. |
| 335 | | |
| 336 | | For each operation there is likely to be a degree of local sharing between grids. The basic approach therefore is to post the receives followed by the sends. Then perform the local sharing before waiting on the receives to complete, and then the sends. Sometimes the posting of the receives is shifted earlier, and the completion of the sends is put off until later. For example the parallel version of !ApplyOverlaps is |
| | 328 | |
| | 329 | Communication in AstroBEAR comes in two varieties: ''data'' communication, which is concerned with passing actual grid data between processors, and ''tree'' communication, which passes AMR tree data. |
| | 330 | |
| | 331 | [[BR]] |
| | 332 | === Data Communication === |
| | 333 | There are four basic data routines that involve sharing of data between grids: |
| | 334 | * {{{ProlongateParentsData()}}} -- Parents to children (Inter-Level) |
| | 335 | * {{{ApplyChildrenData()}}} -- Children to parents (Inter-Level) |
| | 336 | * {{{ApplyOverlaps()}}} -- Old Grids to current grids (Intra-Level) |
| | 337 | * {{{SyncFluxes()}}} -- Current Grids to current grids (Intra-Level) |
| | 338 | |
| | 339 | This description of communication is from a data domain perspective, though, and does not take interprocess communications into account. In a parallel application, the routines above will pass data to other processors as well as moving it between local grids. |
| | 340 | |
| | 341 | AstroBEAR's point-to-point communications are all non-blocking. Once it knows the data requirements, the code uses {{{Post}}} routines set up a level's transmissions, and later on it uses {{{Comp}}} routines to block until the required transmissions are done (see the example below). To ensure the best possible performance, we want to post the data sends as soon as the data is available and wait until we absolutely need the data before completing the receives. This keeps the processors busy crunching numbers and swapping data between local grids while they wait for data to arrive. Completion of sends and posting of receives is a less precise art, but in general early posts and delayed receives lead to better performance. |
| 347 | | |
| 348 | | == Tree == |
| 349 | | In a similar manner there are five tree operations that require some communication between nodes |
| 350 | | * !CreateChildren |
| 351 | | * !InheritNeighborsChildren |
| 352 | | * !InheritOldNodeOverlapsChildren |
| 353 | | * !InheritNewNodeOverlapsChildren |
| 354 | | * !InheritOverlapsOldChildren |
| 355 | | * !InheritOverlapsNewChildren |
| | 352 | [[BR]] |
| | 353 | === Tree Communication === |
| | 354 | |
| | 355 | There are five tree operations that require some communication between processors: |
| | 356 | * {{{CreateChildren()}}} |
| | 357 | * {{{InheritNeighborsChildren()}}} |
| | 358 | * {{{InheritOldNodeOverlapsChildren()}}} |
| | 359 | * {{{InheritNewNodeOverlapsChildren()}}} |
| | 360 | * {{{InheritOverlapsOldChildren()}}} |
| | 361 | * {{{InheritOverlapsNewChildren()}}} |
| | 362 | |
