Understanding AstroBEAR's standard out
When you start a simulation the first thing you'll see is AstroBEAR's banner - the number of cores you are running on
============================================================================== | _ _ ____ _____ _ ____ ____ ___ | | / \ ___| |_ _ __ ___ | __ )| ____| / \ | _ \ |___ \ / _ \ | | / _ \ / __| __| '__/ _ \| _ \| _| / _ \ | |_) | __) | | | | | | | / ___ \\__ \ |_| | | (_) | |_) | |___ / ___ \| _ < / __/ _| |_| | | | /_/ \_\___/\__|_| \___/|____/|_____/_/ \_\_| \_\ |_____(_)\___/ | | | ============================================================================== Running on 128 cores
Next you might see any problem specific IO you have put in your ProblemModuleInit
Background Density = 1.00000000000000 particles/cc Equilibrium temp = 4931.25172997924 K sound speed = 90.6573377245259 7.30876669252163 km/s Background Jeans Length = 1115.07377816552 pc Background free fall time = 45.6773981883929 myr flow velocity = 1.359860065867888E-003 km/s mass flux = 566.140137477777 m_sun/Myr ram pressure = 18492.1939874222 particles K/cc ram density = 2029.78712049456 particles per cc ram temp = 9.11041054537610 K ram density jeans length = 1.06382145423590 pc Ambient%B= 0.993101377501737E+02 0.000000000000000E+00 0.000000000000000E+00
Next you'll see whether or not an existing profile.data file was found - or the code will start generating one.
either
Found existing profile.data file.or
profiling... 70%||||||||||||||||||||||||||||| |
Unfortunately the progress bar doesn't look as good when you view the standard out in a file editor… This is the price you pay for seeing it fill up in the terminal…
Next you should see either grid initialization on every level followed by the first frame dump
Initializing Grids on level 0 Initializing Grids on level 1 Initializing Grids on level 2 Initializing Grids on level 3 Initializing Grids on level 4 Level -2 data written. Level -1 data written. Level 0 data written. Level 1 data written. Level 2 data written. Level 3 data written. Level 4 data written. Time to make output file 0: 15.0011 seconds.
Or if you are restarting you might see
Reloading Grids on level 0 Reloading Grids on level 1 Reloading Grids on level 2 Reloading Grids on level 3 Reloading Grids on level 4 Time to restart from output file 1: 0.0320 seconds.
At this point the simulation will begin and you will see a bunch of Advance lines for each level's step… For every level 0 step there will be 2 level 1 advances, 4 level 2 advances, 8 level 3 and so on… Each line tells you what time the level has been advanced to - the time step taken, the cfl, various max speeds that go into calculating the cfl. These quantities are not globally reduced - so you will only see their values for grids on the Master processor. If the master processor doesn't have any grids on a level - these will appear as 0.000.
Advanced level 0 to tnext= 0.2930E-03 with dt= 0.2930E-03 CFL= 0.4164E-01 max speed= 0.1110E+03 elliptic_speed= 0.1703E+01 particle_speed= 0.0000E+00 Advanced level 1 to tnext= 0.1465E-03 with dt= 0.1465E-03 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.9975E+00 particle_speed= 0.0000E+00 Advanced level 2 to tnext= 0.7324E-04 with dt= 0.7324E-04 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.7123E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.3662E-04 with dt= 0.3662E-04 CFL= 0.9251E-01 max speed= 0.2467E+03 elliptic_speed= 0.4912E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1831E-04 with dt= 0.1831E-04 CFL= 0.9196E-01 max speed= 0.2452E+03 elliptic_speed= 0.3465E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.3662E-04 with dt= 0.1831E-04 CFL= 0.9196E-01 max speed= 0.2452E+03 elliptic_speed= 0.3465E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.7324E-04 with dt= 0.3662E-04 CFL= 0.9251E-01 max speed= 0.2467E+03 elliptic_speed= 0.4912E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.5493E-04 with dt= 0.1831E-04 CFL= 0.9257E-01 max speed= 0.2469E+03 elliptic_speed= 0.3482E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.7324E-04 with dt= 0.1831E-04 CFL= 0.9257E-01 max speed= 0.2469E+03 elliptic_speed= 0.3482E+00 particle_speed= 0.0000E+00 Advanced level 2 to tnext= 0.1465E-03 with dt= 0.7324E-04 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.7123E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.1099E-03 with dt= 0.3662E-04 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.4962E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.9155E-04 with dt= 0.1831E-04 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.3518E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1099E-03 with dt= 0.1831E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.3518E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.1465E-03 with dt= 0.3662E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.4962E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1282E-03 with dt= 0.1831E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.3518E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1465E-03 with dt= 0.1831E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.3518E+00 particle_speed= 0.0000E+00 Advanced level 1 to tnext= 0.2930E-03 with dt= 0.1465E-03 CFL= 0.9263E-01 max speed= 0.2470E+03 elliptic_speed= 0.9975E+00 particle_speed= 0.0000E+00 Advanced level 2 to tnext= 0.2197E-03 with dt= 0.7324E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.7123E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.1831E-03 with dt= 0.3662E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.5012E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1648E-03 with dt= 0.1831E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.1831E-03 with dt= 0.1831E-04 CFL= 0.9264E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.2197E-03 with dt= 0.3662E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.5012E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2014E-03 with dt= 0.1831E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2197E-03 with dt= 0.1831E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 2 to tnext= 0.2930E-03 with dt= 0.7324E-04 CFL= 0.9264E-01 max speed= 0.2470E+03 elliptic_speed= 0.7124E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.2563E-03 with dt= 0.3662E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.5012E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2380E-03 with dt= 0.1831E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2563E-03 with dt= 0.1831E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 3 to tnext= 0.2930E-03 with dt= 0.3662E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.5012E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2747E-03 with dt= 0.1831E-04 CFL= 0.9265E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00 Advanced level 4 to tnext= 0.2930E-03 with dt= 0.1831E-04 CFL= 0.9266E-01 max speed= 0.2471E+03 elliptic_speed= 0.3535E+00 particle_speed= 0.0000E+00
After each step, however, (or really during the next distribution of grids), a logical flag is reduced that checks if any processor needs to restart the entire root step with a smaller dt. If this happens you will see the code print "restart requested" and it will gracefully exit whatever amr step it is in…
In between root steps you will also see some very useful information
Info allocations = 45.6 gb 557.0 mb message allocations = ------ 10.0 mb sweep allocations = ------ 84.9 mb filling fractions = 0.033 0.741 0.744 0.748 Current efficiency = 74% Cell updates/second = 8687 14362 60% Wall Time Remaining = 1.1 mo AMR Speed-Up Factor = 0.2366E+02
- The first 3 lines give you information on memory usage. The first entry is the current global usage - and the second entry is the peak usage on any single cpu.
- Info allocations - memory for all grid data (including ghost zones - fluxes, … everything in info)
- message allocations - Amount of memory allocated for communication
- In between root steps there should be no open messages - so the global message allocation should be 0 or '———'
- sweep allocations - Amount of memory allocated for advancing any single grid
- Likewise the global sweep allocations should be 0 or '———'
- The next line gives you the filling fractions of every level. This is the ratio of the number of cells refined to the total number of cells on that level. So in this case, the entries go like, f(0) = 0.033, f(1)=0.741, f(2)=0.744, f(3)=0.748. This means that ~3% of level 0 's grid is refined to level 1, ~74% of level 1's grid is refined to level 2, and so on. To get the percentage of the grid's volume that is refined to level n, you then would multiply the filling fractions of all the levels less than n together (note this is strictly for n>1). This gives you the total fraction of the grid refined to that level. For example, the volume filling fraction of level 2 is vf(2) = f(0)*f(1). Similarly, vf(4)=f(0)*f(1)*f(2)*f(3). Note that the volume filling fraction doesn't make sense for level 0, and that vf(1)=f(0).
- The next line tells you the current efficiency of advances - that is what fraction of the walltime cpu's are spending in the hyperbolic advance stage. This doesn't include elliptic updates - or any other amr related maintenance (restriction/prolongation/flux-synchronization/regridding etc…)
- The next line tells you how many cell updates/second you are currently achieving. The first number is the number of internal zones - so not double counting redundant computations of extended ghost zones - and the second number is the total number of cell updates including extended ghost zones. And the third number is the ratio of the two. If your grids are really small - this ratio could get small… Although the new sub-grids generating routine works to keep this ratio up.
- The next line tells you how long you will have to wait for the job to finish (assuming that the filling fractions don't change substantially)
- And the last line shows the ratio of the predicted time to take a root step if you were running fixed grid - compared to the current time per root step using AMR. If this drops below 1 you should consider running fixed grid instead. But if it is > 1 - then AMR is saving you time - even with it's overhead.