Changes between Version 52 and Version 53 of u/erica/CFRunStatsHighRes


Ignore:
Timestamp:
08/04/14 15:01:51 (10 years ago)
Author:
madams
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • u/erica/CFRunStatsHighRes

    v52 v53  
    1515'''Frames complete:'''
    1616
    17 Beta10, Shear15 (139/200)
    18 Shear30 (128/200)
    19 Shear60 (151/200)
     17Beta10, Shear15 (144/200)
     18Shear30 (153/200)
     19Shear60 (153/200)
    2020
    21 Beta1, Shear60 (122/200)
     21Beta1, Shear60 (130/200)
    2222
    2323|| '''Beta''' || '''Shear'''  || '''Machine, Directory''' || '''Frame''' || '''Filling Fracs''' || '''Walltime left (predicted and queued remaining)''' || '''Info allocs, Message allocs''' || '''No. cells finest''' || '''Framerate (mins/frame), no. of cores''' || '''Notes''' ||
    2424||  '''1'''   ||   '''15'''    || BS, /scratch/ekamins2/MHD_Shear/ Beta1/xfield/Shear15/Restart_ResJune05_2014/ Less_Levels/Trying_New_Frame_Copy/Test_Same_Levels_As_Orig || restart from frame = 56   || || || || || || code prints hypre errors after restarting files from bamboo using a different number of levels. didn't get these errors when I restarted using same number of levels. ||
    25 ||  '''1'''   ||   '''60'''    || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60   || 67/200 || 0.087  0.541  0.615  0.243  0.286 || 9days/0   ||  265.2 gb  104.5 mb, 128.0 mb  || 19,440,450 || 1.2hr/frame, 4096 cores || Hit time limit  ||
    26 ||     ||      || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart   || 86/200 || 0.096  0.537  0.628  0.295  0.402 || 13days/0   ||  240 gb  167 mb, 64 mb  || 37,101,854 || 3hr/frame,  2048 cores || Hit time limit  ||
    27 ||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart   || 89/200 || 0.097  0.538  0.628  0.298  0.411 || 12days/0   ||  243 gb  168 mb, 64 mb  || ~37,101,854 || 3hr/frame,  2048 cores || Hit time limit  ||
    28 ||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart/Restart || 93/200 || 0.098  0.539  0.625  0.304  0.435 || 12.5days/0 || 250.8 gb,  178.9 mb, 64.0 mb ||  ||  || Files taken off of BS (07/28) ||
    29 ||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart/Restart/Restart || 102/200 || 0.101  0.543  0.631  0.314  0.459 || 17.5days/0 || 267.4 gb  184.0 mb, 64.0 mb ||  || 3hr/frame, 2048 cores ||     ||
     25||  '''1'''   ||   '''60'''    || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60   || 67/200 || 0.087  0.541  0.615  0.243  0.286 || 9days/0days   ||  265.2 gb  104.5 mb, 128.0 mb  || 19,440,450 || 1.2hr/frame, 4096 cores || Hit time limit  ||
     26||     ||      || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart   || 86/200 || 0.096  0.537  0.628  0.295  0.402 || 13days/0days   ||  240 gb  167 mb, 64 mb  || 37,101,854 || 3hr/frame,  2048 cores || Hit time limit  ||
     27||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart   || 89/200 || 0.097  0.538  0.628  0.298  0.411 || 12days/0days   ||  243 gb  168 mb, 64 mb  || ~37,101,854 || 3hr/frame,  2048 cores || Hit time limit  ||
     28||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart/Restart || 93/200 || 0.098  0.539  0.625  0.304  0.435 || 12.5days/0days || 250.8 gb,  178.9 mb, 64.0 mb ||  ||  || Files taken off of BS (07/28) ||
     29||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart/Restart/Restart || 102/200 || 0.101  0.543  0.631  0.314  0.459 || 17.5days/0days || 267.4 gb  184.0 mb, 64.0 mb ||  || 3hr/frame, 2048 cores ||     ||
    3030||    ||     || BS, /scratch/ekamins2/MHD_Shear/Beta1/xfield/Shear60/Restart/Restart/Restart/Restart/Restart || 106/200 || 0.102  0.549  0.627  0.313  0.458 || 1.3mo/0 || 263.9 gb  328.8 mb, 64.0 mb ||  || 4hr/frame, 2048 cores ||  ||
    31 ||    ||     || BS, /scratch/madams15/CollidingFlows/Beta1/Shear60/ || 112/200 || 0.105  0.553  0.621  0.312  0.469 || 13.5days/0 || 271.6 gb, 193.1 mb, 64.0 mb ||  || ~4.5hr/frame, 2048 cores || Reservation. Testing what node fraction would be most efficient. Stayed with 128. ||
     31||    ||     || BS, /scratch/madams15/CollidingFlows/Beta1/Shear60/ || 112/200 || 0.105  0.553  0.621  0.312  0.469 || 13.5days/0days || 271.6 gb, 193.1 mb, 64.0 mb ||  || ~4.5hr/frame, 2048 cores || Reservation. Testing what node fraction would be most efficient. Stayed with 128. ||
    3232||    ||     || BS, /scratch/madams15/CollidingFlows/Beta1/Shear60/Restart/ || 122/200 || 0.106  0.559  0.631  0.321  0.467 || 16.8days/0days || 282.5 gb,  197.6 mb, 64.0 mb || || ~5hr/frame, 2048 cores || Reservation. Stayed with 128 nodes. Started with 48 hours left on reservation. ||
    33 || '''10''' || '''15''' || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15 || 36/200  || 0.142  0.547  0.575  0.183  0.367 || 8days/0 || 233.6 gb   58.7 mb, 192.0 mb || 9,058,432 || 50 mins/frame on 8192 cores  || Reservation ||
    34 ||    ||      || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart || 43/200 || 0.155  0.546  0.565  0.214  0.375 || 7 days/0 || 259.3 gb   62.6 mb, 192.0 mb || 11,588,046 || 1hr/frame, 8192 cores || reservation || 
    35 ||    ||      || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart || 57/200  || 0.186  0.573  0.533  0.266  0.388 || 7 days/0 || 315.9 gb   68.6 mb, 192.0 mb  || 17,705,187 || 1hr/frame, 8192 cores || Non-threaded, non-global partition Hypre, reservation || 
    36 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading    ||  69/200  || 0.228  0.582  0.548  0.356  0.450 || 11 days/0 || 221.7 gb  155.2 mb, 64.0 mb || 35,179,794 || 2hr/frame,2048 cores || Standard queue (2048 cores). Eta. was originally 6/18/2014, but actually started on 6/23, 3 AM. Copied non-threaded version of astrobear into the run directory and named it threaded to match the submitted batch script, because found the threaded version of the code threw errors ||
    37 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart    ||  83/200  || 0.255  0.612  0.562  0.36  0.418 || 14 days/0 days || 326 gb  122 mb, 128.0 mb || 39,856,494 || 2.5hr/frame,4096 cores || reservation ||
    38 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart/Restart    ||  94/200  || 0.277  0.635  0.571  0.391  0.407 || 10 days/0 days || 362 gb  131 mb, 128.0 mb || 48,267,318 || 2.5hr/frame,4096 cores || reservation, disk quota exceeded ||
    39 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart/Restart/Restart ||  95/200  || .279 .638  .57 .395 .409 || 13 days/0 days || 365 gb  143 mb, 128.0 mb || 49,500,836 || NA || NA ||
    40 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart ||  105/200  || .303 .647  .581 .414 .403 || 10 days/0 days || 400 gb  148 mb, 128.0 mb || 57,387,965 || 3hr/frame, 4096 cores || Disk quota exceeded, files are ~9 GB each at this point ||
    41 ||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart ||  109/200  || .325 .663  .596 .465 .436 || 17.5 days/0 || 344 gb  236 mb, 64 mb || 78,627,595 || 5hr/frame, 2048 cores || Still running, size of files = 13 GB ||
    42 ||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart/Restart/ || 119/200 || 0.344  0.675  0.610  0.483  0.438 || 18.3days/0 || 372.7 gb, 250.9 mb, 64.0 mb ||  || 5hr/frame, 2048 cores || Files taken off of BS (07/28) ||
    43 ||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart/Restart/Restart || 120/200 || 0.344  0.680  0.614  0.485  0.436 || 24.4days/0 || 374.3 gb, 259.0 mb, 64.0 mb ||  || 10hr/frame, 2048 cores || Catching chombos for Marissa's runs. ||
    44 ||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear15 || 133/200 || 0.367  0.670  0.565  0.406  0.388 || 6days/0 || 592.6 gb, 116.2 mb, 192.0 mb ||  || ~3-4hr/frame, 2048 cores, ~2hr/frame, 8192 cores || Reservation. Testing what node fraction would be most efficient. Stayed with 128 for 07/24, then bumped up to 256 07/25, then finally 512 for another 24 hours. ||
    45 ||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear15/Restart || 139/200 || 0.386  0.698  0.616  0.496  0.468 || 22.1days/0 || 422.4 gb, 296.5 mb, 64.0 mb ||  || 7hr/frame, 2048 cores || Restarted with 128 nodes on 07/26. ||
     33||    ||     || BS, /scratch/madams15/CollidingFlows/Beta1/Shear60/Restart/Restart || 130/200 || 0.109  0.558  0.640  0.328  0.455 || 16.1days/0days || 288.3 gb, 199.5 mb, 64.0 mb || || ~5hours/frame, 2048 cores || Previous reservation got extended. Stayed with 128 nodes. Ran for 48 hours until it went over the soft limit and BS killed the job. ||
     34|| '''10''' || '''15''' || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15 || 36/200  || 0.142  0.547  0.575  0.183  0.367 || 8days/0days || 233.6 gb   58.7 mb, 192.0 mb || 9,058,432 || 50 mins/frame on 8192 cores  || Reservation ||
     35||    ||      || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart || 43/200 || 0.155  0.546  0.565  0.214  0.375 || 7days/0days || 259.3 gb   62.6 mb, 192.0 mb || 11,588,046 || 1hr/frame, 8192 cores || reservation || 
     36||    ||      || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart || 57/200  || 0.186  0.573  0.533  0.266  0.388 || 7days/0days || 315.9 gb   68.6 mb, 192.0 mb  || 17,705,187 || 1hr/frame, 8192 cores || Non-threaded, non-global partition Hypre, reservation || 
     37||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading    ||  69/200  || 0.228  0.582  0.548  0.356  0.450 || 11days/0days || 221.7 gb  155.2 mb, 64.0 mb || 35,179,794 || 2hr/frame,2048 cores || Standard queue (2048 cores). Eta. was originally 6/18/2014, but actually started on 6/23, 3 AM. Copied non-threaded version of astrobear into the run directory and named it threaded to match the submitted batch script, because found the threaded version of the code threw errors ||
     38||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart    ||  83/200  || 0.255  0.612  0.562  0.36  0.418 || 14 days/0days || 326 gb  122 mb, 128.0 mb || 39,856,494 || 2.5hr/frame,4096 cores || reservation ||
     39||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart/Restart    ||  94/200  || 0.277  0.635  0.571  0.391  0.407 || 10 days/0days || 362 gb  131 mb, 128.0 mb || 48,267,318 || 2.5hr/frame,4096 cores || reservation, disk quota exceeded ||
     40||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/ !Restart/Restart/Restart_With_Threading/Restart/Restart/Restart ||  95/200  || .279 .638  .57 .395 .409 || 13 days/0days || 365 gb  143 mb, 128.0 mb || 49,500,836 || NA || NA ||
     41||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart ||  105/200  || .303 .647  .581 .414 .403 || 10 days/0days || 400 gb  148 mb, 128.0 mb || 57,387,965 || 3hr/frame, 4096 cores || Disk quota exceeded, files are ~9 GB each at this point ||
     42||   ||  || BS,  /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart ||  109/200  || .325 .663  .596 .465 .436 || 17.5 days/0days || 344 gb  236 mb, 64 mb || 78,627,595 || 5hr/frame, 2048 cores || Still running, size of files = 13 GB ||
     43||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart/Restart/ || 119/200 || 0.344  0.675  0.610  0.483  0.438 || 18.3days/0days || 372.7 gb, 250.9 mb, 64.0 mb ||  || 5hr/frame, 2048 cores || Files taken off of BS (07/28) ||
     44||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear15/Restart/Restart/Restart_With_Threading/Restart/Restart/Restart/Restart/Restart/Restart/Restart/Restart || 120/200 || 0.344  0.680  0.614  0.485  0.436 || 24.4days/0days || 374.3 gb, 259.0 mb, 64.0 mb ||  || 10hr/frame, 2048 cores || Catching chombos for Marissa's runs. ||
     45||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear15 || 133/200 || 0.367  0.670  0.565  0.406  0.388 || 6days/0days || 592.6 gb, 116.2 mb, 192.0 mb ||  || ~3-4hr/frame, 2048 cores, ~2hr/frame, 8192 cores || Reservation. Testing what node fraction would be most efficient. Stayed with 128 for 07/24, then bumped up to 256 07/25, then finally 512 for another 24 hours. ||
     46||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear15/Restart || 139/200 || 0.386  0.698  0.616  0.496  0.468 || 22.1days/0days || 422.4 gb, 296.5 mb, 64.0 mb ||  || 7hr/frame, 2048 cores || Restarted with 128 nodes on 07/26. ||
     47||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear15/Restart/Restart || 144/200 || 0.398  0.695  0.617  0.499  0.472 || 20.4days/0days || 432.4 gb, 290.9 mb, 66.1 mb ||  || ~9hr/frame, 2048 cores || Previous reservation got extended until 08/04 at 08:00:00. Stayed with 128 nodes. Job ran for 48 hours until memory went over soft limit. ||
    4648|| '''10'''  || '''30''' || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear30   || 7/200 || 0.058  0.467  0.607  0.000  0.000 || 1 day/0 || 29.1 gb   81.7 mb, 17.3 mb || 2,482,538 || 7mins/frame, 512 cores || Running a bigger box (200,75,75 pc) with same eff. resolution to prevent backflow into colliding flow object. Debug queue, 1 hour, 512 cores ||
    4749||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear30/ Testing_Smaller_Box || 7/200  || 0.163  0.536  0.607  0.000  0.000 || 1 day/0 || 28.5 gb   83.0 mb, 17.5 mb  || || || Running original box size (62.5,75,75 pc) with same eff. resolution to see how framerate changes. Debug queue, 1 hour, 512 cores. Same framerate, so making box longer in x seems to not cost much ||
    … …  
    5658||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear30/Restart/Restart/Restart/Restart/Restart/Restart/Restart/Restart/ || 123/200 || 0.132  0.583  0.595  0.461  0.424 || 21.1days/0 || 367.8 gb, 245.9 mb, 64.0 mb ||  || ~8hr/frame, 2048 cores || Catching chombos for Marissa's runs. ||
    5759||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear30 || 128/200 || 0.136  0.587  0.601  0.465  0.418 || 1.2mo/0 || 377.0 gb, 252.4 mb, 64.0 mb ||  || 9hr/frame, 2048 cores || Started Jul 24 08:46, cancelled to test other runs, started again at Jul 26 16:41. ||
     60||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear30/Restart/ || 143/200 || 0.131  0.620  0.570  0.392  0.367 || 7.8days/0days || 599.5 gb, 122.6 mb, 256.0 mb ||  || ~2-3hours/frame, 8192 cores || Started immediately when received word about reservation extension until 08/04 at 08:00:00. Ran on 512 cores, half of BS, for 48 hours. ||
     61||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear30/Restart/Restart/ || 153/200 || 0.141  0.622  0.583  0.442  0.400 || 9days/0days || 504.3 gb, 175.0 mb, 128.mb ||  || ~4-5hours/frame, 4069 cores || Still on the same reservation. Ran with Beta1Shear60 and Beta10Shear15 (both at 128 cores), while this ran at 256 cores. Stopped eventually when we could no longer write files due to soft limit. ||
    5862|| '''10''' || '''60''' || BS,   /scratch/ekamins2/MHD_Shear/Beta10/Shear60    ||  5/200  || 0.078  0.522  0.680  0.000  0.000 || 1.6days/0 || 37.6 gb  104.3 mb, 22.9 mb || 4,180,579 || 10mins/frame, 512 cores || Debug queue, 512 cores, 1 hour, 200x75^2^ pc ||
    5963||  ||  || BS, /scratch/ekamins2/MHD_Shear/Beta10/Shear60/Restart   ||  91/200 || 0.124  0.484  0.586  0.158  0.205  || 7.2 days/0 day || 254.7 gb   99.0 mb || 11,008,243 || 1hr/frame, 4096 cores || Died with memory error. Reservation ekamins2-20140623 ||
    … …  
    6468||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear60 || 140/200 || 0.174  0.547  0.550  0.303  0.251 || 9.7days/0 || 288.4 gb, 196.4 mb, 64.0 mb ||  || 4hr/frame, 2048 cores || 07/24-07/25 Reservation at 128 cores. ||
    6569||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear60/Restart/ || 151/200 || 0.180  0.547  0.544  0.318  0.269 || 6.7days/0 || 305.3 gb, 211.8 mb, 64.0 mb ||  || ~4hr/frame, 2048 cores  ||  ||
     70||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear60/Restart/Restart/ || 153/200 || 0.184  0.543  0.545  0.318  0.273 || 8.6days/0days || 309.2 gb, 209.7 mb, 64.0 mb ||  || ~4hours/frame, 2048 cores || Ran on 128 nodes in the Standard queue. As our reservation took up half of the machine, this run was pending for a while. Error when soft limit hit: {{{2014-08-02 03:42:44.215 (ERROR) [0x40001069280] ibm.runjob.client.Output: could not write: Disk quota exceeded }}} ||
     71||  ||  || BS, /scratch/madams15/CollidingFlows/Beta10/Shear60/Restart/Restart/Restart || TBA/200 || TBA || TBA || TBA ||  || TBA || Running on 128 nodes in the Standard queue. Currently pending. ||