New Work

1. Notes about extracting diagnostics on energy from the simulation.
2. Analysis of runs:
   • 133 (as low res fiducial run 116 and more aggressive refinement run 125 but with even more aggressive refinement algorithm that de-resolves more of the `ambient' envelope).
   • 135 (as 133 but with accretion turned off).
   • 136 (as 133 but without any relaxation run, so no initial damping of velocities).
   • 132 (similar to 125/133 but with maxlevel increased by 1 so twice higher resolution).
   • 120 (similar to 116 but ambient density reduced by a factor of about 67).

Summary of New Results

1. Procedure to extract energy diagnositics seems quite clear.
2. Key results from runs are as follows:
   • 133: Results are generally consistent with run 116 but the radius of the maxlevel refinement region around the primary point particle should be kept at a minimum of 2 times the particle separation.
   • 135: Results are very similar to 133 with `accretion torus' morphology preserved in spite of accretion being turned off.
   • 136: Results are very similar to 133. This means damping is probably unnecessary and can be omitted in future runs!
   • 132: Results of this first higher resolution run are closer to the result of O+16a. Qualitatively similar to run 133.
   • 120: Results are similar to those of run 116, but expansion and outer shock morphology of expanding envelope are slightly affected.

Notes on energy

I compiled the following notes. Thanks to Bo for a discussion on this:
Please see ​en.pdf

Notes on accretion and drag force (from last blog post)

Please see ​df.pdf

Analysis of runs 133, 135, 136, 132, 120

Old run 116 (for comparison)

Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 168
Total simulation time: 3.36e6 s or ~4.2 RG sound-crossing times or 21.5 days (93 frames)
Machine and partition: Stampede 1 normal
Number of cores: 1024
Total wall time: 96 hours
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set automatically by AstroBear
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
DefaultAccretionRoutine=2 (Krumholz)

Old run 125 (for comparison)

Binary run 125 with longer simulation time and lower resolution in ambient medium
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 335
Total simulation time: 5.2e6 s or 60 days ~6.5 RG sound-crossing times or 60.2 days (260 frames)
Machine and partition: Bluehive standard
Number of cores: 120
Total wall time: about 8.5 days
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm (575 Rsun)
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 50 Rsun of primary and within a cylinder of radius 50 Rsun and height 50 Rsun around secondary
Buffer zones: 0 cells (no buffer zones)
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
DefaultAccretionRoutine=2 (Krumholz)

Run 133 to test making maxlevel refinement zone decrease with time:

Binary run 133 similar to run 125 but now the refinement zone changes with time
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time: 84 days (up to frame 439)
Machine and partition: Bluestreak standard
Number of cores: 8192 (2 cores per task to increase memory)
Total wall time: 6 days
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm (575 Rsun)
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 1.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
DefaultAccretionRoutine=2 (Krumholz)

Comparison between runs 125(top left), 133(top right), O+16a(bottom) (FROM LAST BLOG POST)

Comparison between run 125(left) and 133(right)
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Comments:
FROM LAST BLOG POST (BASED ON ORBITS ONLY):

• The runs are very similar but there is a slightly smaller final separation for run 125(less aggressive refinement)
• This suggests that our refinement criteria for run 133 was probably slightly too aggressive.
• However, the otherwise close agreement tells us that decreasing the refinement zone with time to be a sphere centred on p1 with radius ~2 times particle separation is reasonable.

NEW COMMENTS (BASED ON MOVIES):

• The runs are very similar but having max refinement in a sphere centered around the primary with radius 1.5 times the separation is a bit too aggressive. Could probably get away with about 2 times the separation.

Run 135 to test case where no accretion onto secondary is permitted:

Binary run 135 similar to run 133 but now DefaultAccretionRoutine=0 instead of 2. Also suppress generation of new sink particles.
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 900
Total simulation time: 191 days
Machine and partition: bluehive2.5 standard
Number of cores: 120
Total wall time: 2.8 days
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm (575 Rsun)
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 1.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
Default Accretion Routine=0 (No accretion)

Comparison between runs 133(top left), 135(top right), O+16a(bottom)

Comparison between run 133(left) and 135(right)
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Comments:
FROM LAST BLOG POST (BASED ON ORBITS ONLY):

• The results of 135 and 133 are similar, except:
  • Run 135 (no accretion) has slightly smaller separation at a given time.
  • Run 135 (no accretion) has slightly larger frequency of oscillations.
• Thus accretion cannot fully explain the discrepancy with the O+16a results, but removing accretion does make results slighly closer to those of O+16a, who did not have accretion.
• We must think more about accretion, but it is not making a huge difference at present.

NEW COMMENTS:

• Results are very similar. Most noticeable difference is that density is higher around the point particles for run 135, which does not have accretion. This is most clearly seen in the first, most zoomed out movie. This would be expected since gas is not removed by the particle.
• Particle creation was turned off for run 135 and this may explain some differences.
• Bottom line is that the gas flow around the secondary is very similar regardless of whether accretion is turned on. Even without accretion turned on, there is a torus structure with bipolar outflows.
• Note that for both runs the flow around the secondary undergoes a transition after about 75 days (frame 375) and the torus morphology is no longer present at this time. This corresponds to a particle separation of about 9 Rsun.

Run 136 to test how much relaxation makes a difference:

Binary run 136 similar to run 135 but now start from frame 0 of relaxation run 096 instead of frame 75 (so no damping).
Relaxation run: 096 (but no relaxation! Just start with initial profile
First frame: 0
Last frame: 18, running
Total simulation time: 4 days
Machine and partition: bluehive2.5 standard
Number of cores: 120
Total wall time: 21 hours, running
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm (575 Rsun)
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 1.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
Default Accretion Routine=0 (No accretion)

Comparison between runs 133(left) and 136(right)

Comparison between run 133(left) and 136(right)
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Comments:

• The results are very similar. This suggests that relaxation (velocity damping before the run) is NOT necessary. (It also confirms that most of the differences between runs 133 and 135 or 125 and 133 above are not due to the particle creation on run 133.)

Run 132 that uses twice as high resolution and 10x lower ambient pressure as previous runs:

Binary run 132 with double max resolution, lower resolution in ambient medium, 10x smaller ambient pressure than run 116
Relaxation run: 129
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: Stampde 1 normal/Bluehive 2.5 standard/Bluestreak standard (completed up to frame 375, or 70 sim-days)
Number of cores: 1024
Total wall time: around 14 days (starting from frame 75 of relaxation run)
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm (575 Rsun)
Base resolution: 2.25 Rsun (256³ cells)
Highest resolution: 0.14 Rsun (4096³ cells, 4 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 5e12 cm (71.87 Rsun) of primary center and within a cylinder of radius 20 Rsun and height 20 Rsun around secondary center. After t~11d (~frame 123) refinement radius around primary was halved to 2.5e12cm (36Rsun). After t~31d (frame 210) halved again to 1.25e12cm (18Rsun).
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁵ dyne/cm²
DefaultAccretionRoutine=2 (Krumholz)

Movies of run 132
​face-on full box
​face-on zoom-in
​edge-on full box
​edge-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Comparison of run 132 with Ohlmann+16a run:
​Table

Comparison between run 133(left) and 132(right)
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Comparison between runs 133(top left), 132(top right), O+16a(bottom)

Comments:
FROM LAST BLOG POST (BASED ON ORBITS ONLY):

• The 1st minimum, 2nd maximum, and 2nd minimum are located roughly at separations, with softening length denoted as h:
  • Run 088 (res=0.29Rsun, h=4.8Rsun, cells/h=17): 14 , 21 , 12 Rsun
  • Run 125 (res=0.29Rsun, h=2.4Rsun, cells/h= 8): 13.5, 18 , 14.5 Rsun
  • Run 132 (res=0.14Rsun, h=2.4Rsun, cells/h=17): 14 , 18.5, 11 Rsun
  • O+16a (see above table) : 10 , 23.5, 11 Rsun
• These are found roughly at times:
  • Run 088 (res=0.29Rsun, h=4.8Rsun, cells/h=17): 12.5, 15.5, 18 days
  • Run 125 (res=0.29Rsun, h=2.4Rsun, cells/h= 8): 12.5, 14.5, 17 days
  • Run 132 (res=0.14Rsun, h=2.4Rsun, cells/h=17): 12.5, 14.5, 17 days
  • O+16a (see above table) : 13 , 16 , 19 days
• From the difference between 125 and 132, we conclude that h is not adequately resolved in run 125 (8.5 cells/h vs 17 cells/h)
• O+16a advocates >10 cells/h, so this is consistent with their result.
• In our case, we still cannot tell if 17 cells/h is sufficient.
• We could run 088 with double the resolution to see if it converges.
• The softening length of the point particles decreases with time in the O+16a simulation.
  • It is not clear whether this procedure is justified.
  • The main reason to improve the resolution with time may be to resolve the decreasing softening length.
  • Therefore, if we keep the softening length constant, improving the resolution with time may not be necessary or productive.
• O+16a initially resolves the softening length by 20 cells, whereas we resolve it by 17 cells. They advocate >10 cells.
• At 120 days, O+16a resolves the softening radius by 35 cells.

NEW COMMENTS:

• Runs 133 and 132 are very similar for the first 15 days, or when the separation does not dip below 14 Rsun. This suggests that run 133 has inadequate resolution (or resolution per softening length) when the separation is <14 Rsun. Recall that both runs have softening length 2.4 Rsun but that run 133 has best resolution of 0.29 Rsun while run 132 has best resolution of 0.14 Rsun.
• The amplitude of the oscillations in the separation is about the same for run 132 and 133, both lower than that of O+16a.
• The final asymptotic separation is about 6 Rsun for run 132, compared to about 9 Rsun for run 133 and 4 Rsun for O+16a.
• The transition of the flow around the secondary happens earlier in run 132, at about 55 days (frame 320) compared to about 75 days (frame 375) in run 133. For run 132, this corresponds to a mean particle separation of about 7 Rsun, and for run 133, to a mean particle separation of about 9 Rsun. This supports the idea that the transition to a flow without the torus morphology is probably related to the separation. Further, it also supports the idea that such a transition is an artifact of having a too-low resolution per softening length, because the separation at onset is lower when the resolution is higher.
• There is a tendency for the particles to migrate in the direction of larger x and y. This is visible to a lesser extent in O+16a. This should be explained.

Run 120 to test importance of density of ambient medium

Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 119
Total simulation time: 3.36e6 s or ~4.2 RG sound-crossing times or 10.2 days (48 frames)
Machine and partition: Stampede 1 normal
Number of cores: 1024
Total wall time: 6 days
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set automatically by AstroBear
Softening length: 2.4 Rsun
Ambient density: 1e-10 g/cc
Ambient pressure: 10⁶ dyne/cm²
DefaultAccretionRoutine=2 (Krumholz)

Comparison between runs 116(left) and 120(right)

Comparison between run 116(left) and 120(right)
​face-on
​face-on different color bar limits
​face-on slice through secondary, extra zoom-in

Comments:

• Results of the two runs are very similar.
• Differences are apparent at the interface of the ambient medium and envelope. Specifically, whisps of low density (Kelvin-Helmholtz unstable?) is visible outside the shock in low ambient density run 120. Also, by the end of the run, the envelope has expanded to a slightly larger size in the low ambient density case (the larger upper lobe is about 10% wider). The outer shock structure also shows minor differences.
• In the region around the point particles, differences are very minor. If our focus is on this region, we should not have a problem with the higher ambient density.

Discussion & conclusions

• The energy contributions can be readily analyzed from the chombo files, now that we have written down the important equations.
• Damping is not necessary (and introduces artifacts through grid effects). Can avoid it in future.
• Turning accretion off does not have a big effect on the flow. Since we do not understand how to model the accretion very well, perhaps we should keep it turned off for now. This would avoid the unphysical removal of pressure during the accretion process, as pointed out by Eric. In addition, the analysis of the drag force and energy budget would be simplified, and we could use the chombo files without having to output additional information. But we need to get an accretion rate to use for the jets, so ultimately we need to decide on an accretion model.
• We are able to be reasonably aggressive with refinement, but max level refinement in a sphere around the RG core with radius of 2 times the particle separation seems to be the limit, and trying to make this region even smaller introduces too much error.
• Ambient density does not seem to matter very much, but not sure yet with regard to binding and ultimate escape of envelope.
• High resolution run is consistent, qualitatively, with lower resolution run, but shows quantitative results closer to those of O+16a. Namely smaller final separation and orbital period than our lower resolution runs, but still not as small as O+16a, and amplitude of oscillations still lower than O+16a.
• There is a transition from a torus/thick disk structure to flow which does not show this structure. This transition seems to happen when the inter-particle separation reduces past a certain threshold. Futhermore, the threshold seems to be smaller for a better resolution, suggesting that this transition may be a numerical artifact due to a lack of resolution (or resolution per softening length).
• We should keep track of the center of mass of the system (gas+particles) which should not change. (Some gas leaves the box during the simulation and it might be worth increasing the box size or at least testing the effect of increasing it.)

Next steps

• We should now start to perform the analysis of the drag force and energy budget using results of run 132. We don't have accreted momentum outputted from that run, but we can still make do with the chombo files for most of the analysis.
• Simultaneously, we should be doing runs on stampede 2 and comet with our remaining resources, as well as on bluestreak and bluehive:
  • Rerun from middle of 132 but with softening length now able to decrease with time. Need to increase AMR max level with time accordingly.
  • New run like 132 but with initial RG spin equal to 95% of corotation to see what effect this has.
  • New run like run 088 (low res, twice the softening length of more recent runs) but with twice as high resolution (i.e. same resolution as run 132) to test the convergence of results as the number of resolution cells per softening length increases (currently we know that 8 is insufficient and 17 may or may not be sufficient…here we would bump it up to 34).

Appendix

Movies of run 133
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Movies of run 135
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Movies of run 136
​face-on zoom-in
​face-on slice through secondary, extra zoom-in
​viewed from P1 with P2 at center
​slice through P1 (left side) and P2 (center)

Movies of run 120
​face-on
​face-on different color bar limits