Visualizing External Data in Shape

As we know very well, AstroBEAR generates simulations in HDF5 (Hierarchical Data Format), i.e. Chombo, files. As far as we know Shape has not yet been developed to import HDF5. Currently it accepts ASCII files. Therefore the simulations you want to visualize will need to be in ASCII form.

Shape can read and visualize 3D simulations. However make sure that the columns in your ASCII file are converted to a 3D Cartesian coordinate system. You will need a minimum of 7 columns, of the form: px, py, pz, vx, vy, vz, n. Where all pi are the positions in (x, y, z), all vi are the respective velocities, and n is the density. One can also have a column for temperature T, and pressure P. Unlike VisIt, where you visualize the time evolution of a simulation frame-by-frame, Shape will only require one frame to be visualized. In Shape, you can rotate it and study the morphology at that instant. It may be best to visualize with VisIt first, pick your frame, and then convert it to ASCII. See the Before Starting section for more on this.

Remember that Shape's goal is to be a software that reduces restrictions on physical assumptions so that astronomers can make a realistic reconstruction of an astrophysical object's morphology. Thus the software is useful for defining 3D structural elements to create a model that can be optimized to fit an observation. Here, by importing external data, we are approaching from the opposite angle. We are using a code to generate a simulation of an object, or phenomena, that is comparable to actual astrophysical objects. In Shape we can then visualize, and compare these simulations more easily with observation. Expect the animations you make with Shape to not look as defined as those one could make in VisIt with the same simulation, as they are meant to look like emission maps.

Links

• ​Shape Developers' Documentation on External Data Visualization

• ​Shape Developers' YouTube Tutorial on External Data Visualization

---

Tutorial by Example

In this tutorial in visualizing external data in Shape, we will be visualizing a sphere generated by the following script:

Pseudo ASCII data generator (python)

import random
from math import pi,sin,cos

#Creating Sphere Dataz                                                                                                         
def createSphere(r=5, N=100):
    lst = []
    for phi in [(pi*i)/(N-1) for i in range(N)]:
        M = int(sin(phi)*(N-1))+1
        for theta in [(2*pi*i)/M for i in range(M)]:
            x = r * sin(phi) * cos(theta)
            y = r * sin(phi) * sin(theta)
            z = r * cos(phi)
            lst.append((x, y, z))
    return lst

#Opens/creates new ascii file                                                                                                  
outfile = open("test_sphere_7col.dat", "w")

#Writes the data to the file                                                                                                   
for x,y,z in createSphere():
    rho = random.random()*1000000
    vx = random.random()*10
    vy = random.random()*100
    vz = random.random()                                                                                                      
    print("{0}\t{1}\t{2}\t{3}\t{4}\t{5}\t{6}".format(x,y,z,vx,vy,vz,rho), file=outfile)

#Closes the file                                                                                                               
outfile.close()

a file generated by the script can be found in the attachments, titled test_sphere_7col.dat. In the following sections you'll observe how we come to visualize its "emission map." We suggest downloading the attachment and following along with the tutorial.

Before Starting:

Before Starting

1. Make sure your simulation is in ASCII form. Hopefully you will find this ​guide on how to convert HDF5 to ASCII helpful.

2. The basic format you will want for your ASCII file is a series of 7 columns minimum. They will be of the form: px,py,pz,vx,vy,vz,n, the Cartesian coordinates of position, velocity and the density.

3. Import the data into Excel. Take the averages of all of the columns. You can open the file in terminal and scroll to the bottom to determine how many lines there are. You will use these values later to center your data.

Image 1. A screen shot of the Excel spreadsheet, where the pseudo sphere ASCII data was imported (test_sphere_7col.dat). The averages are bolded. Headers are above the averages, and the raw data below.

4. Once you have looked at your data to see what format it is in (number of columns, delimiters), and gotten the averages for each column, you can open Shape.

5. Create a new project: File → New.

6. Click on the 3D Module, it looks like a 2x2x2 cube.

---

3D Module:

3D Module

When you first open the 3D Module, it will look like this:

Image 2. A screen shot of the 3D Module once a new project is opened.

Now following the numbers in Figure 3:

Image 3. A screen shot of the 3D Module after creating a mesh sphere.

In order to import external data, we will need to use one of the mesh objects listed at the top of the 3D Module (Figures 2 & 3) (Sphere, Torus, Cone, Cube, Plane, Cylinder, etc.) to carry it into the software. This is all done in this module, and then we visualize it in the Render Module. We start in the General tab.

1. In this case we chose the sphere. Simply click the icon, then click in any of the four windows. Then click again to choose the size of the sphere. The size does not matter, as we will get rid of the sphere later on. We just need it to carry the data into Shape.

2. Note that under the general tab we can change the name of the object. We do this here in Figure 4 (below), to External Sphere. You don't have to change it. But it might be useful in case you want to import other simulations in the same session.

3. Choose the ASCII format, as that is the format our simulation is in. Then click the square button next to the drop down tab, to import the data in the Input Parameters dialog. This is what it looks like before we direct Shape to where our ASCII file is.

Image(s) 4 & 5. (left) The 3D module after opening the ASCII renderer, and having changed the name of the mesh object. (right) The input parameters for the ASCII simulation we are aiming to import into Shape.

On Input Properties:

Input

1. Filename: The path used to reach your ASCII file is input here.
2. dNum: Keep this as the default (1E4).
3. Format: As seen in Figure 4 the default is px,py,pz,n,vx,vy,vz,T,P. These are all the variable columns you may input to Shape: position, density, velocity, temperature, and pressure. However in this case we are simply going to use the position, velocity coordinates, and density. Therefore the format will be px,py,pz,vx,vy,vz,n.
4. Comment: How comments are denoted in your ASCII file. Keep the default of #.
5. n: Keep this as the default (1).
6. Lines per entry: How many rows are in your ASCII file? Open the file in whatever text editor your prefer, and get the number for the last line. In this example there were 6284 rows, or lines per entry.

Column Separator

How is your file delimited? Typically when you input the filename, Shape will adjust the column separator automatically. However check that it is correct.

Center

For the x,y,z inputs, enter the averages from your Excel file for the position coordinates. If you click to detect extents after entering the averages, they should be 0,0,0.

Preview

Allows you to preview your data file.

Now you have imported your data. However you will not be able to see it in the 3D Module yet. Now move onto the Modifiers tab.

Image 6. Under the Modifiers tab in the 3D Module.

After importing the data under the General tab, we now move to the Modifiers tab. There you will see two modifiers (temperature, density). Delete these using the X button, while they are highlighted. A new window will appear asking if you want to delete these.

Now we will want to move into the Physics Module:

Image 7. In 3D Module directing you to the Physics Module for the purpose of creating a species for our emission map to be viewed in the Render Module.

Click the icon at the top of the Shape GUI that looks like a chalkboard. Now move on to the next section of this tutorial.

---

Physics Module:

Physics Module

In this section of the tutorial, we will be making a species for our emission map. To do this we will need to create a new custom species, and edit the emission coefficient so that it uses a Gaussian template. Depending on what sort of simulation you've made, you may want to do something different.

Image 8. Screen capture of the default physics module. Note the red circle, indicating where to create a new species.

1. Click the Add button at the top of the Physics Module.
2. After doing this, you will be prompted to add a certain type of species. There are many options (Quick, Custom, Atomic, Molecule, Dust, Scale, Multiple, Plasma, Black Body), however here we will choose Custom.

Image 9. Prompt for adding a species. Here we have selected a Custom species.

3. Once you have clicked Ok the Physics Module will now give you the option to edit a Custom species. The module will look similar to what is presented in Image 10 (sans red text, of course).

Image 11. The Physics Module after creating a new Custom species. Note the only user change here is the Name of the species, which is "Line."

Now that you have created a new Custom species we need to edit it.

4. Here the Name of the species has been changed to "Line." However one may choose whatever name they prefer. The same name is used in the youtube tutorial provided above.
5. Note that once you choose a Custom species, the main view in the Physics Module may still be that of the Default. Simply switch from Default to Custom in the left-hand Species list by clicking the Custom option you have made.

In this tutorial we will simply keep the default options illustrated in Image 11. Here the Emission contribution is sufficient. Now we go on to edit the Emission Coefficient, which is indicated in Image 11.

6. Click the Edit button.
7. Also note that if you scroll down the main options page in Shape for your new Custom species, you'll see that we have n, or Density, as our variable mapping.
8. Recall that our format string for our imported data is in terms of the columns with headers px,py,pz,vx,vy,vz,n.

Image(s) 12, 13, & 14. The Emission Coefficient dialog.

Once you click Edit for your emission coefficient, you'll see a new dialog pop up as illustrated in Image 12. You are given the analytic function:

f(x) = 1E-15*n^2

We do not want to use this analytic function.

9. So as illustrated in Image 12, click the Templates option and choose Gaussian. There are the options of (Planck (m), Gaussian, Lorentzian, Wein).
10. After clicking Apply, you will notice that the analytic function, shown under Functions has changed to:

[Analytic] 1.0/(sqrt(2*pi) * c) * e^(-(x-b)^2/(2*c^2))

11. Notice that this equation does not involve our variable mapping term, n}}, otherwise known as density. As illustrated in '''Image 13''' click back to the ''Function'' property, and delete the coefficient on our Gaussian function, {{{ 1.0/(sqrt(2*pi) * c) and replace it with n.
12. Then hit enter in the white box, so that it shows up under your Functions.
13. Also note that there are other variables in our Gaussian function, namely, x,c,b. Clearly x is the variable of the function, and not a constant, where as c, b are constants. As illustrated in Figure 14 input the following numbers:

x = 9.8
c = 5.0E-7
b = 2.0E-11

Leave n alone. Note:

• n, the coefficient, is the height of the Gaussian's peak.
• c is the standard deviation, and controls the spread of the peak. Here we have it spread at 5.0E-7 Angstroms, which is essentially corresponds to the energy of a photon in a vacuum (gamma rays). This is similarly the case with b
• b is the position of the center of the Gaussian peak.
• Thus our emission is within the realm of visible light in the spectrum.
• We simply choose x based on what is recommended in the video tutorial.

14. When you are done with the Emission Coefficient dialog, simply click the X-out button at the top of the dialog.

You are done. Now we can go back to the 3D Module, so proceed to the next section of this tutorial.

Image 15. The 3D Module is the 2x2x2 cube near the left-hand corner of the Shape GUI. See the red circle.

---

3D Module:

3D Module

Importing Species from Physics Module

Now that are we are back in the 3D Module we will need the species we have just made. So we need to import it into this module.

Image 16. The Species tab in the 3D Module.

1. Go to the Species tab in the 3D Module.
2. You should see no species are listed.
3. Go to the drop down, and choose the new species you have made. In this tutorial we have named it "Line."
4. Click Add.

Importing Particles

Now we move on to importing our data as a set of points, or Particles. In the 3D Module you will see on the right-hand side that there is a tab for Particles.

1. Click the Particles tab.

Image(s) 17, & 18. (Right) The Particles tab: red boxes indicate that you want to load data for a Volume rendering. (Left) The dialog filled in for the data set test_sphere_7col.dat after pressing Load in the Particles tab.

2. Click Load. See Figure 17.
3. A new dialog will pop up. This will look comparable to how we imported our ASCII file originally in the General tab. However now we need to do it for particles as well. Note that not everything will be the same. Here just do the following:

• Import the file under Filename.
• Change the Format string to px,py,pz,vx,vy,vz,n, or whatever format you used originally in the General tab.
• Make sure the Column Separator is correct.
• Input the same values for the Center as you did originally. See your excel file for reference if necessary.

4. In the dialog, when completed, press OK.

Image 19. The data points from the ASCII file are indicated by the red circle in the Right view. However note that it is present in all of the views.

Now you will see that your ASCII file has been officially imported into Shape, as you can see by Figure 19.

Image 20. Zoomed up on the ASCII file imported into Shape.

You may use your mouse, or the Camera options to zoom in on your data in any of the perspectives of the 3D Module.

Note:

• You can edit the Size of each of the particles.
• You can change the percentage of your data, or particles, displayed.
• You can also change the Seed.

Now we want to rid ourselves of the mesh sphere we used to import the data originally on.

Image(s) 21, & 22. (Right) The default Primitive tab. Note that originally we have a mesh sphere of radius 45.6395. (Left) Now we have a custom object.

5. Click on the Primitive tab.
6. In the drop-down that says Sphere, switch from sphere to Custom. You can see this clearly in Image(s) 21 & 22.

Image 23. Our imported ASCII file, as viewed in the 3D Module without a mesh object.

If the simulation looks like a series of points, such as the sphere we can see in Image 23, you have successfully imported your data into Shape. Now we can move on to Render the imported data, to create an emission map.

Note:

• One may use the modifiers listed under the Modifiers tab to rotate their data, etc. This may enhance the emission map.

---

Render Module:

Render Module

Under construction

Image 24.

Image 25.

Image(s) 26 & 27.

Image(s) 28, 29, 30, 31, & 32.

Image 33.

Image 34.

---

Open Questions

Under construction