Tutorial "Access to Gaia in CDS services" - Gaia data workshop @ Heidelberg
Thomas Boch

For easy access to URLs and easy copy/paste, an online version of the instructions is available at http://cds.unistra.fr/~boch/gaia-workshop-nov2016/ .
A PDF version is also available.

Each section (numbered 1. to 7.) can be done independently.
Sections 1 to 6 do not require any prerequisite ; section 7 requires some Python knowledge.

1. Explore Gaia DR1 in Aladin Lite: density map and HiPS 

Open http://cds.unistra.fr/Gaia/DR1/AL-visualisation.gml in your browser. By default, it shows a density map of Gaia DR1 sources. You can zoom in, zoom out and pan around.

For now, switch off the layer Gaia DR1 sources

Notice the satellite scanning patterns near the galactic center.



Click on the icon to jump to a given target. Type in Andromeda and submit. Zoom in if needed. The Andromeda galaxy is clearly visible.

Repeat the same process for objects M 33 and NGC 55.
You can switch the base image layer to DSS or 2MASS and see how these galaxies look like in optical or infrared.
 

Switch to the DSS image layer, and jump to LMC (Large Magellanic Cloud). Adjust zoom to frame the whole galaxy.
 

Switch on the Gaia DR1 sources HiPS progressive catalogue. When the field of view is large, only the brightest sources  are displayed.

Click on one of the source and see the associated measurements displayed in the right panel.
As you zoom in, fainter stars appear.
Click on another object and select More details to see the VizieR page with the whole set of measurements.

NB: the same steps can be performed in Aladin Desktop. Type http://cds.unistra.fr/Gaia/DR1/hips/GaiaSourceDR1-density-map-8192/ in Aladin command bar and submit to load the Gaia density map. HiPS progressive catalogues can be loaded from the HiPS form (menu File —> Open, tab HiPS, then open node Catalog )

2. Query Gaia tables in VizieR

2.1 Query table I/337/gaia

Open VizieR homepage http://vizier.u-strasbg.fr/viz-bin/VizieR in your browser, type Gaia in the first text field and submit

Click on I/337/gaia

Type Cen A as the target name and 20 arcmin for the search radius and submit


By default, VizieR outputs the result an HTML table, limited to 50 rows.
This limit, as well as the output format, can be modified from the left sidebar shown below:

Change the maximum to 9999 and resubmit to get the whole list of rows in the requested cone.

Clicking on  will show the sky positions of the rows in Aladin Lite.


Clicking on  will let you visualize 2D scatter plots on catalogue attributes.


In our resut, only default columns are visible. Click on Modify query, select a few more columns and Submit. Check that the selected columns are visible in the output.

We are now going to add a constraint on our positional query, in order to keep only sources with parallax measurement greater than 5 mas.
Click again on Modify query and add the constraint:

Submit. You should retrieve 14 sources matching the constraint.

This result can be easily transmitted to Virtual Observatory tools:



2.2 Query table I/337/cepheid

Go back to http://vizier.u-strasbg.fr/viz-bin/VizieR , search for Gaia and select the Cepheids table I/337/cepheid

Submit, change the max. number of rows to Unlimited as to retrieve all the 599 rows.

Click twice on the arrows above P1 label to sort the rows according to the period, in ascending order.


The first row is now the star with the shortest period.
Clicking on the corresponding fov link will show the different measurements for this star.

Click on  to display the light curve of this star.


We will now fold according to the cepheid period :




3. Retrieve Gaia measurements for stars in globular clusters, selected from Simbad

Open http://simbad.u-strasbg.fr/ and click on Queries by criteria

Type the following search expression 
otype = '*inCl' & Bmag>11 & Bmag<12 & nbref>5

This query will search for stars in cluster having magnitude B between 11 and 12 and cited in more than 5 papers

Submit query. 1638 objects match the criteria


Go back to the query page, select Display and resubmit the query.

Go to the bottom of the result page, and store result in the CDS Portal

On the following page, change the filename and click on Save


We can now use this result as input for the cross-match service.

Go to http://cdsxmatch.u-strasbg.fr/
At the top of the page, click on Table management and on Add metadata for the table we previously saved.
Change unit to sexa (sexagesimal) for both RA and DEC and click Update.


Go back to the tab
Choose simbad-stars-cluster as the 1st table and GAIA DR1 TGAS as the 2nd one.


Open the Options panel, change Radius to 1 arcsec and submit by clicking on Begin the X-Match
You can follow computation progress in the bottom table. Once the job is completed, click on the info icon to get a summary of the result and click on the histogram icon to see the distribution of distances for found associations.


Download the result as a VOTable, and load it in Topcat (File —> Load table)
Create an histogram (Graphics —> Histogram Plot) and visualize the distribution of the parallax in our sample.



4. Gaia data in Aladin Desktop

In this part, we will learn how to access to and visualize Gaia data from Aladin Desktop.

Launch Aladin with the following command:
java -Xmx1024M -jar Aladin.jar -beta

4.1 Visualize proper motion of Gaia stars in Aladin Desktop

In the location bar at the top of the main Window, type Praesepe and submit. Zoom out to see the whole open cluster
Open the server selector: menu File —> Open
Gaia DR1 can be accessed from a dedicated tab. 
Click on the Gaia tab, change radius to 2°, select table I/337/tgasptyc and Submit.

From menu Catalog, select Create a scatter plot and plot pmRA vs pmDE.
Stars which are member of Praesepe are clustered around (pmRA=-35mas/yr, pmDE=-12mas/yr)
Select these points in the scatter plot and notice they are automatically selected in the spatial view.


Go back to a single panel (menu View —> Panels —> 1 panel)
Create a filter by clicking on the icon and select the predefined filter Draw proper motions of stars.
To make the arrows bigger, switch to Advanced mode edit the filter as below and Apply:
{
  draw white plus
  draw pm(10*$[pos.pm;pos.eq.ra], 10*$[pos.pm;pos.eq.dec])
}
Size of arrows can also be controlled with a slider available in the stack after selecting the Gaia catalogue plane:


Select a few sources by drawing a rectangle. Associated measurements appear at the bottom of the window:


As the catalogue contains positions, proper motions along with the epoch of observation, Aladin can compute the positions at a different epoch:



4.2. Retrieve Gaia sources within HST coverage

Goal: In this subsection, we will learn to use MOCs in order to retrieve Gaia data which are in the common coverage between HST and SDSS observations.

Open the server selector (File —> Open) and click on the HiPS tab. Expand the nodes Image—>Optical—>SDSS and select SDSS9 colored.  Expand also nodes Image—>Optical—>HST and select HST-R
 
Submit.
The 2 corresponding HiPS are loaded. For each HiPS plane:


Open menu Coverage —> Logical operations , select the 2 MOCs and compute intersection.


The resulting MOC plane is the common coverage between SDSS9 and HST-R.
VizieR tables can be queried to retrieve only sources within a given MOC:

Zoom in and pan around to verify that returned rows fall into the MOC coverage.



5. Access to CDS cross-match service from Topcat : find Gaia and 2MASS couterparts for a list of sources and draw a color-color diagram


Download on your machine the table located at http://cds.unistra.fr/~boch/gaia-workshop-nov2016/data/allwise-SMC-sample.vot
It is a sample of 313k AllWISE sources located in the Small Magellanic Cloud.

Launch topcat:
java -Xmx1024M -jar topcat-full.jar

Load the table (File —> Load Table)

Open the CDS X-Match window from VO —> CDS Upload X-Match
Select Gaia DR1 in the list of VizieR tables
Select allwise-SMC-sample.vot as the input table.
Set the match radius to 1 arcsec and submit.



Once the cross-match is done, open the result table and notice that each row contains the fields of the original file plus the fields coming from the Gaia data.
Click on icon and create the histogram for column angDist (separation in arcsec between AllWISE source and Gaia source) as to assess the quality of the match.


In order to retrieve J, H, K magnitudes, we will now search for counterparts in 2MASS.
Open the CDS X-Match window, select 2MASS in the list of VIzieR tables and 1xGAIA DR1 as input table.


We should retrieve 185,417 rows.
Create a scatter plot for this result (Graphics —> Plane Plot) and plot X: phot_g_mean_mag - Jmag vs Y: Jmag - W1mag



6. More complex queries with TAP VizieR

6.1 Select high proper motions stars

Goal: retrieve 10 TGAS stars with higher proper motion near the galactic center

Go to TAP VizieR web interface: http://tapvizier.u-strasbg.fr/adql/

Search tables for the Gaia keyword.
Select I/337/tgas and click on Construct your query at the top right

Click on Sky area and enter Galactic center as cone center
Update radius to 5 deg, max records to 10 and click on Update query
Click on Quickview to get a preview of the result

Edit the query to keep only source_id, ra, dec and compute total proper motion:
SELECT TOP 10 source_id, ra, dec, sqrt(pmra*pmra+pmdec*pmdec) as pm
FROM "I/337/tgas"
WHERE 1=CONTAINS(POINT('ICRS',"I/337/tgas".ra,"I/337/tgas".dec), CIRCLE('ICRS', 266.416833, -29.007806, 5.))

Click on Quickview to get a preview

We still need to sort the result according to the proper motion value, using the ORDER BY clause.

The final query should look like this:
SELECT TOP 10 ra, dec, sqrt(pmra*pmra+pmdec*pmdec) as pm
FROM "I/337/tgas"
WHERE 1=CONTAINS(POINT('ICRS',"I/337/tgas".ra,"I/337/tgas".dec), CIRCLE('ICRS', 266.416833, -29.007806, 5.))
ORDER BY pm DESC

Click on Run to launch the query and retrieve the result in the requested format.

6.2 Recreate HR diagram from Gaia DR1 paper

Goal: we will recreate figure 3c of the Gaia Data release 1 paper ( https://arxiv.org/pdf/1609.04172v1.pdf )

Launch Topcat: java -Xmx1024M -jar topcat-full.jar

Open the TAP query window (menu VO —> Table Acces Protocol (TAP) Query)
Select and click on 


In the appendix of https://arxiv.org/pdf/1609.04172v1.pdf , table B.1 provides with the ADQL query used to create the HR diagram:

SELECT gaia.source_id, gaia.hip, gaia.phot_g_mean_mag+5*log10(gaia.parallax)-10 as g_mag_abs, hip.b_v
FROM gaiadr1.tgas_source as gaia
inner join public.hipparcos_newreduction as hip
on gaia.hip = hip.hip
WHERE gaia.parallax/gaia.parallax_error >= 5 and hip.e_b_v > 0.0 and hip.e_b_v <= 0.05 and 2.5/log(10)*gaia.phot_g_mean_flux_error/gaia.phot_g_mean_flux <= 0.05

This query is meant to be executed on GACS archive at ESA. A few changes must be made in order to make it work with TAP VizieR:
    As the dash is a special character in ADQL, the column name must be put between double quotes:  "B-V" and "e_B-V"

We will also add position fields ra and dec from TGAS

The updated ADQL query to be typed in Topcat is:
SELECT gaia.ra, gaia.dec, gaia.source_id, gaia.hip, gaia.phot_g_mean_mag+5*log10(gaia.parallax)-10 as g_mag_abs,
hip."B-V"
FROM "I/337/tgas" as gaia
inner join "I/311/hip2" as hip
on gaia.hip= hip.HIP
where gaia.parallax/gaia.parallax_error >= 5 and hip."e_B-V" > 0.0 and hip."e_B-V" <= 0.05 and
2.5/log(10)*gaia.phot_g_mean_flux_error/gaia.phot_g_mean_flux <= 0.05

Copy/paste this query in the ADQL text panel, at the bottom of the window and Run Query:


We should retrieve 74,817 sources.

Create a scatter plot (Graphics —> Plane Plot) and select B-V for X axis and g_mag_abs for Y axis.
Click on  and flip Y axis   .
Here is our HR diagram!



6.3 Retrieve spectral types from Simbad

We will now try to retrieve the spectral types of our sources, by querying Simbad through the CDS cross-match service.
Click on  , select simbad in remote table list. Select TAP_1_I_337_tgas,I_311_hip2 as input table and launch cross-match at 1 arcsec.

On the result table, we will create some subsets according to the spectral type of the star.
Click on  and define a new subet named O stars matching the following expression: sp_type.startsWith("O")



Repeat the same steps to create subsets for spectral types B, A, F, G, K and M.

Once the 6 spectral types have been created, plot the HR diagram following steps of section 6.2
Click on the Subsets tab, unselect All and select all 6 subsets:
    

You should end up with the following plot:



7. Accessing CDS services with Python

In this part, we will redo part of the previous sections, but in a programmatical manner, using Python.
A static version of the notebook is available at https://github.com/tboch/VO-access-GaiaDR1/blob/master/notebooks/gaia-cds-services.ipynb , providing examples on how to access Gaia data from CDS services.

If you want to modify the notebook and adapt queries to your own use cases, you can either: