ESA-VO DALToolKit (Images, Spectra)

Aurelien Stebe & Inaki Ortiz

Abstract

SIAP (Simple Image Access Protocol) and SSAP (Simple Spectrum Access Protocol) are two successful protocols in the Virtual Observatory context. Due to the simplicity and flexibility of these two simple protocols, it is easy to adapt existing archives to be VO compliant. It is also easy to make use of the available protocols to consume images and spectra in the VO context with appropriate clients and tools.

The goal of this tutorial session is to ingest and publish spectra / images according to the SSAP / SIAP standards. Participants will be able to locally create their own database, ingest test FITS files and publish them making use of the software and the support we will provide. Later on, people who wish to work with their own data will be able to perform the necessary tasks to achieve such result.

During this tutorial session, the user will follow all the necessary steps to create real SSAP / SIAP services on their machine. We will provide a few test FITS files which have to be ingested in a database according to the rules covered by a configuration file. Users will also be able to configure and install their own service which will display the data in SSAP /SIAP formats.

Tutorial steps

• Phase 1
  • Check your configuration and get all the necessary software
  • DALToolKit configuration files description and editing
  • Ingestion, compilation, deployment and testing of the sample
• Phase 2
  • Same process is repeated using participants FITS files
  • Deployment on Workshop server and registration of service
• Phase 3
  • Further the mapping efforts with participants custom data
  • Architecture, source code, extension points, questions, ...

Software requirements

• Workshop software requirements
• DALToolKit software packages

Tutorial guide

Check your configuration

First of all, you should check that you have installed correctly all the necessary software common to all the tutorial sessions. Check particularly that you get the right outputs to the verification steps at the bottom of the page. Ask a tutor for help if you get any problem.

Get the necessary packages

Then, you should retrieve on your machine, in a working directory all the DALToolKit software packages. These include the DALIngestor, used as the name suggest during the ingestion process, the actual DALToolKit server source package, SIAP_IMAGES and SSAP_SPECTRA, the test FITS data you will try to use first.

Note that two flavors of the DALIngestor and DALToolKit exist : one for MySQL users (DALIngestor and !DALToolKit) and on for PostGreSQL users (DALIngestor4PSQL and !DALToolKit4PSQL). The DALIngestor is quite specific to the RDBMS you are using and hence required two different versions. The DALToolKit source code is actually identical in the two flavors, be we prepared them with the correct specific database configuration for ease of use. Note that the PostGreSQL flavors are only available on the online DVD.

Deploy the DALIngestor and data

Now we will deploy the DALIngestor under the Tomcat application server and make the FITS data files available online.

You should copy the DALIngestor (DALIngestor directory or DALIngestor4PSQL.war file) inside the Tomcat webapps directory. Copy also the FITS data file directory under the same webapps directory : it is now time to choose if you will start by creating a SIAP service (SIAP_IMAGES directory) or a SSAP service (SSAP_SPECTRA directory).

You may now start your Tomcat server if it is not running already (go to http://localhost:8080/ to check if it is) : type startup.sh in a terminal

Create a new database in your RDBMS

Start your database server if it is not running already and create a new database for your DALToolKit service. Use the administration GUI or login directly into your database server using the Shell application. Under the shell terminal the command to create a new database is : CREATE DATABASE database_name;

Create a new database called SSAP or SIAP depending on your service type.

Edit the DALToolKit configuration files

We will now configure the DALToolKit service according to the test data you are using the type of service you wish to build.

Go to the DALToolKit directory inside the src/conf/ directory. If you are using the Images use the ULSSIA directory, if you are using the Spectra use the XMMSSA directory.

Open the ServiceConfig.properties file and check / edit the different values according to your database installation and set-up.

Open the RequiredParams.xml and OptionalParams.xml files and check / edit the various configured MetaColumns.
Here is the list of possible configuration values :

• ColumnName indicates the source of the data and can take the following formats : 'COLUMN_NAME' or 'SQL_FORMULA AS NAME'
• ParamName controls the input capability (delete to ignore)
• UTypeName controls the output capability (delete to ignore)
• FitsKeyword is not required/used by the DALToolKit server, but is by the DALIngestor
• Description is used in the output of 'FORMAT=METADATA' calls
• DataType, ArraySize, Unit and UCD should follow the VOTable v1.1 Recommendation, they are used for the VOTable output
• Values is used in the output of 'FORMAT=METADATA' calls, and can take the following formats : '[min,max]' or 'val1,val2'

Compile and deploy the DALToolKit server

To deploy the DALToolKit Server you have to compile it first. To compile the source code (Java) you will be using Ant, a Java based build tool. Once you have compiled the DALToolKit Server source code, you have to package it as a standard web application file (WAR) and install it into the web application server (Tomcat) were the service will be finally up and running.

Go back to the root of the DALToolKit directory and run the following ant commands :

• ant build to check everything is compiling correctly
• ant deploy to pack the service and send it to Tomcat
• note that you may also use ant undeploy or ant redeploy as needed

The Ant script will ask you for various inputs like the name of the configuration directory you wish to use (ULSSIA or XMMSSA), where to deploy (local) and your Tomcat password.

You may now open http://localhost:8080/ULSSIA/ or http://localhost:8080/XMMSSA/ to check the service is there.
You cannot query the service yet as we still need to ingest the metadata and create the database table.

Use the DALIngestor to extract the metadata

Open the DALIngestor application at : http://localhost:8080/DALIngestor/ or http://localhost:8080/D DALIngestor4PSQL/

Supply it with all the necessary information (service type, FITS files locations, configuration files) and launch the ingestion process.

Test your final DALToolKit service

You may now re-open your DALToolKit service and submit a test query to check everything worked fine.