Publishing catalogues with SAADA

Software configuration and installation

Java 1.5

Java JDK 1.5 can be installed from the workshop CD.

Tomcat web server

Install Tomcat 5.0 from the workshop CD.

Aladin

Download the workshop-customized Aladin.jar

PostgreSQL

Setup for Linux and Mac OS

First download and install PostgreSQL :

• Linux users : choose appropriate package from http://www.postgresql.org/ftp/binary/v8.2.4/linux/
• Mac users : download Postgres for Mac from http://sourceforge.net/project/showfiles.php?group_id=133151&package_id=147280

During the installation, a user postgres will be created. This user owns the database system.

Detailed instructions about setting up PostgreSQL can be found here : http://amwdb.u-strasbg.fr/saada/article.php3?id_article=108
Following is a summary of these instructions. We assume an account called johndoe, whose HOME directory is /home/johndoe/ . We also assume that Postgres has been installed in a repertory POSTGRES.
You might want to grant johndoe to execute some commands as postgres by adding the following line in /etc/sudoers.

johndoe     ALL=(postgres) /usr/bin/initdb, /usr/bin/psql, /usr/bin/pg_ctl

You need first to initialize your database. From your home directory, create a directory postgresdata :

mkdir /home/johndoe/postgresdata

and change user owner to postgres :

sudo chown -R postgres postgresdata

This postgresdata directory is a placeholder for Postgres tables.
Then, type the following command to initialize Postgres database :

sudo -u postgres initdb -D /home/johndoe/postgresdata

You are ready to start the DBMS :

sudo -u postgres pg_ctl -D /home/johndoe/postgresdata -l /home/johndoe/postgresdata/postmaster.log start

If something went wrong, have a look in /home/johndoe/postgresdata/postmaster.log to solve the problem.
Otherwise, you can try to connect to the DB using the interactive client psql :

sudo -u postgres psql

Type \q to quit psql.
We will now change Postgres security settings, in order to authorize any user to connect to the DBMS.
First, stop the database :

sudo -u postgres pg_ctl -D /home/johndoe/postgresdata stop

Edit the /home/johndoe/postgresdata/pg_hba.conf file and modify it as follows :

# TYPE  DATABASE    USER   CIDR-ADDRESS   METHOD
# "local" is for Unix domain socket connections only
local   all         all           trust
host    all         all    127.0.0.1/32   trust

Save your changes, and restart Postgres :

sudo -u postgres pg_ctl -D /home/johndoe/postgresdata -l /home/johndoe/postgresdata/postmaster.log start

Try to connect as user postgres on database postgres :

psql -U postgres postgres

Make sure this step works properly before going further.
If yes, create a role for johndoe allowing him to login to Postgres and to create databases (don't forget the final ';') :

 postgres=> CREATE ROLE johndoe CREATEDB LOGIN;
 postgres=>\q

We can now try to create and drop a database as johndoe :

createdb TEST
psql TEST
postgres=>\q
dropdb TEST

If you succeeded, Postgresql setup is fine for using SAADA.

Setup for Windows

Retrieve the Windows installation package from the workshop CD (in directory CDS/softwares/postgresql/windows) or from http://wwwmaster.postgresql.org/download/mirrors-ftp?file=%2Fbinary%2Fv8.2.4%2Fwin32%2Fpostgresql-8.2.4-1.zip

During the installation, you will be prompted for a password for the postgres user. Please write this password down, as it will be needed in the next steps, to connect to the DMBS for instance.
Once installed, Postgres can be easily started up and shut down from the Postgres item in the Startup-->Programs menu.

Postgres comes along with an administration tool called pgAdmin . Launch pgAdmin, startup Postgres if not already done, connect to the server (right-click on item), and create a role with your Windows username allowed to login and create databases (LOGIN and CREATEDB attributes).
Edit the pg_hba.conf file (accessible from Startup-->Programs-->PostgreSQL-->Config files) and modify it as follows :

# TYPE  DATABASE    USER   CIDR-ADDRESS   METHOD
# "local" is for Unix domain socket connections only
local   all         all                   trust
host    all         all    127.0.0.1/32   trust

Add POSTGRES\bin to the PATH environment variable.

SAADA

SAADA introduction.

Download latest SAADA release corresponding to your platform from http://amwdb.u-strasbg.fr/saada/article.php3?id_article=100
If you already installed SAADA, replace it with the latest version which has new features we will use during this hands-on session.
Installation is pretty straightforward, don't forget to fill the Tomcat path field.
When prompted for the Saada operator name, enter your username, for instance johndoe .

Once you have installed SAADA, use the DEMO to check your setup.

The detailed instructions are available here.

• Make sure the Postgresql DBMS is running
• Run the script ${SAADAHOME}/demo/scripts/buildDemo (buildDemo.bat for Windows users) : this will populate your DEMO database
• Make sure Tomcat is running (TOMCAT_HOME/bin/startup.sh)
• Connect to http://localhost:8080/DEMO
• Click on Database Map and browse the DEMO collection

Publishing test data to the VO

1. Description of test dataset

We will work on a small subset of the RC3 catalogue, containing the 50 galaxies of the Southern hemisphere with the largest angular diameter.

You will find on the CD a csv file, rc3.csv, containing the data, and a description file, rc3.desc, giving a basic description of the columns.

You will have to ingest this list of sources in SAADA, and build a ConeSearch service to access these data.

2. Saada configuration

There are a few steps before uploading your data.

2.1 Creation of a SaadaDB

Details on this page

• Create two directories named saadadbs and repository
• You need to create the database in Postgresql createdb workshop (or via pgAdmin under Windows)
• Then create the saadadb
  • Chdir to the directory SAADA_HOME/bin/
  • Run the script newDB (newDB.bat for Windows users)
  • In the GUI, type workshop for new Saada database and DBMS database name
  • Enter your login name in DBMS user name
  • Set web server host name to http://localhost:8080/workshop
  • Fill the root directory and repository with the full paths to directories saadadbs and repository
  • Click on Create Saada Database
  • Exit the GUI

2.2 Creation of a Collection

• Chdir to SAADADBS/workshop/bin/
• Run the script saadadbgui (saadadbgui.bat for Windows users)
• Go to the Collections tab
• Set Collection name to rc3
• Click the Create button

2.3 Creation of a Configuration

• Go to the Sources tab, and Creation sub-tab
• Enter rc3_config in the Product name for this configuration field
• Set Identification of the files matching ... to By filename match , and enter .*xml
• Select One class for all product files as Class mapping strategy
• Set Class name to rc3_sources
• In Optional mapping, set the 2 first fields to _RAJ2000 and _DEJ2000
• Click on Save

3. Ingesting data

• Download the rc3.xml data file to a distinct local directory
• Go to the Sources tab, and Loading sub-tab
• Select the previously built configuration
• Enter the directory containing the data product rc3.xml
• Press Load

4. Providing metadata

Most of the metadata were already included in the VOTable file rc3.xml. However, some UCDs were missing. Saada offers the possibility to retrieve them from the column description :

• Relaunch saadadbgui for this SaadaDB
• Go to the UCD tab
• Expand the Collections folder, and select the Class
• Update the attributes where the UCD is missing

Saada also allows loading of FITS tables, like rc3.fits. If the FITS file does not contain the UCDs, they need be added later. The UCDs corresponding to the column descriptions can be assigned using online tools, like the UCD builder. They can be added by updating the attributes of of the sources entries in Saada.

When starting from a basic CSV file containing very little metadata, TopCat or the STILTS library can be used to convert the file to FITS. The UCD assignment can be automated using the Web Services, as was described in a previous tutorial.

5. Acces data with a web form

Before accessing our data from the Saada web interface, we need to deploy the service to Tomcat :

• Run SAADADBS/workshop/bin/tomcatdeploy
• Make sure Tomcat is running ( launch TOMCAT/bin/startup.sh )

• Open a browser, and go to http://localhost:8080/workshop/
• This is the home page of the web interface of your SaadaDB
• Click on Database Map to visualize all ingested sources
• You can ask Saada to generate a FITS or a VOTable document with all sources (click on Get a FITS table or Get a VOTable)
• If clicking on the Det green button in the first, you will access to the details of the corresponding source
• Querying by constraint is also possible : click on HTML Editor from the SaadaDB homepage
• Set the category of searched data to Entries
• In the Additional constraints panel, enter the constraint Declination Less than -65 , and click on Apply
• Finally, execute the query by clicking on Submit query

6. Cone Search in VO interface

• From your SaadaDB home page (http://localhost:8080/workshop/), click on VO portal
• This web form allows you to perform ConeSearch queries on data previously ingested
• In the CS queries panel, set RA to 00 50 00, DEC to -72 50 00 , Collections to rc3(rc3_sourcesEntry) and press Submit CS query . The result is a VOTable file conforming to the ConeSearch standard, ie having the 3 mandatory fields (ucd=ID_MAIN, ucd=POS_EQ_RA_MAIN, ucd=POS_EQ_DEC_MAIN)

7. Simple access in Aladin

We will now see how to access our ConeSearch service from a VO tool : Aladin.

• Open the SaadaDB home page ( (http://localhost:8080/workshop/)) in a browser, and click on VO portal
• In the =CS queries panel, set Collections to rc3(rc3_sourcesEntry), choose get Register and press Submit CS query
• The result is a GLU record, a description of our service which will allow Aladin to query it
• Select in your browser lines starting with a '#', and copy/paste them to a new text file named myservers.txt
• Start Aladin by typing java -jar Aladin.jar -beta -glufile=myservers.txt. The -glufile parameter points toward a file describing additional servers
• Once Aladin started, click on Load
• In the right most column, click on Saada CS-->Saada CS workshop
• Set the target to LMC for instance and press the Submit button
• You have successfully queried your ConeSearch service from Aladin !

In the next section, we will learn how to register our ConeSearch service, so that it can be automatically discovered and accessed from VO client tools, without prior knowledge of its existence.

8. Registering in the test registry

• Open the conesearch.xml template file in a text editor. We will modify this template to adjust it to our service
• Change the title
• Change the shortName
• IMPORTANT : change the identifier to ivo://org.astrogrid.demo/saada/conesearch/your-last-name . This identifier has to be unique.
• IMPORTANT : change the accessURL with the base URL of your ConeSearch service. Replace localhost with the IP of your laptop (find it by typing ifconfig ). The accessURL should look like http://131.176.165.61:8080/workshop/conesearch?collection=[rc3(rc3_sourcesEntry)]&
• Once the template file has been modified, point your browser to http://beacon3.star.le.ac.uk:8080/astrogrid-registry/
• Set current contract to 1.0
• Click Enter resource
• Enter login=astrogrid, password=W6G117
• We will use the Upload from a local file option. Check the Validate radio button, enter the path to your modified conesearch.xml file, and press Upload
• If no error message appears, your service has been successfully registered. It should appear now in the list of resources available from the Registry browser (Browse link in Investigate menu)

9. Consume service in Aladin

Once your service has been registered in the workshop test registry, VO client tools will be able to automatically discover it.

• Launch Aladin by typing java -jar Aladin.jar -beta
• Click on Load
• Go in the All-VO tab
• Press on the Detailed list button
• The new window lists services entries coming from the NVO registry
• Click on Query workshop registry and click on Refresh
• The workshop registry will then be queried, and discovered services will appear with a pink background
• Unselect all services, then select services of your interest (for instance the one you registered previously)
• In the Aladin server selector, set the target to LMC, and press Submit

Building your own Cone Search service

-- SebastienDerriere - 21 Jun 2007

• conesearch.xml: Cone Search template resource description

• rc3.xml: RC3 sample in VOTable format

• rc3.fits: RC3 sample in FITS format

• SAADA_introduction.pdf: SAADA introduction