Publishing data with Saada (images, spectra, catalogues)
Abstract
Saada transforms a set of heterogeneous FITS files or VOTables of various categories (images, tables, spectra....) into a powerful database deployed on the Web and provides standard VO interfaces to the data (ConeSearch, Simple Image Access, Simple Spectrum Access). The database (a SaadaDB) is located on your host and stays independent of any external server. This job does not require to write any line of code.
In this session, participants will go through the following steps : creating a Saada DataBase, ingesting the data files in the database, assigning metadata, registering the service in a VO registry.
Finally, participants will be able to access the data from various VO tools using standard interfaces provided by the Saada system.
Participants are highly encouraged to bring their own datasets, as the ultimate goal of the session is to teach how they can easily deploy VO services for their own purposes.
Supported systems
Saada has been successfully tested under:
- MacOS 10.4
- Linux 32 and 64-bits
- Windows XP
- Windows Vista (with difficulties: see Saada Web page
)
External References
Advisors (CDS)
- Laurent Michel
- Thomas Boch
- Sébastien Derrière
Software Requirements
Additionally to the
common workshop software requirements, this session requires:
Step by Step Tutorial
Notice
- The Saada package delivered with the workshop material is more advanced than the 1.5.beta3 available on line. We stronly suggest to use it instead of the public one.
- The Aladin 5.028 beta version
must be used to handle VOTables with associations
- Worshop material (software and data) can be found here
Session Case
- A set of data focused on M33 is provided for the workshop.
- 5 images (XMM, DSS, NVSS, EGRET and HST)
- 38640 sources in 5 tables (XMM, bright UV, SN remnants, variable stars and planetary nebulae)
- 335 spectra (UV and X)
- Following configuration parameters are related to this dataset.
PostgresQL Installation:
The details of the installation procedure depends on the plaform, but the sketch always remains the same. Shell commands are replaced on Windows with adapted clicking. The database system initialisation is made by the Windows installer.
- Software installation (sudo or root)
- Database system initialisation (sudo or root):
initdb -D data_directory
. Give data_directory
enough room to host your databases.
- Server startup (sudo or root):
postgres -D data_directory
- Account setup: see Saada Web page
. Roles can be created either by command lines or by using the Pgadmin
utility. This utility is included within the Windows package. It must be installed by hand on others platforms.
- Create the roles used by the data loader. This role must have administrator privileges
createuser -s [-p password] saadmin
. Password is optional.
- Create the role used by Tomcat (Web interface). This role just requires a connect permission:
createuser -l [-p password] reader
.Password is optional.
Tomcat Installation
- see Saada Web page
.
- Once the installation is complete start the server (
TOMCAT_HOME/bin/startup.sh
. On Windows, Tomcat must be started with the task monitor start->programs->Tomcat->Monitor Tomcat
- Try to connect
http://localhost:8080
Setup of the Workshop Directory
- Create the following directories (doc
):
-
saada
Saada distribution installation directory
-
saadadbs
Directories where database created by Saada (the SaadaDBs) will be installed. This directory must not be the same as saada
directory
-
repository
Directory where ingested files are copied in order to feed up the download facility.
Saada Installation
- Download the Saada distribution (jar file
SaadaXXX.jar
) from http://amwdb.u-strasbg.fr/saada/DCA2008/software
- Run that jar file :
java -jar SaadaXXX.jar
- Choose the
saada
directory created above as target for Saada install
Create an empty SaadaDB.
- Go in
install_dir/Saada1.5.beta3/bin
and run ./newsaadadb
(see here
)
- The database creation is a step by step process. System Parameters (directories, PSQL ...) values are given above. Parameters specific to the SaadaDB are given below:
SaadaDB name |
e.g. DCAWorkshop |
SaadaDB Installation Directory |
saadadbs |
SaadaDB Repository |
repository |
-
- Step 5 (click Add for each attribute):
User Attribute Definition for images |
Survey (String) |
All images will be tagged with the name of the survey they come from |
User Attribute Definition for spectra |
Domain (String) |
All spectra will be tagged with their spectral range |
User Attribute Definition for entries |
ObjectClass (String) |
All table entries will be tagged with their object class |
Gobal Spectral unit |
Wavelength (nm) |
The dispersion range of input spectra will be converted (as far as possible) in nm . This conversion relies on light speed in vacuum. |
Populate the SaadaDB.
- Go in
install_dir/saadadbs/DCAWorkshop/bin
and run ./saadmintool
(see complete doc here
)
- Create a data collection
M33
- Loader parameters for images.
- Select the=Data Loader Configuration tab=
- Select image
- Open the data file you want to load with the "Data Sample= button
- Fill the parameter as show below. Quoted mapping parameters are taken as constants. Others are keyword names.,
Apply
and Save
- Right click on the image node on the dta tree
- Select
Load using the current configuration
Config Name | Class Name |
Instance Name | Ignored att. | Survey |
DSS | DSS (Fusion) |
'DSS',OBJCTRA,OBJCTDEC | AMD* | 'DSS' |
EPIC | EPIC_Img (Fusion) |
'XMM',OBJECT,OBSERVER | EM* | 'XMM' |
NVSS | NVSS (Fusion) |
TELESCOP,INSTRUME,OBSRA,OBSDEC |
| 'NVSS' |
HST_IRAF | HST_IRAF(Fusion) |
OBJECT,ORIGIN |
| 'IRAF' |
- Loader parameters for spectra.
Config Name | Class Name | Instance Name |
extension | Domain |
Position | Spectral Range |
HST | HST (Fusion) | TARGNAME
|
#0 | 'UV' |
RA_TARG,DEC_TARG mapping=only |
range='1085 1605' unit=Angstrom mapping=only |
|
RGS | RGS_Spc (Fusion) | OBJECT
|
SPECTRUM | 'X-ray' |
no mapping |
range='0.6 6' unit=Kev mapping=only |
|
EPIC | EPIC_Spc (Fusion) | OBJECT
|
SPECTRUM | 'X-ray' |
no mapping |
range='0.2 12' unit=Kev mapping=only |
|
- Loader parameters for tables.
Config Name | Class Name | EntryName |
extension | ObjectClass |
Position | Position Error |
XMM_Cat | XMM_Catalog (Fusion) | IAUNAME |
| 'X-ray Source' |
RA_SIAP,DEC_SIAP mapping=only | RADEC_ERR |
PN | PlanetaryNebulae (Fusion) | 'PN',_RAJ2000,_DEJ2000 |
#0 | 'Planetary Nebulae' |
| |
SNRemnants | SNRemnants (Fusion) | 'SNR',_RAJ2000,_DEJ2000 |
#1 | 'SN Remnant' |
| |
UVBright | UVBright (Fusion) | 'UVB',_RAJ2000,_DEJ2000 |
#0 | 'UV Bright' |
| |
VariableStar | VariableStar (Fusion) | 'Var',_RAJ2000,_DEJ2000 |
#1 | 'Variable Star |
| |
Name | Starting Collection | EndingCollection |
Qualifier | Correlator |
SourcesInField |
M33.IMAGE | M33.ENTRY |
| Trivial (All images linked with all entries) e.g. p.oidsaada > 0 |
CandidateSpectra |
M33.ENTRY | M33.SPECTRUM |
distance | 1st spectrum at less than 10 arcsec |
SaadaDB access.
Your database can be accessed by the Web once it has been deployed (Deply Web App.
button on the admin tool).
- The base address of the SaadaDB is http://localhost:8080/DCAWorkshop
- The Web interface relies on SaadaQL queries (see doc [[http://amwdb.u-strasbg.fr/saada/spip.php?rubrique124][hre])
- Simple queries can be edited from the form on the bottom if the data page.
- The
Refine Query
opens a smart editor of SaadaQL.
- Query Examples:
- Select any entries at less than 1 arcmin from the M33 center
Select ENTRY From * In M33
WherePosition{ isInCircle("M33",1.0,J2000,FK5) }
-
- Select any entries at less than 1 arcmin from the M33 center and having one candidate spectrum
Select ENTRY From * In M33
WherePosition{ isInCircle("M33",1.0,J2000,FK5) }
WhereRelation {
matchPattern {
CandidateSpectra
}
}
-
- Select variable stars at less than 1 arcmin from the M33 center and having one candidate spectrum
Select ENTRY From VariableStarEntry In M33
WherePosition{ isInCircle("M33",1.0,J2000,FK5) }
WhereRelation {
matchPattern {
CandidateSpectra
}
}
VO Publishing
The SaadaDB comes with 3 VO services (one SIAP for images, one Cone Search for sources and one SSA for spectra). These services can accessed by the voportal page http://localhost:8080/DCAWorkshop/voportal
Notice that your database can only be accessed from your local machine because its base URL was set as http://localhost...
. It should be run on a machine with a fixed IP address to be visible from other PCs.
- VO access from the Web interface:
- VO services can accessed by the voportal page
http://localhost:8080/DCAWorkshop/voportal
- Each Saada collection can be seen as one VO resource as well as the whole database (collection
ANY
).
- Saada VO queries always return VOtables.
- The
XML
button makes VOtables to be displayed on the browser
- The
Aladin
button starts the Aladin applet with he returned table. This feature does not support associations.
- VO access from the GLU
- A local registry (a
gludic
can used by Aladin). All resources declared in that gludic
will be seen by your local Aladin as any other VO resource.
- GLU marks (similar to registry records) are delivered by the vo portal when the
Glu Mark
box is checked.
- Copy these marks in a file (e.g.
gludic.txt
)
- Run aladin with the following command:
java -jar Aladin.jar -glufile=/PATHTO/gludic.txt
- VO resources of your SaadaDB will appear and can be queried from the
load
window (menu File-->Open
).
- VO access from the registry
- The
Registry Record
check box returns an XML template of a valid registry record. This template can be completed by hand and declared in a registry. Your SaadaDB becomes then a VO resource.
- Notice that being declared in a VO registry supposes your services to have a world wide access and a high disponibility.
- Valid registry records can be downloaded here
- Update these files with your identity
- Cut and past file content here
(user name: registry , password : test ) and submit.
- The declared resources are visible from the Aladin
All VO
frame :
- Go in menu
File-->Open
- Click in the
All VO
tab, and then click on Detailed list
- In the new window, select
Query workshop registry
and click Refresh
- You should see your registered services in the displayed list
- Select and query one of your service using target
M33
- Published services can be accessed by any client (e.g. Aladin) running in local.
Meta Data Mapping
Once data are ingested in the SaadaDB, units, UCDs and Utypes can be mapped on keywords (see doc here
). The mapping is set class by class:
- Double click on the class icon on the data tree. That opens the meta data tab.
- Right click on the metadata tab and select
Tag Meta Data
- Drag and Drop metadata from the relevent tree to the metadata row.
- Right click on the metadata tab and select
Save Meta Data
- Deploy the Web interface again.
These metadata are used by SaadaQL to express constraints:
- Set UCD
phot.mag;em.opt.V
on the column Vmag on the VariableStar
Class .
- Set this keyword as
queriable
(means queriable by UCD)
- Do the same for the
UVBright
class.
- Try the following query:
Select ENTRY From * In M33
WhereUCD {
[phot.mag;em.opt.V] [] (19.7,20) [unit]
}

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback