Integrating Event Data with AstroGrid with STAP

Publication of the VOEvent message to the Event Backbone network can be done in three ways:

Parsing a VOEvents into a database

Setting up the perl software

1. download the VOEvent parsing module
2. install the module into your perl distribution.
   • if you have write access to the perl installation then probably the neatest way to install is using CPAN
   • if you do not have write access to your perl installation then
     • create a new directory to work in - call this $WORKDIR
     • download module from http://search.cpan.org/CPAN/authors/id/A/AA/AALLAN/Astro-VO-VOEvent-0.9.4.tar.gz
     • unpack the tar file tar xvf Astro-VO-VOEvent-0.9.4.tar.gz - this will create a $WORKDIR/Astro-VO-VOEvent-0.9.4
     • create a new empty directory under your working directory $WORKDIR/perlmodule
     • cd into $WORKDIR/Astro-VO-VOEvent-0.9.4
     • perl Makefile.PL PREFIX=$WORKDIR/perlmodule
     • make install
     • set enviroment variable PERL5LIB =$WORKDIR/perlmodule/lib/perl5/site_perl/
     • test that you have what you need with perl -e "use Astro::VO::VOEvent;"

Event metadata can be placed directly into a database. In this case, follow the instructions in the "create event tables" step. Upload data into MySQL using a form, a script using INSERT INTO, or mysqlimport. Alternatively, VOEvent metadata can be parsed from packets provided by a live VOEvent feed. In this case, follow both the "create event tables" and "parse packets from a VOEvent feed" steps.

Create event tables

Create a table for event metadata including event ivorn ("id" attribute from the <VOEvent> tag) url (location of the VOEvent packet as an XML file) ra and dec if applicable, at least one of start time, stop time, or peak time, event concept ("flare", "grb", etc) and name ("GRB 070616") if available, name and email of VOEvent author, params (comma-separated list of <What> param tag name, value, and unit if available: "rate=12.4 ct/s"), and refs (comma-separated list of <What> reference URL tag name and values: "photometry data=http://www.server.com/photometry.fits").

For example, create this MySQL table:

mysql> create table ogle (ivorn varchar(200), url varchar(200), ra float, dc float, start datetime, stop datetime, instant datetime, concept varchar(50), name varchar(50), contactname varchar(50), contactemail varchar(50), params varchar(500), refs varchar(500));

mysql> describe ogle;

+--------------+--------------+------+-----+---------+-------+

Parse packets from a VOEvent feed

Skip this step if you do not wish to harvest VOEvents from live feeds into your database. Note that you need to edit the file to put in your database connection details.

# USER CONFIGURATION - CUSTOMIZE THE FOLLOWING 4 VALUES my $dbuser="stapuser"; my $dbpasswd="stappwd"; my $dbhost="localhost"; my $db="stap";

The event_clientDB.pl code attached can be executed as a background process to parse GCN, OGLE, or SDSS events from the eSTAR VOEvent broker at vo.astro.ex.ac.uk. The machine hosting this client must have outbound access to port 8099. Added security: this access can be optionally restricted to the eSTAR broker's IP address (144.173.229.22). This restriction is recommended since the event client has write access to your database. Customize this client to search for other event feeds by adding further elsif statements to the code beginning at line 121, "# Parse ivorn and select $table based on ivorn". Make sure that you have created a database table to hold the new events. Example client running as a background process, connected to eSTAR broker:

% perl event_clientDB.pl -host estar6.astro.ex.ac.uk -port 8099

Manually inject some packets

As you can see from http://estar6.astro.ex.ac.uk/ogle/cgi-bin/timeline.cgi OGLE events only occur every few days, so we will be lucky to harvest even one event into our database during the tutorial. You can use the script parseVOEventDB.pl to insert events directly into the database.

Deploying a STAP web service on top of an event database

Instructions:

1. Pre-requisites: Tomcat (5.5+), java (1.4+), MySQL (any recent version), a jar file containing the appropriate MySQL JDBC driver in $CATALINA_HOME/common/lib.
2. Create a MySQL database to hold one or more event tables.
3. Create a read-only MySQL user with select access to all tables that should be visible to your STAP web service(s).
   • you can skip this step and use the root user for this tutorial - for a properly deployed service a user with read-only privileges should be created for extra security.
4. Store event parameters in a table with one row per event.
   • At a minimum, include at least one column for time information (start time, end time, or peak time)
   • If VOEvent packets are available for individual events, include a column for URLs pointing to each event's packet.
5. Download the AstroGrid STAP service astrogrid-stap.war file. Rename the .war file to stap.war and copy it into $CATALINA_HOME/webapps
6. Configure the web service's $CATALINA_HOME/webapps/stap/WEB-INF/classes/astrogrid.properties file.
   • Don't forget to specify the name of the event database, host machine and port, read-only username and password, access URL, and full SQL statement.
     • "accessURL" can point to a web page describing the events if VOEvent packets are not available for each event.
     • Two SQL statements can be configured: full.sql.syntax (for time-only queries) and full.sql.syntax.pos (for time plus cone search queries). Both statements may query over one or more tables.
     • Note: the current implementation of VOExplorer expects VOEvent param elements to be returned as a comma-separated string of key / value pairs (e.g., "flux=1.5 keV" or "xray class=A"). The same is true of reference URLs (e.g., "light curve="http://www.lightcurvesrus.com/path/to/lightcurve.fits"). This can be accomplished with one column each for comma-separated params and references, or by creating a comma-separated string from individual param (or reference) columns. * In addition to the 7 mandatory parameters specified in the config file, an unlimited number of optional parameters may be returned to the user. These parameters and corresponding db columns can be specified as parameters results.more.0, results.more.1. (Note that these parameter names should start from 0, not 1.)
7. Configure the registry template $CATALINA_HOME/webapps/stap/WEB-INF/classes/regmetadata.properties file. Next, view the registry entry at http://localhost:8080/stap/MetaDataXML0_10.jsp. Use this entry to register the service with an IVOA compatible registry.
   • Be sure to include one <subject> line with value "voevent". Multiple subject lines can be used to increase filtering capability within VOExplorer.
8. Restart the web application (or restart Tomcat - whichever is less hassle).
9. Test the STAP deployment - perform a time query inside the Workbench application's HelioScope tool or VOExplorer's AstroScope tool and see if your STAP service is returned.

Querying the Events with VODesktop

• screenshot of VODesktop's AstroScope showing VOEvent services:
  

Future work

AstroGrid's VOEvent STAP services can be queried through HelioScope and the new VOExplorer tool. HelioScope returns event timestamps with URLs pointing to VOEvent XML packets. VOExplorer provides additional functionality: users can filter VOEvent STAP services by event type, query events by time or a combination of time and cone search, view matching event parameters in a tooltip, and open or save event reference files such as light curves, images, movies, and tables.

Simple Event Access Protocol (SEAP) will replace STAP as the query mechanism for VOEvents. A SEAP working draft is in progress by Matthew Graham, Elizabeth Auden, and Phil Warner at http://wiki.astrogrid.org/bin/view/Astrogrid/SimpleEventAccessProtocol - all comments, suggestions, and proposed use cases are welcome.

In addition, a proposal for VOTimeSeries is underway by several IVOA astronomers.This schema will allow providers of transient data to encapsulate time series data in a simple XML format.

Software

• stap.war The stap server to be installed under Tomcat.
• Other scripts - voevent-scripts.zip
  • VOEvent client and broker pair for local testing: see attached event_client.pl and event_test.pl from eSTAR
  • VOEvent parsing library customized to extract metadata required for STAP service: see enclosed VOEventSTAP.pm.
  • VOEvent client customized to parse VOEvent packet, upload metadata to db for STAP service, and (optionally) write VOEvent XML file to local web directory: see attached voevent_clientDB.pl
• This code uses the eSTAR Sourceforge Astro::VO::VOEvent module code, see the VOEvent sourceforge project to download perl modules.