Astrogrid DSA (Catalogues)

Dave Morris & Paul Harrison

Abstract

The aim of this hands-on session is to show how to publish tabular data, stored in an RDBMS system, using the AstroGrid DSA/Catalog component.

Sample data will be made available, but participants are also encouraged to try publishing some of their own data. This may be installed on the participant's laptop, or, firewall issues permitting and subject to appropriate JDBC access, on a remote RDBMS. See the Software and data preparation requirements page for more information.

For the active installation phase, participants may choose to participate in a "step-by-step" assisted-installation group, using pre-prepared sample data. Alternatively, they may choose to work independently at their own pace, using the supplied documentation (with support from AstroGrid advisors as needed).

External References

• Astrogrid Home
• Astrogrid Software Development Home

Tutorial Session Outline

Phase 1

1. Introduction to the DSA/Catalog component - This section will present a high-level overview of the DSA/Catalog component, exploring its motivations, strengths and weaknesses, and its larger context within the VO.

1. Demonstration - This section will provide a demonstration of the administration and public interfaces of a working DSA/Catalog installation.

Phase 2

Active installation will commence, using the DSA/Catalog's online and embedded documentation, and advisor support where required.

---

Installing the workshop DSA

The following instructions assume that you have installed the Workshop software requirements.

The following workshop instructions will use the term CATALINA_HOME to refer to the home directory for the demo Tomcat server. Whenever CATALINA_HOME appears in the instructions, you will need to substitute the home directory for the demo Tomcat server.

For example, if you installed tomcat into C:\AIDADemo\ on your laptop, then the Tomcat home directory will be C:\AIDADemo\tomcat\. If the instructions refer to CATALINA_HOME/webapps/dsa/, you will need to look in C:\AIDADemo\tomcat\webapps\dsa\ on your laptop.

Download the DSA war file

Download the latest version of the DSA war file, astrogrid-dsa-catalog-2008.2.war from the workshop server. Future releases of this component will be available from the AstroGrid maven repository http://www.astrogrid.org/maven2/org/astrogrid/astrogrid-dsa-catalog/

Note - once you have downloaded the file rename it to dsa.war this will save typing later and ensure that the pre-prepared scripts will work correctly.

Install the war file

Copy the war file into your Tomcat webapps directory CATALINA_HOME/webapps/.

    CATALINA_HOME
        |
        +-- bin
        |
        +- ....
        |
        \-- webapps
               |
               +- ....
               |
               \-- dsa.war

If Tomcat is configured to auto-detect new webapps (on by default), then it will automatically unpack the war file and install the webapp. If not, you may need to restart Tomcat to get it to recognise the new webapp.

This should have installed the basic DSA webapp as dsa.

You can check this works by pointing your browser at http://localhost:8080/dsa/.

Configuring the admin account

The dsa webapp uses a combination of settings in the context file and the default properties shipped in the WAR file to configure the WebService .

You can check where the webapp is getting the configuration settings from by checking the webapp fingerprint page. However, before you can access the fingerprint page, you will need to configure the DSA admin account.

The DSA webapp uses a Tomcat role and user account to control access to the DSA administration pages.

Edit the tomcat-users.xml file in the CATALINA_HOME/conf directory, and add the "dsaadmin" role and assign that role to your admin user:

    <?xml version='1.0' encoding='utf-8'?>
    <tomcat-users>
  <role rolename="manager"/>
  <role rolename="admin"/>
  <role rolename="dsaadmin"/>
  <user name="admin" password="admin" roles="admin,manager,dsaadmin" />
</tomcat-users>

 

Tomcat will not automatically load the new user and role information, so you will need to stop and start Tomcat to make it recognise the new user account.

Once you have restarted Tomcat, you should be able to login and access the fingerprint page.

The fingerprint page shows which configuration properties are being read from the JNDI values configured in the context file, and which properties are being read from the default.properties file shipped in the WAR file.

At this stage the DSA is configured to serve a small synthetically generated dataset - we need to configure in our real test dataset. The DSA is intended to be self documenting and has the full instructions for configuration at http://localhost:8080/dsa/configure.jsp - However, the major steps in this process will be outlined below, and in particular the registration procedure has to be altered because of the network configuration within the classroom environment.

Installing the JDBC driver

The pre-installed AIDA workshop database server is running MySQL ,so we need to use the appropriate JDBC 3 database drivers.

You can download the mysql-connector-java-5.1.7-bin.jar JAR file attached to this page.

Download the JAR file, and save it in the $CATALINA_HOME/common/lib directory.

    CATALINA_HOME
        |
        +-- bin
        |
        +- ....
        |
        \-- common
              |
              +- classes
              |
              +- endorsed
              |
              \-- lib
                   |
                   +-- ....
                   |
                   \-- mysql-connector-java-5.1.7-bin.jar

Creating a new context

(Step 4 http://localhost:8080/dsa/docs/configure_step4.jsp)

In order to be able easily to customize the DSA it is necessary to tell it where to find a local properties file. A suitable template set of properties is attached to this page sample.properties. Note that the properties file will need a small amount of customising for your own particular installation: search for the word "WORKSHOP" in the file to find the properties that will need to be changed. The file contains detailed documentation on each of the configuration settings, but again the instuctions on this page will point out the main steps.

Create an XML file called dsa.xml containing the following:

<?xml version='1.0' encoding='utf-8'?>
<Context displayName="Astrogrid DSA/catalog" docBase="dsa" path="/dsa">
<Environment description="Configuration filename" name="org.astrogrid.config.filename" override="false" type="java.lang.String" 

value="/Users/pharriso/Work/votech/AIDAWorkshop09/first.properties"/>

</Context>
you need edit the line starting "value=" to point to the location where you have saved the properties file.

Add this new context file to the Tomcat context files in CATALINA_HOME/conf/Catalina/localhost.

    CATALINA_HOME
        |
        +-- bin
        |
        +- ....
        |
        \-- conf
             |
             +- ....
             |
             \-- Catalina
                    |
                    \-- localhost
                           |
                           +-- admin.xml
                           |
                           +-- ....
                           |
                           \-- dsa.xml

If Tomcat is configured to auto-detect changes to context files (on by default), then it should automatically detect the new context file and deploy a new webapp. If not, you may need to restart Tomcat to get it to recognise the new context file.

Once the webapp has been reloaded, check the fingerprint page to see if the new properties have been recognised.

 Datacenter Configuration

  Configuration loaded from: JNDI, file:/Users/pharriso/Work/votech/AIDAWorkshop09/first.properties
        ....

Once Tomcat has reloaded the webapp, have another look at the home page for the dsa webapp. The first section of text in the welcome section should now contain something like this:

• DSA/catalog installation: Example DSA for FIRST Survey Catalogue

Generating Database Metadata

At this stage the DSA has been configured to connect to the test dataset (actually located in Cambridge). However, the necessary metadata has not been created to allow the DSA server to be published to a registry to allow other tools in the VO to discover and use it. The main task in actually publishing a data set is in assigning the correct metadata to the table columns. The DSA assists in this task by generating a template from reading the database structure as described at http://localhost:8080/dsa/docs/configure_step5.jsp. You should generate this file and examine it to see what would need to be added, but to save time a completed database metadata document is attached.

You should now complete the customizing of your properties file, which consists of

1. configuring the URL at which your service is installed.
2. configuring the location of the database metadata file.
3. assigning a name that will be used in the IVOA identifier of the registered service.

Registering the DSA

The next stage of the configuration involves registering the service with the workshop registry. The registry provides a resource discovery service for the VO services, datasets and tools, and once you have registered the service it becomes "public".

There is a registry service for the workshop, available at http://manaslu.star.le.ac.uk/astrogrid-registry//. This registry is actually private to the workshop, so any mistakes that you might make will not be propagated into the wider VO.

Because of the network topology of the workshop (the fact that outside services cannot talk directly to your laptop) the registration procedure has to be changed from that described on http://localhost:8080/dsa/docs/configure_step7.jsp, where the registry would query the service for metadata. Instead we are going to gather all the registration metadata into a single file and upload that to the registry.

You need to download the ant project file build.xml and run it with a command

ant -Did=identifier

where you should replace the string "identifier" with the value of datacenter.resourceKey property that you set. This will produce a file register.xml, which you should upload to the registry here http://manaslu.star.le.ac.uk/astrogrid-registry/admin/editEntry.jsp

You will need a password to access the registry - this will be given to you in the session. On accessing the page it should look as below, and you should choose the "upload from a local file option"

Client Software

You can find Aladin and the VODesktop here http://esavo.esac.esa.int:8080/AIDAWS09/DVD/Software/Common/