CEA one to one

This page contains some information for the background to the CEA (Common Execution Architecture) one to one sessions

• IVOA Note describing the original CEA concept, which was to provide a standard interface for data processing services which needed to be asynchronous. AstroGrid implemented an early version of the interfaces and services described in many of its services and the AstroGrid VODesktop acts as a client to call these services. The most useful version of the CEA server that exists is the "commandline" CEA server that can be used to "publish" a legacy unix commandline processing application as a VO compatible web service.
• This idea has been adopted by the IVOA now and has become known as UWS (Universal Worker Service) - This note explains the relationship between the two terms. It seems that the most popular binding of the UWS service will be a REST based binding rather than the SOAP based binding that was originally proposed in the CEA note.
• The current progress within the IVOA Grid and web services working group is tracked by this page
• CEA source code is available (via CVS) at http://www.astrogrid.org/viewcvs/astrogrid/applications/
• In future more detailed developer information will be available at http://deploy.astrogrid.org/

Installing a beta CEA server

To install the server you need to have a J2EE container such as Tomcat

need to create a directory structure for the configuration and run time temporary files

$CEAROOT/config
$CEAROOT/temp
$CEAROOT/records

note $CEAROOT should not be within the $CATALINA_HOME tree.

inside the config directory you must create two files

• app-description.xml
• registration-template.xml

The first of these files defines the application and its parameters, and the second describes how the service should be described in the registry. Examples of each file are attached.

JNDI properties need to be set for the newly installed application server.

• cea.base.dir should be set to the full path to $CEAROOT above
• org.astrogrid.registry.query.endpoint should be set to the endpoint of a registry - the best one to use is the standard astrogrid registry http://registry.astrogrid.org/astrogrid-registry/services/RegistryQuery
• cea.webapp.url should be set to the service root as seen by the outside world.

For tomcat this can be done by editing a file at $CATALINA_HOME/conf/Catalina/localhost/cea.xml (as long as your web application was deployed under the /cea context)

   <Context path="/cea">
    <Environment name="cea.base.dir" override="false" type="java.lang.String" value="/Users/pharriso/Work/ag/src/applications/commandline/ceabasesafe" />
    <!-- registry endpoints - these are essential for the proper working of CEA -
        it needs to be able to query the registry to do myspace operations and can update the registry with its own configuration
        t-->
    <Environment name="org.astrogrid.registry.query.endpoint" description="registry query service endpoint URL" type="java.lang.String"
        value="http://registry.astrogrid.org/astrogrid-registry/services/RegistryQuery" />
    <Environment name="org.astrogrid.registry.query.altendpoint" description="fallback registry query service endpoint URL" type="java.lang.String"
        value="http://msslxv.mssl.ucl.ac.uk:8080/mssl-registry/services/RegistryQuery" />
    <!-- service endpoint as seen by the outside world - here because of possible NAT firewall issues-->
    <Environment name="cea.webapp.url" type="java.lang.String"
        override="false"
        value="http://localhost:8080/cea/"
        description="service end point as seen by outside world"/>
</Context>
   

Advanced configuration

Warning only do this if you are sure what you are doing.

you can further configure the CEA server by editing the WEB-INF/cec-spring.xml file - you might want to do this to alter the default cpu and storage limits for the server.

Notes about the application description

The Commmand line Execution Controller is a component that can be used to wrap legacy command line applications in a specialized CEA web service so that it can become part of the VO.

Command Line Application prerequisites

For the CEA Command Line Execution Controller to be able to run an application, it must fit into the model of a command line application that the CEA server understands. Amongst other characteristics, it must be possible to run the application to completion with arguments supplied on the command line.

Characteristics of the application that cannot be supported include

• requiring user input from a GUI
• requiring a GUI to display results
• requiring user input of any kind after execution had been started

The command line application parameters should be specified by one of the "standard" unix methods

• by position
• by a switch of the form -param value
• by a switch of the form param=value

It is possible that an application that does not quite meet these requirements in a small fashion could be accommodated by writing an adapter class that extends the standard org.astrogrid.applications.commandline.CommandLineApplication model.

The CEA Application model is also described in an IVOA document (in preparation)

Brief description of elements

<commandLineExecutionControllerConfig> that contains <CmdLineApplication> elements (you will have one for each of your scripts) - In addition to the standard registry metadata elements each of the <CmdLineApplication>s has a < applicationDefinition> with a set of <parameters> and one or more <interfaces>. The UML model for an application is shown in the diagram below

For this type of CEC we need to define a series of <parameterDefinition xsi:type="impl:CommandLineParameterDefinition"> elements corresponding to each of the command line parameters that there are. For command line applications parameters are typically specified in one of 3 ways under unix and the CEC can cope with any of these (theoretically in any combination, but I have not exhaustively tested this, and I doubt that I have covered everything)

Each <parameterDefinition xsi:type="impl:CommandLineParameterDefinition"> has a set of attributes that specify which of the above categories the parameter falls under, as well as some subelements that further specify the parameter.

id

this is the name of the parameter - the default behaviour is that this name is the same as the string that is used to denote the parameter on the command line (if there is one) - however you can give the parameter any name you like (as long as it is unique for a particular application as you can specify the name that is used on the command line using the commandSwitch attribute. Alternatively you can specify the position of the parameter with the commandPosition attribute if there is no switch.

type

this specifies what type the parameter. This value should be set to be as close as possible to what the application expects the type of the parameter to be to allow for possible type checking. Note that amongst the values there are several types that can effectively be used as "catchall" types, which can be used if the application does not have any strong typing. You should look in the schema for the up to date list, but a description of the meaning of the types is given below.

fileRef

This specifies whether the application expects the parameter to refer to a file or simply a value that is passed on the command line

commandSwitch

This is the text of the command switch (without any hyphen or equals characters). If this is set to the special value

"---"

then it means that the parameter will not be added to the command line at all - this is useful sometimes in allowing a value to be passed to the java wrapper class.

commandPosition

This is specified as an integer index position that the parameter will be specified on the command line - if a value is specified for this attribute, then the parameter will be presented "bare" with no switch

switchType

this can be 'normal', in which case the switch will written in the -X val style or 'keyword' in which case the x=val style will be used.

OptionList

this can be used to specify a parameter that takes its value from a fixed set. Each of the possible values is specified by an OptionVal element

parameter types

complex

if a complex value is to be passed

real

a floating point number should be passed

boolean

a boolean value should be passed

anyURI

a URI should be passed

anyXML

this should probably not be here - too loose...do not use if possible

VOTable

A VOTable is passed

angle

A value to be interpreted as an angle - whether it is RA or dec is determined by UCD

ADQL

Standard Astronomical Data Query Language is passed

FITS

A FITS file is passed

table

a table passed

image

an image is passed

text

A general piece of alphanumeric data is passed - should generally be what unix considers to be a text file

binary

An arbitrary piece of binary data is passed

The elements that can occur within the <parameterDefinition xsi:type="impl:CommandLineParameterDefinition">

<name>

the name that will be used for the parameter by the client to display to the user

<description/>

a description for the parameter by the client to display to the user

<ucd>

the ucd for the parameter if it has one

<defaultValue>

can be used to display a default value

<unit>

can be used to specify the units.

<UType>

can be used to specify the UType

<mimeTyoe>

can be used to specify the mime type

An <Interface> is simply a collection of parameters - you might have a simple interface that consists of only one of the possible parameters and a complex one that consists of all the possible parameters. Each interface consists of a number of <input> and <output> parameters (designated by the pref element that points to the <parameterDefinition> above) - This classification is particularly important in the case of parameters that specify VOSpace/!MySpace file references, as this is how CEC knows whether to fetch the values of put them.

Modifying default application behaviour

It might be that it is necessary to modify the output of the commandline application in some fashion to make the application more "VO friendly", this can be done in two general ways

1. Wrapping the application in a shell script and specifying the script rather than the application directly in the ExecutionPath element. A description of this method of customization is outside the scope of this documentation, but it is a valuable, and often quick-to-implement way to slightly alter the behaviour of an appliction.
2. Writing a custom adapter class - it is possible to write a specific class to alter the way that an application is run and to specify the class name as the instanceClass attribute on an Application element.

Note about Schema

The schema that are used to describe the CEA server configuration and the CEA server registration are in a state of flux at the time of writing, so that they do not match those in the released version of the registry for instance. The changes are relatively minor, but result in the schema having different formal namespaces and so any instance documents will be invalid simply because of this (even if otherwise structually valid)

It is possible to see the complete set of schema that are used in this version of the CEA server by extacting the WEB-INF/lib/astrogrid-contracts.jar from the cea.war file and then extracting the schema/ directory (and subdirectories) from the astrogrid-contracts.jar. You can use these schema to validate the app-description.xml file if the CEA server seems to be having trouble reading it.

Running CEA applications from the VODesktop

It is possible to run an application directly with the AstroGrid VODesktop GUI and is possible to use the python scripting component that communicates with the VO via the VODesktop.

If you make sure that you have Aladin running then this script will run sextractor remotely as a CEA application and display the simple image and the extracted catalogue in Aladin using the PLASTIC inter tool desktop communication protocol.

CEA Security

The CEA server has the ability to authenticate the caller with X509 and proxy certificates. The current trusted root certificates are listed in http://technology.astrogrid.org/raw-attachment/wiki/vodesktopResources/trust-anchor-list.xml

-- PaulHarrison - 24 Jun 2008