ISSGC'06 :: GT4 Advanced Practical

Title:	GT4 Advanced Practical
Subtitle:	Search For Knowledge
Tutor:	Lisa Childers
Authors:	GT4 Team

Advanced Practical

This tutorial involves working with GT4 services that are used as components to solve common problem in distributed systems programming. The problem involves searching through a large data set looking for patterns of interest. The tutorial will walk you through the various components and show case some of the interactions.

Various services have been written to provide the functionality and this tutorial will demonstrate how clients can be written to use these services. Details about the service itself, functionality and design are descibed in GT4 Practical Services Details. You don't need to know the design of services to complete the exercise, but is provided as additional reading.

Tutorial Services

As described in the overview, here are the services provided for use in this exercise:

The Sequence service generate sequences of numbers within a given range.
The Cross Product (XProduct) service takes two arrays of numbers [x1...xn] and [y1...yn] and generates all possible (x_i,y_j) pairs of points from those numbers.
The FileFactoryService and File Service can be used to store data files with query points (for input to the surface service) and data files with the surface data (as output from the surface service).

Your job!

Your job is to complete work on the client code. The instructors are running a webservices container which hosts the above services. You have been given three Java files: FileServiceUtil.java, GeneratorServicesUtil.java and GTAdvancedTutorial.java. The GTAdvancedTutorial file contains code to interact with all of the above service. Unfortunately it relies on code in the *Util.java files that is missing! Your job is to complete the *Util.java files so the GTAdvancedTutorial code runs successfully.

All the files are in ws-clients/src/org/globus/tutorial/client/ directory. The points where code need to be filled up are clearly marked with ADD #, where # is the comment number.

1. Installation Steps

You can reuse installation of the Java WS Component of the Globus Toolkit from the introductory tutorial.

Set GLOBUS_LOCATION to point to that directory.

 $ export GLOBUS_LOCATION=/gt4/$USER/intro/globus-4.0.2-java-norft

Create directory to untar service code.
```
 $ mkdir -p /tmp/$USER/gt4-services 
```

Download GT4 Advanced Tutorial Servicesinto /tmp/$USER/gt4-services

$ cd /tmp/$USER/gt4-services
$ wget http://www.gs.unina.it/~gt4/advanced/gt4-services.tar.gz

In /tmp/$USER/gt4-services,

$ tar xvzf gt4-services.tar.gz

Deploy Services from the gt4-services directory:
```
$ ant deploy 
```
Create a directory to edit client code.
```
$ mkdir /tmp/$USER/gt4-ap-client
```
Download GT4 Advanced Tutorial Client Template into /tmp/$USER/gt4-ap-client

$ cd /tmp/$USER/gt4-ap-client
$ wget http://www.gs.unina.it/~gt4/advanced/gt4-ap-practical.tar.gz

Untar the client download.

$ tar xvzf gt4-ap-practical.tar.gz

The code templates for the rest of the exercises are in ws-clients/src/org/globus/tutorial/client. The next section will teach you how to fill the code.
To compile the client code,
```
$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy
```
Note: The code should compile after each exercise!
When you are done writing code, refer to Test Advanced Client to test your code.

2. FileServiceUtil.java

Under gt4-ap-clients/ws-clients, you will find a file called /src/org/globus/tutorial/client/GTAdvancedTutorial.java. This is the fully written Advanced Practical Client. It relies on functions in two utility files, called FileServiceUtil.java and GeneratorServicesUtil.java. If you look inside GT4AdvancedTutorial.java, you can find the calls to the utility functions. Here's the first call in the method exercise1 in GTAdvancedTutorial:

// (1) Create a file to store 1D data of size 10 bytes and
// lifetime of 100. Use createFile method in FileServiceUtil 
EndpointReference epr1 = 
    FileServiceUtil.createFile(10, RequestFileType.value1, 100,
			       hostPort, persist);

So let's look at that method inside src/org/globus/tutorial/client/FileServiceUtil.java.

2.1 The FileServiceUtil::createFile method

Open FileServiceUtil.java and go to the createFile method. You will see that there are four comments marked ADD. These are the points where you need to write code to complete the function. The function is supposed to interact with the FileStoreFactoryService and create a new file resource. Here are the pieces you need to add:

Constructing service URLs

First we need an endpoint for the FileStoreFactoryService as protocol://host:port/servicePath

You should use the hostPort and the private static final string constants to construct the URLs to the service. When you have a string containing the url, create an URL object with:
```
   URL myUrl = new URL("string that points to the File Store Factory Service");
```
Your code should go in the section after comment (ADD 1).
Creating input messages

Okay, now we have an address to talk to, but we need to know what to send it. The key to this puzzle is in the service's WSDL file. We are going to be accessing the createFile method, which is described by the following WSDL from /tmp/$USER/gt4-services/ws-fileStore/schema/build/schema/filestore/filestore_factory_port_type.wsdl:
```
<operation name="CreateFile">
<input name="createFileRequest" message="tns:request"/>
<output name="createFileResponse" message="tns:response"/>
<fault name="FileStoreException" message="tns:FileStoreException"/>
</operation>
```
The tns:request is defined as:
```
      <element name="request">
<complexType>
  <sequence>
    <element name="fileSizeInBytes" type="long"/>
    <element name="lifetimeInMins" type="int"/>

    <element name="fileType">
    <simpleType>
	<restriction base="string">
	  <enumeration value="1D"/>
	  <enumeration value="2D"/>
	  <enumeration value="3D"/>
	</restriction>
     </simpleType>
    </element>
  </sequence>
</complexType>
</element>
```
Whew, that looks a little complicated. How are we going to render this XML type definition in java? Fortunately, there is a WSDL to Java converter that has already done it for us! You can find the stubs in: /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/types/filestore/Request.java
Also, you can find javadocs for that stub at GT4 Practical API.That will give you that java class name of the Request object, as well as the constructor methods. You will want to use the constructor that sets numBytes, fileType, and lifeTime. Put that constructor after the comment marked (ADD 2)

You might have to import relevant classes to get this code to compile.
Port Types and Locators

Now we have a URL for the service, and a message to send to it. We are going to use an AddressingLocator (section 5.2.1.1.2) to send our message. There is a different AddressingLocator class for each of our services. Because we are going to talk to the FilestoreFactoryService, we will use a FilestoreFactoryServiceAddressingLocator. This is another automatically generated class which you can find at: /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/filestore/service/FilestoreFactoryServiceAddressingLocator.java
Also, you can find javadocs for that stub at GT4 Practical API

First you need to create the object, then use the getFileStoreFactoryPort method, passing it the URL you wish to contact. You will then have a FileStoreFactory object you can use to invoke operations on the remote service. Store that object in the predefined variable named "factory". Place this code after comment number (ADD 3).
Invoking a method

Now invoking methods on the remote service are as easy as invoking the class operations on the FileStoreFactory object. We want to call createFile using the request object we made in step (2). Place the invocation after comment (ADD 4) and return the EPR you got as the result of the function in the predefined variable "ref".

Compile the code

Compile the code by running:

$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy

2.2 The FileServiceUtil::storeContents method

Now we can continue reading through the exercise1 method in GTAdvancedPractical.java. The next method it uses from our FileServiceUtil class is this:

        FileServiceUtil.storeContents(epr1, oneDData);

Go back to FileServiceUtil.java and find that method, and fill it out according to these instructions:

Constructing Service URLs

In this example, we are going to be given an EndpointReference (EPR) which refers to an existing resource. In this case, we do not need to construct a service URL - that information is already contained inside the EPR.
Port Types and Locators

In this function we will be communicating with the FileService, not the factory. Therefore we'll be using the FileServiceAddressingLocator. You can find the stubs for that class at /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/filestore/service/FileServiceAddressingLocator.java and the javadocs at GT4 Practical API.
You should see that the getFileServicePort method takes an EPR as input, so we can pass our EndpointReference to a new FileServiceAddressingLocator() and get back a FileService object. This code goes in the (ADD 1) comment.
Invoking a method

Now inspect the store method in /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/filestore/FileService.java. You can see that it takes Data as input, so we do not need to construct a new Data object - it has already been passed to our function as a parameter. Therefore you can use the FileService object to invoke the store function using our input Data object as a parameter. This code goes in the (ADD 2) comment.

Compile the code

Compile the code by running:

$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy

2.3 The FileServiceUtil::fetchContents method

The GTAdvancedPractical.java calls the createFile and storeContents several more times on different kinds of array data. The next new function we need to complete is the fetchContents method, used here:

        Data output1 = FileServiceUtil.fetchContents(epr1);

Constructing service URLs

Again, we will be using an EPR, so we don't need to create a separate service URL.
Port Types and Locators

The fetch method is defined on the FileService, so we will use a FileServiceAddressingLocator. Create a new FileServiceAddressingLocator, then use it to create a FileService object, exactly like you did in the last exercise. Put this code under (ADD 1).
Creating input messages

The input message for the fetch operation is VoidType, as you can see from the stubs generated at /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/filestore/FileService.java:
```
public org.globus.tutorial.types.filestore.Data fetch(org.globus.tutorial.ty
pes.filestore.VoidType parameters) throws java.rmi.RemoteException, org.globus.t
utorial.types.filestore.FileStoreException;
```
You might have to import relevant classes to get this code to compile.

To see what goes into the VoidType, we can look at /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/types/filestore/VoidType.java:
```
    public VoidType() {
    }
```
Okay, that's easy enough. This is an example of a method that doesn't really need any input parameters, but we still need an object to send to the method. You can see what it looks like in the WSDL at /tmp/$USER/gt4-services/ws-fileStore/schema/filestore/filestore_types.xsd:
```
      <element name="voidType">
        <complexType/>
      </element>
```
This is a common pattern for methods that don't require input data. Create a new VoidType variable under (ADD 2).
Invoking a method
Invoke the fetch method on the FileService object you created, with the empty VoidType message as an argument. This returns the Data type we wish to return. Store the Data object in the predefined "returnedData" variable. Put this code under (ADD 2) also.

Compile the code

Compile the code by running:

$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy

2.4 The FileServiceUtil::concatenate method

The next function used in the exercise1 method is the concatenate method:

        FileServiceUtil.concatenate(concatFile, 
                                    new EndpointReference[] { epr3, epr4 }, 
                                    hostPort, persist);

Constructing service URLs
For the concatenate method, we will communicate with FileStoreService and reference two EPRs. Construct a URL for the FileStoreService from the hostPort like you did in the first method.
Creating input messages

Find the definition of the concatenate method in /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/filestore/FileService.java:
```
    public org.globus.tutorial.types.filestore.VoidType concatenate(org.globus.t
utorial.types.filestore.ConcatInput parameters) throws java.rmi.RemoteException,
 org.globus.tutorial.types.filestore.FileStoreException;
```
Our input message should be a ConcatInput, which is defined in /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/types/filestore/ConcatInput.java:
```
    public ConcatInput(
           org.globus.tutorial.types.filestore.ArrayOfEPR inputFiles,
           org.apache.axis.message.addressing.EndpointReferenceType outputFile) 
{
           this.inputFiles = inputFiles;
           this.outputFile = outputFile;
    }
```
In turn, an ArrayOfEPR is defined in /tmp/$USER/gt4-services/ws-fileStore/schema/build/stubs/src/org/globus/tutorial/types/filestore/ArrayOfEPR.java
```
    public ArrayOfEPR(
           org.apache.axis.message.addressing.EndpointReferenceType[] value) {
           this.value = value;
    }
```
You might have to import relevant classes to get this code to compile.

Ok! We have to do two things before our input message will be ready. We can create an ArrayOfEPR from our EndpointReference[] input. Then we can create a ConcatInput object using the ArrayOfEPR and the resultFile EPR that was passed to our function. Put that code under (ADD 2).
Port Types and Locators
Create a FileServiceAddressingLocator, and get a new FileService object, much like you did in the first exercise.
Invoking a method
Using the FileService port type from the last step, invoke the concatenate method with the input message you created.

Compile the code

Compile the code by running:

$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy

2.5 The FileServiceUtil::deleteFile method

The last thing the exercise1 method does is cleanup all the files it created while it ran using the deleteFile method:

        FileServiceUtil.deleteFile(epr1);

Constructing service URLs

We do not need to construct an URL for this function because we are being provided an EPR.
Creating input messages

The destroy operation takes no input parameters, so we can create an empty org.oasis.wsrf.lifetime.Destroy() object to send as the input.
Port Types and Locators

The destruction operation is not defined in the FileStoreService WSDL. Instead, it is inherited from the WS-ResourceLifetime WSDL, which is part of the WSRF specification:
```
./ws-fileStore/schema/build/schema/wsrf/lifetime/WS-ResourceLifetime.wsdl
      
```
We are going to invoke the destroy operation defined in the ImmediateResourceTermination port type. To do this, we will create a WSResourceLifetimeServiceAddressingLocator. Then we can construct an ImmediateResourceTermination port type object by using the EPR constructor instead of the URL constructor.
Invoking a method
Use the destroy() method of the ImmediateResourceTermination port type with the empty Destroy() input message. Because your ImmediateResourceTermination was created using the EPR of the file resource we wanted to destroy, the correct file resource will be destroyed.

Compile the code

Compile the code by running:

$ cd /tmp/$USER/gt4-ap-clients/ws-client 
$ ant deploy

3. The GeneratorServicesUtil::* methods

Using what you learned from the FileServiceUtil code, complete the GeneratorServicesUtil methods by looking at the WSDL and stubs generated by the Generator Service.

4. Testing the advanced practical client

Set environment variable:

$ export GLOBUS_LOCATION=/gt4/$USER/intro/globus-4.0.2-java-norft 
$ cd $GLOBUS_LOCATION

Create a proxy credential:
```
bin/grid-proxy-init
```
The service to contact using the advanced client is running at http://globus.gs.unina.it:8080. To contact the service and use the PersistentFileStoreFactory use,
```
bin/gt4-advanced-practical globus.gs.unina.it:8080 true
```
To run the command, contacting the file store service that doesn't persist:
```
bin/gt4-advanced-practical globus.gs.unina.it:8080 false
```

Once the client has been completed, the output should look like:

$ bin/gt4-advanced-practical globus.gs.unina.it:8080 false


Exercise 1


Data from 1D file
Data has 1 sets of data
         Set 0 size: 2
                0.01    0.04

Data from 2D file
Data has 2 sets of data
         Set 0 size: 2
                0.03    0.04
         Set 1 size: 2
                0.23    0.24

Data from 3D file
Data has 3 sets of data
         Set 0 size: 2
                0.03    0.04
         Set 1 size: 2
                0.23    0.24
         Set 2 size: 2
                0.223   0.224

Data from second 3D file
Data has 3 sets of data
         Set 0 size: 2
                0.031   0.041
         Set 1 size: 2
                0.231   0.241
         Set 2 size: 2
                0.2231  0.2241

Data from concatenated file
Data has 3 sets of data
         Set 0 size: 4
                0.03    0.04    0.031   0.041
         Set 1 size: 4
                0.23    0.24    0.231   0.241
         Set 2 size: 4
                0.223   0.224   0.2231  0.2241

Accessing deleted file org.globus.wsrf.NoSuchResourceException


Exercise 2

Data from sequence generated for X
Data has 1 sets of data
         Set 0 size: 3
                1.0     3.0     5.0

Data from sequence generated for Y
Data has 1 sets of data
         Set 0 size: 2
                2.0     4.0

Data from cross product
Data has 2 sets of data
         Set 0 size: 6
                1.0     1.0     3.0     3.0     5.0     5.0
         Set 1 size: 6
                2.0     4.0     2.0     4.0     2.0     4.0

Advanced Practical

Tutorial Services

Your job!

1. Installation Steps

2. FileServiceUtil.java

2.1 The FileServiceUtil::createFile method

Constructing service URLs

Creating input messages

Port Types and Locators

Invoking a method

Compile the code

2.2 The FileServiceUtil::storeContents method

Constructing Service URLs

Port Types and Locators

Invoking a method

Compile the code

2.3 The FileServiceUtil::fetchContents method

Constructing service URLs

Port Types and Locators

Creating input messages

Invoking a method

Compile the code

2.4 The FileServiceUtil::concatenate method

Constructing service URLs

Creating input messages

Port Types and Locators

Invoking a method

Compile the code

2.5 The FileServiceUtil::deleteFile method

Constructing service URLs

Creating input messages

Port Types and Locators

Invoking a method

Compile the code

3. The GeneratorServicesUtil::* methods

4. Testing the advanced practical client