SPECjAppServer2001 User's Guide Version 1.04 Last modified: Thursday, August 29, 2002

Table of Contents

Section 1 - Introduction

• 1.1 Terminology

• 1.2 Overview

Section 2 - Installing SPECjAppServer2001

• 2.1 Directory Structure

• 2.2 Java Package Structure

Section 3 - Building and Deploying SPECjAppServer2001

• 3.1 Build and Deployment Process

Section 4 - Running SPECjAppServer2001

• 4.1 The SPECjAppServer Driver

• 4.2 Configuring and Running the Driver

• 4.3 Running the Atomicity Tests

• 4.4 Tips and Troubleshooting

Section 5 - Results

• 5.1 Creating a Submission File

• 5.2 Generating a Report

• 5.3 Submitting the Results

Section 1 - Introduction

SPECjAppServer2001 is a J2EE™ benchmark to measure the scalability and performance of J2EE servers and containers. SPECjAppServer2001 strives to stress the middle-tier rather than the client tier or the database server tier.

This document is a practical guide for setting up and running a SPECjAppServer2001 benchmark. This user guide discusses some, but not all, of the rules and restrictions pertaining to SPECjAppServer2001. Before continuing with the benchmark, we strongly recommend you read the complete SPECjAppServer2001 Run and Reporting Rules contained in the kit. For an overview of the benchmark architecture, see the SPECjAppServer2001 Design Document also contained in the kit.

1.1 Terminology

BOPS is the SPECjAppServer2001 performance metric and denotes the average number of successful Business Operations Per Second completed during the Measurement Interval. BOPS is composed of the total number of business transactions completed in the Customer Domain, added to the total number of workorders completed in the Manufacturing Domain, normalized per second.

price/BOPS is the SPECjAppServer2001 price/performance metric and is the total cost of the SUT divided by the number of BOPS.

The Domain is a logical entity that describes a distinct business sphere of operations.  The four SPECjAppServer2001 domains are: Customer, Manufacturing, Supplier and Corporate.

The Driver refers to the client code that drives the benchmark, monitors requirements, and reports results.

SPECjAppServer2001 Kit refers to the complete kit provided for SPECjAppServer2001. This includes the SPECjAppServer2001 Reference Beans, the Driver, load programs, and documentation.

SPECjAppServer2001 Reference Beans refers to the implementation of the Enterprise Beans provided for the SPECjAppServer2001 workload.

The Supplier Emulator is a Java Servlet that can run inside any Java enabled web server, and emulates the process of sending and receiving orders to/from suppliers.

The SUT is the System Under Test and is comprised of all components that are being tested. This includes Application Servers/Containers, Database Servers, network connections, etc. The Driver and Supplier Emulator are not part of the SUT.

1.2 Overview

The SPECjAppServer2001 workload emulates a heavyweight manufacturing, supply chain management, and order/inventory system representative of one in use at a Fortune 500 company. This workload stresses the ability of EJB containers to handle the complexities of memory management, connection pooling, passivation/ activation, caching, etc. The SPECjAppServer2001 Design Document includes a complete description of the workload and the application environment in which it is run. This section of the user's guide describes software and hardware environment required to run the workload.

1.2.1 Hardware Environment

Although SPECjAppserver2001 can be run on a single machine for testing purposes, compliance with the SPECjAppServer2001 Run and Reporting Rules requires that the driver and supplier emulator be run on a machine outside the SUT. Therefore, a compliant hardware configuration must include a network and a minimum of two systems – one or more systems to run the components within the SUT and at least one system to run the driver and supplier emulator outside the SUT. A typical configuration is illustrated below.

1.2.2 Software Environment

SPECjAppServer2001 is a J2EE application that requires a J2EE 1.2 compatible application server and database server to run as part of the SUT. Outside the SUT, a Java Runtime Environment (JRE) version 1.3 or later is required for the driver and supplier emulator. A web server supporting HTTP 1.1 is also required for the supplier emulator

Section 2 - Installing SPECjAppServer2001

The SPECjAppServer2001 Kit is supplied as a tar file that should be extracted into a directory. We will call this top level directory $JAS_HOME.

SPECjAppServer2001 can be run on any Java platform. Most of the code is implemented in Java, however there are a few scripts used to start certain programs. These scripts are provided in .sh format for Unix platforms and in .bat format for Windows.

2.1 Directory Structure

ant : The ant installation directory.

bin : Driver and database load scripts. All additional scripts except specific database related scripts will be in this directory.

classes : This is a directory for compiling Java classes and packaging them. It comes with all the necessary class files necessary to run SPECjAppServer2001.

classes/META-INF : Temporary place for holding deployment descriptors just before packaging into JAR format files.

classes/WEB-INF : Temporary place for holding web deployment descriptors just before packaging the WAR files.

config : Environment configuration and properties files used by the Driver, make and load programs. Once SPECjAppServer2001 has been successfully deployed, this is the only place where the configuration should be modified.

config/security : Security policy files used for the benchmark programs. These should not be modified.

docs : The documentation for all SPECjAppServer2001 beans generated by running 'make doc'.

jars : The JAR files created by the make files. The files distributed in this directory are driver.jar, launcher.jar, charts.jar , xerces.jar. The application server specific jars for deployment need to be generated by the user.

schema : Schema information and static data to be loaded into the database.

schema/sql : Generic SQL scripts for creating the tables, etc. in the database. These should be copied and modified as necessary for a particular database product.

src : Root directory of the source files. This contains generic and application server specific make files.

src/com : Entry into the com.sun.jini.debug and com.sun.jini.thread package source.

src/deploy : The deployment descriptors.

src/docroot : HTML and JSP pages used for the web interface.

src/org : Entry into the org.spec.jappserver package source.

src/output : Default directory in which output of runs go.

Note: For some of the popular DBMS products, DBMS specific files are in schema/<DBMS vendor> and DBMS specific SQL scripts are in schema/<DBMS vendor>/sql.

2.2 Java Package Structure

All SPECjAppServer2001 Java classes are located in the org.spec.jappserver package. The following lists the sub-packages of org.spec.jappserver:

• corp
• orders
• mfg
• supplier
• common
• ruleengine
• webbeans
• driver
• launcher
• load
• util
• reporter

2.2.1 EJB Package Structure

The beans adhere to the EJB 1.1 specification. There are currently two versions for some of the beans - CMP and BMP. The CMP version is to be used with Container Managed Persistence and the BMP version must be used with Bean Managed Persistence. The BMP beans are sub classed from CMP.

The source files for the CMP and BMP are respectively named <bean>CmpEJB.java and <bean>BmpEJB.java. The kit includes pre-compiled class files for all the beans (both CMP and BMP versions) and you can package them into appropriate ejb-jars for your environment. For CMP, object to database (OR) mapping information is available in the README.CMP file located in the src/deploy directory.

Of the packages listed above, corp, orders, mfg, and supplier contain the beans used for the four SPECjAppServer2001 domains respectively. Each domain package contains a distinct sub-packages for each EJB and its helper classes. The EJB package contains a sub-package named "ejb" which holds the bean implementation including the home and remote interfaces. For instance, the following shows the fully qualified class names for the OrderLineEnt entity bean:

     org.spec.jappserver.orders.orderlineent.ejb.OrderLineEntHome
     org.spec.jappserver.orders.orderlineent.ejb.OrderLineEnt
     org.spec.jappserver.orders.orderlineent.ejb.OrderLineEntPK
     org.spec.jappserver.orders.orderlineent.ejb.OrderLineCmpEJB
     org.spec.jappserver.orders.orderlineent.ejb.OrderLineBmpEJB

The helper classes for the orders domain are in the helper sub-package under orders as follows:

     org.spec.jappserver.orders.helper.CustomerInfo
     org.spec.jappserver.orders.helper.CustomerStatus
     org.spec.jappserver.orders.helper.InsufficientCreditException
     org.spec.jappserver.orders.helper.ItemQuantity
     org.spec.jappserver.orders.helper.OrderStatus

2.2.2 Other Java Packages

Besides the four domains, several other packages are included:

The common package contains the source code for helper classes used by all the beans (i.e. it is common across different domains as well as beans)

The ruleengine package contains the rule parser used by the corp domain to evaluate order discounts

The util package contains the beans used to generate primary keys

The webbeans directory contains Java Beans and support classes used by the web interface

The supplier.emulator package contains the Supplier Emulator Servlet.

The charts package contains a stand-alone Java application that can be used to chart transaction throughput during a SPECjAppServer2001 benchmark run (source not included). Note: To run the charts application you need JClass Chart from Sitraka Software

The driver package contains the SPECjAppServer2001 drivers.

The load package contains the database data generation and load programs required to load the SPECjAppServer2001 database(s) according to the benchmark scaling rules.

xerces.jar is used by Delivery Servlet in Supplier domain and the Supplier Emulator Servlet. Add this jar file in the Application server classpath if XML DOM Parser packages are not part of your application server. In the case of J2EE RI server this jar should be added in the J2EE_CLASSPATH by editing J2EE_HOME/bin/userconfig.sh. To include this jar in the Tomcat server, copy xerces.jar file to $TOMCAT_HOME/lib directory.

Section 3 - Building and Deploying SPECjAppServer2001

There are several components required to build, deploy, test and run SPECjAppServer2001. These are :

• An EJB Container to deploy all the SPECjAppServer2001 Beans.
• A Web Container to deploy the web client as well as the Delivery Servlet in the Supplier Domain.
• An external Web Server to run the Supplier Emulator Servlet. This web server should turn off the keep-alive option. The emulator is not part of the SUT and may not be deployed on the same system as the EJB container or web container.
• The SPECjAppServer2001 Driver that runs on client systems external to the SUT

Building and deploying the benchmark requires ant, which is available from http://jakarta.apache.org/ant.

3.1 Build and Deployment process

Several steps must be accomplished prior to running the SPECjAppServer2001 benchmark:

• Create the SPECjAppServer2001 database(s)
• Configure the environment
• Build the .jar, .war, and .ear files
• Load the database
• Deploy the .ear files
• Deploy the emulator
• Test the SPECjAppServer2001 deployment using the Web interface
• Configure the driver to run SPECjAppServer2001

3.1.1 Create the Database(s)

For the Standard Workload, you can create a single database that houses all the 4 Domains. For the Distributed Workload, you must create 4 separate databases, one for each domain.

Standard SQL scripts for creating the database schema are provided in schema/sql. These are intended to give a starting point for creating schemas for other database products.

For convenience database creation scripts are provided for Oracle 8 & 9i, DB2 and Sybase in schema/<DBMS>. These are example scripts and can be modified. However, see section 1.4 of the SPECjAppserver2001 Run and Reporting Rules for modification requirements.

3.1.2 Building and Deploying on the J2EE RI

This section describes the steps required to build and deploy SPECjAppServer2001 on the J2EE Reference Implementation (RI). These directions assume that the J2EE RI version 1.3.1 is used and that it is installed in the directory $J2EE_HOME.

This example also uses an Oracle database assumed to be running on host 'orahost' with instance name 'specdb' and accessed through the Oracle Thin driver.

3.1.2.1 Configure the RI EJB Server Classpath

Set the J2EE_CLASSPATH variable in the $J2EE_HOME/bin/userconfig.sh file to include your JDBC Driver class libraries as well as the XML DOM Parser library - $JAS_HOME/jars/xerces.jar.

Example for Oracle (assuming that Oracle is installed in $ORACLE_HOME on the same machine):

     J2EE_CLASSPATH=$ORACLE_HOME/jdbc/lib/classes12.zip:$JAS_HOME/jars/xerces.jar

3.1.2.2 Configure JDBC Driver, URL and Data Sources

Use the RI's j2eeadmin tool to add a new JDBC driver, URL and the Data Source jdbc/SPECDB.

Example for Oracle:

     $ cd $J2EE_HOME/bin
     $ j2eeadmin -addJdbcDriver oracle.jdbc.driver.OracleDriver
     $ j2eeadmin -addJdbcDatasource jdbc/SPECDB jdbc:oracle:thin:@orahost:1521:specdb

Note: By default SPECjAppServer2001 assumes that the account used to access the database has username 'spec' and password 'spec'. If you are using different username and/or password you should include these in the JDBC URL above as follows:

     $ j2eeadmin -addJdbcDatasource jdbc/SPECDB \
     jdbc:oracle:thin:username/password@orahost:1521:specdb

3.1.2.3 Configure the Kit for your Deployment Environment

Make sure that the file $JAS_HOME/config/appserver contains "ri.1.3.1" (as by default).

Edit the $JAS_HOME/config/ri.1.3.1.env properties file.

You need to set the following properties:

• JAVA_HOME - the directory where your Java runtime environment is installed.
• J2EE_HOME - the directory where your J2EE RI is installed.
• JDBC_CLASSPATH - your JDBC driver class libraries (e.g. for Oracle's thin driver use the file $ORACLE_HOME/jdbc/lib/classes12.zip where $ORACLE_HOME is the top-level directory of your Oracle installation)
• JAS_HOST - the URL of the server in which the SPECjAppServer beans should be deployed.
• EMULATOR_HOST - the URL of the server in which the Supplier Emulator should be deployed.

Ensure you can get to the ant binaries by including ant in your PATH.

Unix: $ export PATH=$JAS_HOME/ant/bin:$PATH

Windows: C:\>set PATH=%JAS_HOME%\ant\bin;%PATH%

3.1.2.4 Build the .JAR, .WAR and .EAR files

     $ cd $JAS_HOME
     $ ant clean // makes sure the workspace is clean 
                 // before starting to build
     $ ant       // build the SPECjAppServer2001 application 
                    components using the default build.xml

3.1.2.5 Build Load Programs and Initialize the Database(s)

Edit the $JAS_HOME/config/*db.properties files to establish the correct JDBC connection properties for your database.

Example for Oracle:

     dbURL = jdbc:oracle:thin:@orahost:1521:specdb
     dbDriver = oracle.jdbc.driver.OracleDriver
     jdbcVersion = 2

Note: In case you are using different database username/password than the default spec/spec you will also have to edit the dbUser and dbPassword properties in the above files.

Now load the database:

     $ cd $JAS_HOME
     $ ant loaddb

Note: To load the database for higher Injection Rate runs use the bin/loaddb.sh script.

3.1.2.6 Start the J2EE RI Server on host $JAS_HOST

     $ cd $J2EE_HOME/bin
     $ j2ee

3.1.2.7 Deploy the SPECjAppServer EJBs

To deploy the SPECjAppServer EJBs to the running J2EE RI server on $JAS_HOST do the following:

     $ cd $JAS_HOME
     $ ant -f ri.xml

3.1.2.8 Deploy the Supplier Emulator

To deploy the Emulator to the running J2EE RI server on $EMULATOR_HOST do the following (for simplicity we assume that the Emulator is deployed in the same J2EE RI Server as the EJBs):

     $ cd $JAS_HOME
     $ ant -f ri.xml emulator-deploy

or

To build and deploy emulator on Tomcat server, edit the $JAS_HOME/config/tomcat.env properties file.

     $ cd $JAS_HOME
     $ ant -f tomcat.xml
     $ cp jars/Emulator.war $TOMCAT_HOME/webapps
     $ rm -rf $TOMCAT_HOME/webapps/Emulator // if it is a redeployment

3.1.3 Building and Deploying on another Application Server

This section describes the general steps necessary to build and deploy SPECjAppServer2001 on an application server from scratch. Additional information on configuring SPECjAppServer2001 for use with specific application servers is available from:

http://www.spec.org/jAppServer2001/vendor_setup/index.html

3.1.3.1 Steps required to Build and Deploy SPECjAppServer2001 on an Application Server

1. Choose a short name for your AppServer. For this example, let's call your AppServer "mango"
2. Copy the file config/ri.env to config/mango.env and edit the properties particular to mango. It may very well be that you won't need to make any changes to these settings at this time, but you may need to add settings later to support deploy actions.
3. Copy the directory src/deploy/reference to src/deploy/mango with all its contents. Add vendor specific deployment descriptors as necessary. There are two Stateful Session Beans in the benchmark, CartSes and BuyerSes. The idle time-out for CartSes should be set to run duration, as the driver reuses them randomly during the run, and for BuyerSes it should be set as short as possible.
4. Try a build by doing "ant -Dappserver=mango" This will do a full compile and try to build all the files needed for deployment.
5. Edit the *db.env files in the config directory so they have correct DB driver info in preparation for doing a DB load.
6. Try a database load by doing "ant -Dappserver=mango loaddb" Once those are working, you should create a new file, mango.xml which has the deploy rules. These will vary widely but you can use ri.xml as an example. You should try to create the targets:
   
7. • ecperf-war-deploy
   • ecperf-ejb-deploy
   • supplier--deploy
   • emulator--deploy

When these are working, you should package up the mango.xml, config/mango.env, and src/deploy/mango/* files, as well as any of the confg/*db.env files that may have native JDBC drivers referenced in them. For convenience, include instructions on how to deploy SPECjAppServer2001 for your AppServer.

3.1.3.2 Tips for Building and Deploying on another Application Server

Using the CMP version of SequenceEnt requires that UtilDataSource defined in util.xml have its isolation level set to SERIALIZABLE. The CMP version will only work if your Container/DBMS support the Serializable isolation level correctly. Consequently, we suggest that you use the BMP version of SequenceEnt.

The web client and emulator are deployed as .war files. If your AppServer does not support these formats, then you will have to make appropriate changes to <appserver>.xml to compile and deploy them.

3.1.3.3 Summary of steps required to run SPECjAppServer2001

1. Set up the AppServer and database
2. Add the ant bin dir to your path / classpath
3. Adjust settings in config/mango.env for your local environment.
4. On the main SPECjAppServer2001 server from $JAS_HOME do "ant -find mango.xml ecperf-ejb deploy"
5. On the main web server, even if it's the same, from $JAS_HOME do "ant -find mango.xml ecperf-war-deploy supplier-war-deploy"
6. On the emulator server from $JAS_HOME do: "ant -find mango.xml emulator-war-deploy"
7. Load the database from $JAS_HOME by doing "ant -find mango.xml loaddb"

3.1.4 Loading the Database(s)

Now that the database and tables are  created, and the load programs are compiled, follow these steps to load the database:

Edit the $JAS_HOME/config/appsserver file and replace the current name with the name of the application server being used.

If the file $JAS_HOME/config/<appsserver>.env does not exist, copy it from $JAS_HOME/config/ri.env

Edit the $JAS_HOME/config/<appsserver>.env file and ensure the JDBC_CLASSPATH variable points to a valid JDBC driver and $JAS_HOME points to root of SPECjAppServer2001 distribution. Note that the other variables are not used during the load.

Uncomment the JDBC URL and Driver class in the $JAS_HOME/config/*db.properties files depending on what database and driver you are using to access it. In the centralized configuration, the contents of all files are the same.

Run the bin/loaddb.sh script from the $JAS_HOME directory to load all the databases. Read the script to understand the order in which loads must be done. The LoadCorp program writes out data to temporary files in /tmp. These files are then read by the other load programs to populate their respective domains. If any of the load programs fail, follow these steps:

• If LoadCorp failed, all of the domains will have to be re-loaded. Re-run bin/loaddb.sh.
• If one of the other domain loads fail, you can selectively re-load only this domain as long as the /tmp/*pipe files created by LoadCorp have not been deleted.

Note: If you reload the database you need to restart the AppServer as the SequenceSes bean caches primary key values to insert new rows in the database.

3.1.5 Testing the Deployment using the Web Interface

Assuming the application server is configured properly and the beans have been successfully deployed, you should be able to test the functionality of the SPECjAppServer2001 application using the web interface. If the deployment descriptor puts the web pages under the SPECjAppServer2001 context as defined by the deployment descriptors supplied for the J2EE RI, you should be able to access the web page by pointing to the URL

http://<JAS_HOST>:<JAS_PORT>/SPECjAppServer2001/

Test that all the functionality is in place by trying the following transactions through the web interface:

3.1.5.1 Orders Domain

In Orders Domain:

• Create a new order with one of the items having a quantity of more than 100 . This should cause a large order to be created in the Manufacturing Domain
• Change the order
• Get order information
• Get information on all orders of a customer
• Cancel an order

3.1.5.2 Manufacturing Domain

In Manufacturing Domain:

• Get a large order and start processing a work order based on it
• Move the resulting work order through its various stages.
• 
  
  
• Create a new work order
• Cancel the work order
• 
  
  
• Schedule a work order
• Order 20000 items (this will force an interaction between the SUT and the Emulator)
• Move the resulting work order through its various stages.

3.1.5.3 Delivery and Emulator Servlets

To make sure Delivery and Emulator servlets are functioning you should be able to connect to the following URLs.

http://<JAS_HOST>:<JAS_PORT><JAS_PREFIX>Supplier/DeliveryServlet

Note: In most cases the JAS and EMULATOR PREFIX will be '/'

If all of the above run successfully, you are ready to run the benchmark using the Driver.

Section 4 - Running SPECjAppServer2001

Running SPECjAppServer2001 requires that the driver be configured for your environment and particular application server. Configuring the driver properly requires understanding how the driver works, which is described below.

4.1 The SPECjAppServer2001 Driver

The SPECjAppServer2001 Driver consists of several Java programs and is designed to run on multiple client systems, using an arbitrary number of JVMs to ensure that the Driver has no inherent scalability limitations. Note that none of the client systems are part of the SUT.

4.1.1 Driver Definitions

The Master machine: This is the machine from which you drive the benchmark run. It is typically one of the client systems.

The Agent machines: These are the client systems on which the various workloads run. The Master machine can also be an Agent machine.

4.1.2 Driver Components

The Driver consists of the following components:

The actual applications that implement the workload are defined in the specifications. These are OrderEntry, PlannedLine and LargeOrderLine. 

There is one Agent type for each of the applications. The Agents are OrdersAgent, MfgAgent and LargeOLAgent. You can configure as many OrdersAgent and MfgAgent agents as you wish on any number of client systems.  There must be at least one of each Agent type on the Master machine if OrdersAgent and/or MfgAgent are to be run.

There is one Controller which runs on the Master machine and with which all the Agents register.

There is one Driver which runs on the Master machine and which is responsible for doing a benchmark run.

4.1.3 How The Driver Works

The Driver communicates with all the Agents using RMI. The Driver reads the run properties and configures the Agents appropriately. Each Agent will then run as many threads of their respective workload. For example, the OrdersAgent will run OrderEntry. The number of threads is determined by the scaling rules of the specification and are equally distributed amongst all Agents. Each thread runs independently, executing the workload according to the rules defined in the spec. When the run completes, the Driver co-ordinates with the Agents to retrieve all the stats and prints out the reports.

4.1.4 The Driver's InitialContext and Lookups

In the past, some people have had problems with the way the Driver gets its InitialContext and lookups. This section explains exactly how this is done.

The Driver obtains the JNDI InititalContext by executing the following code in each of the client threads in Agent processes :

     try {
          context = new InitialContext();
     } catch (NamingException ne) {
          throw new RemoteException("InitialContext failed. " + ne);

Each of the bean home interfaces that the applications require is looked up using the absolute home name of the bean. An example is shown below :

     OrderSesHome orderSesHome =
                (OrderSesHome) PortableRemoteObject.narrow
                (context.lookup("OrderSesHome"), 
                OrderSesHome.class);

If your app server needs extra properties to be set for the InitialContext, this can be done by setting them in jndi.properties in the CLASSPATH or changing the variable JAVA in config/<appserver>.env. The example below shows a generic implementation. Specific values for several application servers are provided in the subsequent sections.

     JAVA="$JAVA_HOME/bin/java
     -Djava.naming.factory.initial=vendor naming factory goes here
     -Djava.naming.provider.url=vendor url goes here

iPlanet

     -Djava.naming.factory.initial=com.sun.jndi.cosnaming.CNCtxFactory
     -Djava.naming.provider.url=iiop://${JAS_HOST}:${IIOP_PORT}

WebLogic

 
     -Djava.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
     -Djava.naming.provider.url=t3://${JAS_HOST}:${JAS_PORT}

WebSphere

     -Djava.naming.factory.initial=com.ibm.websphere.naming.WsnInitialContextFactory
     -Djava.naming.provider.url=iiop://${JAS_HOST}:${IIOP_PORT}"

4.2 Configuring and Running the Driver

Follow these steps to configure the Driver for you environment and particular application server:

Edit $JAS_HOME/config/appsserver to contain the name of the application server you want to run.

Edit $JAS_HOME/config/<appsserver>.env to contain the correct values for the following variables :

• JAVA_HOME - Path name of your JDK installation
• <APPSERVER>_HOME - Path to your app server installation
• JAS_HOME - Path to the root directory of SPECjAppServer2001 distribution
• CLASSPATH - Path to all the library and client jar files needed to run an EJB client for your particular application server
• BINDWAIT - Time to wait for the RMI server to bind when starting the Driver; defaults to 5 seconds
• JAVA - The Java executable to run. Some application servers require the client to call a wrapper instead of the java command. If this is not set, it defaults to $JAVA_HOME/bin/java

Edit $JAS_HOME/config/run.properties and set the SPECjAppServer2001 run parameters appropriately. The parameters are described in comments in this file. Ensure that dumpStats is 0 if you are not running the charting application, as otherwise the Driver will hang waiting for the reader on the pipe.

Edit $JAS_HOME/config/agent.properties. There are currently no values you can change in this file.

Check that the method used by the Driver to get its InitialContext and do lookups will work for your application server.

Now, run the driver by running the script bin/driver.sh [<driver host>] from the $JAS_HOME directory.

You can abort a run at any time by pressing Ctrl-C. This will cause all the processes to exit gracefully.

If there were errors during pervious run, M_largeorder table may have entries which will affect the BOPS of the run. So make sure before each run the M_largeorder table is empty by reloading the database or by deleting entries from M_largeorder table using a SQL command.

4.2.1 Driver Configuration Tips

If no <driver host> is specified the default local host is used.

We have sometimes experienced binding exceptions at the start of the first time run. For now, just cancel the script (by pressing Ctrl-C) and rerun it again. This problem usually does not persist to the second run.

The driver.sh script starts all the Agents on the Master machine. If you want to start the Agents on different client systems, the script needs to be edited appropriately. "bin/driver.sh <driver host>" must be started on the other client systems after starting the script on the Master machine. <driver host> is the name of the Master machine. The properties file must be configured correctly on the client systems, with the appropriate configurations in the $JAS_HOME/config/run.properties for the OrderAgent and the MfgAgent to be run on the client systems.

A driver.bat script is provided for running SPECjAppServer2001 on MS Windows systems.

4.3 Running the Atomicity Tests

Section 2.2 of the SPECjAppServer Run and Reporting Rules specifies the requirements for atomicity of the transactions. The code to run the various atomicity tests is implemented via the Debug class in the OrderEnt and LargeOrderEnt beans. Debug level 4 is used for the atomicity tests. To run Atomicity Tests 1 and 2, change the environment variable debuglevel of OrderEnt to 4 in ejb-jar.xml in the META-INF directory of the Orders EJB JAR orders.jar. Similarly, to run Atomicity Test 3, change debuglevel of LargeOrderEnt to 4 in ejb-jar.xml in the META-INF directory of the Manufacturing EJB JAR mfg.jar.

4.4 Tips and Troubleshooting

See the following URL for the latest tips and troubleshooting information on SPECjAppServer2001:

http://www.spec.org/jAppServer2001/docs/SupportFAQ.html

Section 5 - Results

After completing a run, the SPECjAppServer2001 driver will generate a result.props file in the output directory containing the raw results from the benchmark. The contents of result.props are used in the Submission File from which the SPECjAppServer2001 report is generated.

5.1 Creating a Submission File

The Submission File contains a detailed description of the SUT in text format that is used by the SPECjAppServer2001 reporter to produce a report suitable for publication.

To create a SPECjAppServer2001 Submission File:

• Start with the Sample_Submission.txt file provided.
• Edit the fields to describe the details of your SUT. A detailed description of each field and example are provided in the sample file.
• Append the contents of the result.props file to the end of the Submission File.

5.2 Generating a Report

SPECjAppServer2001 includes a utility for generating a results page called the reporter. The report contains a summary of the key results and a complete system description in a format designed to be consistent with other SPEC reporting pages.

To run the reporter, use the following command:

     # java -classpath reporter.jar reporter [-a] <submission file>.txt

Where <submission file>.txt is the Submission File created in the previous step.

An HTML file named <filename>.report.html is created by default.

The "-a" option will create a text report page named <filename>.report.txt.

5.3 Submitting the Results

Once you have a successful run, you can submit the results to the SPEC OSG Java subcommittee for review. To submit a result to SPEC:

1. Create a configuration diagram (in PNG, JPEG or GIF format) for your SUT (see SPECjAppServer Run and Reporting Rules section 5.2 for a description of the Configuration Diagram).
2. Create an Full Disclosure Archive (in ZIP, TAR, or JAR format) with the results of your run (see SPECjAppServer2001 Run and Reporting Rules section 5.3 for a list of the items in the Archive).
3. Create a Submission File as described in section 5.1 above. The Submission File should contain the file name for the Configuration Diagram and the Archive.
4. Put just these three files in a JAR file.
5. Mail this JAR file as an attachment to subjAppServer2001@spec.org.

Every submission goes through a minimum two-week review process, starting on a scheduled SPEC OSG Java sub-committee conference call. During the review, members of the committee may ask for additional information or clarification of the submission. Once the result has been reviewed and approved by the committee, it is displayed on the SPEC web site at http://www.spec.org/.

Copyright (c) 2002 Standard Performance Evaluation Corporation