AppSuite:Load Tests

Software Requirements

• Java SE JDK6 or JDK7
• Git
• SBT

Getting Performance Repository

• Cloning Repository

 $ git clone https://git.open-xchange.com/git/wd/testing/performance

• Checkout a branch, e.g. develop

 $ git checkout develop

Install SBT

Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.

Installing sbt on Mac

• Macports

 $ port install sbt

• Homebrew

 $ brew install sbt

Installing sbt on Windows

• Download the msi installer and install it.

Installing sbt on Linux

• RPM package
• DEB package

Installing sbt manually

• Manual instructions

Building with SBT

After SBT has been successfully installed we can switch to the project's directory:

 $ cd ~/performance/com.openexchange.test.performance.gatling

Development Environment

If you want to execute the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:

• Eclipse 4.2 or later
• Scala IDE plugin

Importing Project into Eclipse

1. Open Eclipse
2. Click on File → Import... → Maven → Existing Maven Project
3. Select the gatling folder within the performance repository

Configurable Parameters

List of all available System-Properties and JVM-Arguments:

protocol

Which protocol to use. Possible values are: "http" or "https". Default value: "http".

host

Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".

users

How many concurrent users to simulate. Default value: "5".

durationTime

Duration time of the simulation. Default value: "30".

durationUnit

The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".

successpercent

Successful responses in percentage. Default value: "100".

reqpersec

Mean number of requests per second (lower bound). Default value: "0".

resptime

Mean response time (lower bound). Default value: "0".

csv_users

Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".

syncmsgmax

How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".

pimObjects

How many PIM Objects, e.g. contacts, should be created. Default value: "5".

Snippet from the <properties>-section in the pom.xml

 <properties>
   <protocol>http</protocol>
   <host>localhost</host>
   <port>80</port>
   <users>5</users>
   <durationTime>30</durationTime>
   <durationUnit>SECONDS</durationUnit>
   <successpercent>100</successpercent>
   <reqpersec>0</reqpersec>
   <resptime>0</resptime>
   <csv_users>ox_user_credentials.csv</csv_users>
   <syncmsgmax>50</syncmsgmax>
   <pimObjects>5</pimObjects>
 </properties>

Snippet from the <jvmArgs>-section in the pom.xml

 <jvmArgs>
   <jvmArg>-Dprotocol=${protocol}</jvmArg>
   <jvmArg>-Dhost=${host}</jvmArg>
   <jvmArg>-Dport=${port}</jvmArg>
   <jvmArg>-Dcsv_users=${csv_users}</jvmArg>
   <jvmArg>-Dusers=${users}</jvmArg>
   <jvmArg>-DdurationTime=${durationTime}</jvmArg>
   <jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
   <jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
   <jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
   <jvmArg>-Dresptime=${resptime}</jvmArg>
   <jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
   <jvmArg>-DpimObjects=${pimObjects}</jvmArg>
 </jvmArgs>

Test Execution

Pre-Requisites for the Open-Xchange Groupware Server

Disabling rate limitation

• Modify the parameter maxRateLenientClients @server.properties:

 $ vim /opt/etc/open-xchange/etc/server.properties

• Specify a new User-Agent which is excluded from rate limit checks

 "GATLING*"

Resources/Data

In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.

• Structure of the csv file:

 
 username,password
 user1,secret
 user2,secret
 user3,secret
 ...
 user998,secret
 user999,secret
 user1000,secret
 

Pre-compiled version of the gatling tests

Execute the gatling tests via bash script (Linux) or batch file (Windows).

The precompiled version is available here: https://jenkins.open-xchange.com/view/QA/job/qaPerformanceGatlingTests/lastSuccessfulBuild/artifact/performance/com.openexchange.test.performance.gatling/target/

• Unzip ox-gatling-*-bin.zip
• Change working directory

 $ cd Downloads/ox-gatling-1.0.0-SNAPSHOT

• Please adjust the JAVA_OPTS in the scripts bin/box-gatling.sh or bin/ox-gatling.bat to match your needs.
  • The following example will stress the server with the IP 10.0.0.1 for 120 seconds and hold the load with permanently 100 user:

 exec "$JAVACMD" $JAVA_OPTS -Dprotocol=http -Dhost=10.0.0.1 -Dcsv_users=ox_user_credentials.csv -Dusers=100 -DdurationTime=120 -DdurationUnit=SECONDS \
 -classpath "$CLASSPATH" \
 -Dapp.name="ox-gatling" \
 -Dapp.pid="$$" \
 -Dapp.repo="$REPO" \
 -Dapp.home="$BASEDIR" \
 -Dbasedir="$BASEDIR" \
 io.gatling.app.Gatling \
 "$@"

• Execute the script

 $ /bin/bash bin/ox-gatling.sh

 Choose a simulation number:
    [0] com.openexchange.test.performance.gatling.tests.QAPerfOX6
    [1] com.openexchange.test.performance.gatling.tests.QAPerfOXAppSuite
    [2] com.openexchange.test.performance.gatling.tests.QAPerfUSM
    [3] com.openexchange.test.performance.gatling.tests.QAPerfOXDrive
    [4] com.openexchange.test.performance.gatling.tests.QAPerfEAS

Now you will see some output of the compilation process but finally you should be able to start the simulation by entering a number.

Execute the gatling tests via maven plugin

• Switch to the root directory of the gatling project

 cd /GIT_REPO/performance/com.openexchange.test.performance.gatling/

• Now you can start the performance tests either by using the default values from the pom.xml,

 mvn clean gatling:execute

or by temporarily passing system properties:

 mvn clean gatling:execute -Dhost=localhost -Dcsv_users=ox_user_credentials.csv -Dusers=5 -DdurationTime=20 -DdurationUnit=MINUTES -Dgatling.simulationClass=com.openexchange.test.performance.gatling.tests.QAPerfOX6

Practical hints and tips

• If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
• If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.