AppSuite:Load Tests

Software Requirements

• Java SE JDK6 or JDK7
• Git
• SBT

Getting Performance Repository

• Cloning Repository

 $ git clone https://git.open-xchange.com/git/wd/testing/performance

• Checkout a branch, e.g. develop

 $ git checkout develop

Install SBT

Installing sbt on Mac

• Macports

 port install sbt

• Homebrew

 brew install sbt

Development Environment

If you want to execute the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:

• Eclipse 4.2 or later
• Scala IDE plugin

Importing Project into Eclipse

1. Open Eclipse
2. Click on File → Import... → Maven → Existing Maven Project
3. Select the gatling folder within the performance repository

Configurable Parameters

List of all available System-Properties and JVM-Arguments:

protocol

Which protocol to use. Possible values are: "http" or "https". Default value: "http".

host

Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".

users

How many concurrent users to simulate. Default value: "5".

durationTime

Duration time of the simulation. Default value: "30".

durationUnit

The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".

successpercent

Successful responses in percentage. Default value: "100".

reqpersec

Mean number of requests per second (lower bound). Default value: "0".

resptime

Mean response time (lower bound). Default value: "0".

csv_users

Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".

syncmsgmax

How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".

pimObjects

How many PIM Objects, e.g. contacts, should be created. Default value: "5".

Snippet from the <properties>-section in the pom.xml

 <properties>
   <protocol>http</protocol>
   <host>localhost</host>
   <port>80</port>
   <users>5</users>
   <durationTime>30</durationTime>
   <durationUnit>SECONDS</durationUnit>
   <successpercent>100</successpercent>
   <reqpersec>0</reqpersec>
   <resptime>0</resptime>
   <csv_users>ox_user_credentials.csv</csv_users>
   <syncmsgmax>50</syncmsgmax>
   <pimObjects>5</pimObjects>
 </properties>

Snippet from the <jvmArgs>-section in the pom.xml

 <jvmArgs>
   <jvmArg>-Dprotocol=${protocol}</jvmArg>
   <jvmArg>-Dhost=${host}</jvmArg>
   <jvmArg>-Dport=${port}</jvmArg>
   <jvmArg>-Dcsv_users=${csv_users}</jvmArg>
   <jvmArg>-Dusers=${users}</jvmArg>
   <jvmArg>-DdurationTime=${durationTime}</jvmArg>
   <jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
   <jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
   <jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
   <jvmArg>-Dresptime=${resptime}</jvmArg>
   <jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
   <jvmArg>-DpimObjects=${pimObjects}</jvmArg>
 </jvmArgs>

Test Execution

Pre-Requisites for the Open-Xchange Groupware Server

Disabling rate limitation

• Modify the parameter maxRateLenientClients @server.properties:

 $ vim /opt/etc/open-xchange/etc/server.properties

• Specify a new User-Agent which is excluded from rate limit checks

 "GATLING*"

Resources/Data

In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.

• Structure of the csv file:

 
 username,password
 user1,secret
 user2,secret
 user3,secret
 ...
 user998,secret
 user999,secret
 user1000,secret
 

Pre-compiled version of the gatling tests

Execute the gatling tests via bash script (Linux) or batch file (Windows).

The precompiled version is available here: https://jenkins.open-xchange.com/view/QA/job/qaPerformanceGatlingTests/lastSuccessfulBuild/artifact/performance/com.openexchange.test.performance.gatling/target/

• Unzip ox-gatling-*-bin.zip
• Change working directory

 $ cd Downloads/ox-gatling-1.0.0-SNAPSHOT

• Please adjust the JAVA_OPTS in the scripts bin/box-gatling.sh or bin/ox-gatling.bat to match your needs.
  • The following example will stress the server with the IP 10.0.0.1 for 120 seconds and hold the load with permanently 100 user:

 exec "$JAVACMD" $JAVA_OPTS -Dprotocol=http -Dhost=10.0.0.1 -Dcsv_users=ox_user_credentials.csv -Dusers=100 -DdurationTime=120 -DdurationUnit=SECONDS \
 -classpath "$CLASSPATH" \
 -Dapp.name="ox-gatling" \
 -Dapp.pid="$$" \
 -Dapp.repo="$REPO" \
 -Dapp.home="$BASEDIR" \
 -Dbasedir="$BASEDIR" \
 io.gatling.app.Gatling \
 "$@"

• Execute the script

 $ /bin/bash bin/ox-gatling.sh

 Choose a simulation number:
    [0] com.openexchange.test.performance.gatling.tests.QAPerfOX6
    [1] com.openexchange.test.performance.gatling.tests.QAPerfOXAppSuite
    [2] com.openexchange.test.performance.gatling.tests.QAPerfUSM
    [3] com.openexchange.test.performance.gatling.tests.QAPerfOXDrive
    [4] com.openexchange.test.performance.gatling.tests.QAPerfEAS

Now you will see some output of the compilation process but finally you should be able to start the simulation by entering a number.

Execute the gatling tests via maven plugin

• Switch to the root directory of the gatling project

 cd /GIT_REPO/performance/com.openexchange.test.performance.gatling/

• Now you can start the performance tests either by using the default values from the pom.xml,

 mvn clean gatling:execute

or by temporarily passing system properties:

 mvn clean gatling:execute -Dhost=localhost -Dcsv_users=ox_user_credentials.csv -Dusers=5 -DdurationTime=20 -DdurationUnit=MINUTES -Dgatling.simulationClass=com.openexchange.test.performance.gatling.tests.QAPerfOX6

Practical hints and tips

• If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
• If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.