AppSuite:Load Tests

Software Requirements

• Java SE JDK6 or JDK7
• Git
• SBT

Getting Performance Repository

• Cloning Repository

$ git clone https://git.open-xchange.com/git/wd/testing/performance

• Checkout a branch, e.g. develop

$ git checkout develop

Install SBT

Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.

Installing sbt on Mac

• Macports

$ port install sbt

• Homebrew

$ brew install sbt

Installing sbt on Windows

• Download the msi installer and install it.

Installing sbt on Linux

• RPM package
• DEB package

Installing sbt manually

• Manual instructions

Build and execute the tests with the SBT-Plugin

After SBT has been successfully installed we can switch to the project's directory:

$ cd ~/performance/com.openexchange.test.performance.gatling

SBT can be used both as a command line script and as a build console. Most of the commands can be run standalone by passing the command as an argument to SBT, e.g.

$ sbt update

Display all available projects:

$ sbt projects
[info] Loading project definition from ~/performance/com.openexchange.test.performance.gatling/project
[info] Set current project to root (in build file:~/performance/com.openexchange.test.performance.gatling/)
[info] In file:~/performance/com.openexchange.test.performance.gatling/
[info] 	   project-ajax
[info] 	   project-common
[info] 	   project-drive
[info] 	   project-guard
[info] 	   project-in8
[info] 	   project-jolokia
[info] 	   project-usm
[info] 	 * root

Switch to a specific project:

root> project project-ajax
[info] Set current project to project-ajax (in build file:~/performance/com.openexchange.test.performance.gatling/)
project-ajax>

To execute all tests in the project just enter:

project-ajax> gatling:test
[info] Formatting 21 Scala sources {file:~/performance/com.openexchange.test.performance.gatling/}project-common(test) ...
[info] Formatting 66 Scala sources {file:~/performance/com.openexchange.test.performance.gatling/}project-ajax(test) ...
[info] Compiling 21 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-common/target/scala-2.10/test-classes...
[info] Compiling 66 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-ajax/target/scala-2.10/test-classes...
[info] Compiling 66 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-ajax/target/scala-2.10/gatling-classes...
Simulation com.openexchange.test.performance.gatling.simulations.QAPerfOX6 started...

Note that if a command takes arguments, you need to quote the entire argument path, e.g.

$ sbt 'project-ajax/gatling:testOnly com.openexchange.test.performance.gatling.simulations.QAPerfOX6'

The following command line script will execute the Gatling OX6 Performance test:

$ sbt "project project-ajax" 'set javaOptions ++=Seq("-Dgatling_host=10.0.0.1", "-Dgatling_protocol=http", "-Dgatling_port=80", "-Dgatling_user=100", "-Dgatling_duration_time=120", "-Dgatling_duration_unit=SECONDS")' clean "gatling:testOnly com.openexchange.test.performance.gatling.simulations.QAPerfOX6"

Development Environment

If you want to execute/debug the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:

IntelliJ IDEA

Download and install the IntelliJ IDEA Community Edition

Eclipse

• Eclipse 4.2 or later
• Scala IDE plugin

Since there are a few problems with the Scala IDE for eclipse we recommend to use IntelliJ which works out of the box.

Importing Project into IntelliJ

Create project files

To create the Idea project files please use the sbt task gen-idea.

$ cd ~/git/performance/com.openexchange.test.performance.gatling

$ sbt gen-idea
[info] Loading project definition from ~/git/performance/com.openexchange.test.performance.gatling/project
[info] Set current project to root (in build file:~/git/performance/com.openexchange.test.performance.gatling/)
...

This task will resolve all project dependencies and create the IDEA modules accordingly.

Import project

1. Start IntelliJ
2. Click on Import Project...
3. Select the gatling folder within the performance repository
4. Import project from external model → SBT Project → Finish

Configurable Parameters

List of all available System-Properties and JVM-Arguments:

gatling_protocol

Which protocol to use. Possible values are: "http" or "https". Default value: "http".

gatling_host

Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".

gatling_port

Which port to use. Possible values are: "80", "8009"

gatling_user

How many concurrent users to simulate. Default value: "5".

gatling_duration_time

Duration time of the simulation. Default value: "30".

gatling_duration_unit

The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".

gatling_successpercent

Successful responses in percentage. Default value: "100".

gatling_reqpersec

Mean number of requests per second (lower bound). Default value: "0".

gatling_resptime

Mean response time (lower bound). Default value: "0".

gatling_user_feeder

Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".

gatling_basic_auth_user

The user name of the basic access authentication.

gatling_basic_auth_pass

The password of the basic access authentication.

gatling_syncmsgmax

How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".

gatling_pimObjects

How many PIM Objects, e.g. contacts, should be created. Default value: "5".

Test Execution

Pre-Requisites for the Open-Xchange Groupware Server

Disabling rate limitation

• Modify the parameter maxRateLenientClients @server.properties:

 $ vim /opt/etc/open-xchange/etc/server.properties

• Specify a new User-Agent which is excluded from rate limit checks

 "GATLING*"

Resources/Data

In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.

• Structure of the csv file:

 
 username,password
 user1,secret
 user2,secret
 user3,secret
 ...
 user998,secret
 user999,secret
 user1000,secret
 

Practical hints and tips

• If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
• If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.