Skip to content

Test startup process

Summary

As your test gets started, we will automatically create instances and inside them docker containers.

Since there are a number of ways you can influence this startup process, we have taken the time to describe it here. As you will see most of them require adding files to the files menu, but their order is important depending on your goal:

# Action name Description
1 downloadTestResources Download zip of all files and unzip it in /resources
2 copyCustomHosts If there is a hosts file in /resources override system hosts file with it
3 downloadJMX Download the JMX from OctoPerf backend
4 installUserProperties Get the default user.properties file from OctoPerf backend
5 downloadJMeterPlugins Try to download Plugins from jmeter-plugins.org
6 configureKerberos Copy Kerberos configuration files from /resources to /bin
7 copyCustomJars Copy *.jar from /resources to /lib/ext
8 initJmxMonitoring Activate JMeter JVM internal monitoring
9 runUserScript "before-test" If it exists, execute the before-test.sh
10 processCsvFiles CSV file splitting process
11 rendezVous Rendez-vous until all the JMeter containers are ready to get started
12 startJMeter Execute allocated Threadgroup(s)
13 stopJMeter At the end of planned duration, send shutdown signal
14 uploadLogs Upload JMeter logs to OctoPerf backend
15 uploadJtls Upload JTLs to OctoPerf backend
16 runUserScript "after-test" If it exists, execute the after-test.sh

Detailed process

Download Resources

The very first step is to get the resources from the OctoPerf server. This is exactly the same service called by the export VU menu.

Once the zip file is downloaded, it will be unzipped under ${JMETER_HOME}/resources.

Note

Some of the files will be used in the next steps and may be removed or copied in another folder.

Custom hosts file

If the /resources folder contains a file named hosts we will copy that file over the operating system hosts file. This allow you to override the DNS automatically before the test even starts.

Warning

The load generator needs access to api.octoperf.com (or your own OctoPerf server), make sure not to override this DNS entry.

Download JMX

During this step, we download the JMX file from the OctoPerf server. The file is named scenario.jmx and will be copied to ${JMETER_HOME} folder.

Warning

Since it is the JMX that will be executed by JMeter later, tamper with it at your own risk.

User properties file

By default we provide our own user.properties file to ensure optimal JMeter settings for the test.

In some situations it can be interesting to edit or customize this property file for your own requirements. You can do this through a before-test.sh script, as you can see from this guide the changes will be applied at a later stage and thus will override the default file.

Download Plugins

By default, we try to download any plugin JAR file automatically from jmeter-plugins.org. To that end we use the plugins manager automatic detection. This is a good way to avoid storing the plugin JAR files in the project since that could affect the test start time.

You may get an error message when you execute the test from an on-premise agent that doesn't have access to internet:

ERROR: java.lang.RuntimeException: Failed to perform cmdline operation: jmeter-plugins.org: Name or service not known

As it only impacts the download of the plugin, you can instead provide the JAR file and disregard this message.

Alternatively if you're running your own OctoPerf EE server you can deactivate this step in the settings.

Kerberos

Kerberos authentication requires additional configuration files:

If these files are present, we will overwrite the default JVM files with them.

Custom Jars

JMeter allows you to use your own custom JAR files in the same way you can use plugins. Any JAR file provided through the files menu will be automatically copied to ${JMETER_HOME}/lib/ext so that JMeter loads it when it starts.

Note

If you disabled the Download plugins step and you are using a plugin, the plugin JAR files must be provided this way.

JMX Monitoring

We automatically monitor the load generators during the test. This steps prepares the required configuration so that we can capture the JVM internal metrics while the load test executes. This step is very important otherwise there's no way to assess if the JVM is overloaded during the test.

Info

The list of metrics is available on the agent monitoring page.

Before test script

The before-test.sh script offers a way to execute system commands before the test starts. If there's such a file present, we will execute it. This is a perfect way to tune the JMeter container before it gets started. There are many examples available on the files menu page.

The before-test.sh script is almost the last step before we start JMeter, which means it can override any previous change. But also for debugging purpose you can use it for instance to log the contents of another file to make sure it was properly copied.

After this step, the CSV will be split automatically according to our splitting mechanism, but here you can still override the CSV file with another one before that happens.

Csv Splitting

We will split files that correspond to CSV variables configured on "shared".

To ensure a set of unique values for each load generator, we give each one a unique number and the entire CSV file. We then proceed to do a modulo on the file on every load generator. After this, the load generator proceeds to pick the split that corresponds to its unique number.

RendezVous

This is a critical step because here we wait for all the load generators to be ready before sending the start signal. Otherwise their start time may be slightly off and the load policy would not execute as planned. In general it is just a matter of a few seconds but they can make all the difference when you want to reproduce advanced behavior.

Start JMeter

This is where the JMeter test actually starts and the test status moves to RUNNING.

We will first execute the setup thread group, then the main thread group and finish with the teardown. At this point, the UI redirects the OctoPerf user to the bench report page and starts displaying the live metrics.

On the load generator what happens is that our JMeter plugin sends the results to its OctoPerf server every 5 seconds by default. Results are sent over HTTPS, as described on the on-premise agent page.

Stop JMeter

When the test reaches FINISHED or ABORTED, we stop JMeter and proceed to the next step.

Info

If you're curious, we've written a guide on all the possible test ending events here.

Upload Logs and JTLs

Before we remove the docker container for good, we upload the relevant log files to the OctoPerf server. The JMeter logs are always uploaded, whereas the JTLs are optional. In the end they will be available in the report under the test logs menu.

After test script

And the last step before the docker container is completely removed is to execute the after-test.sh if there's one.

This is the perfect time to upload a local file to the OctoPerf server, in particular if you used this test to create a CSV file for another test. We even have a script example for this.

Warning

After this step there will be nothing left of the load generators or the test aside from the results stored in our database and the files that have been uploaded to the test logs. It is not possible to recover anything else afterwards. Make sure to save anything you need with your after test script.