Skip to content

Execution Scenario

A scenario is a collection of user runtime profiles with settings for each. This section explains in depth how to configure and launch a scenario.

Scenario page

Scenarios allow you to easily compose a test with virtual users having:

  • Different behavior,
  • Different test locations (cloud and on premise),
  • Expected network conditions (like 3G or DSL).

User Runtime Profiles

Introduction

A scenario can contain one or more user runtime profiles, each of them having their own settings. The maximum number of profiles is defined by your subscription plan.

You can run multiple Virtual Users, using different configurations (geographical locations, user agents, bandwidth, etc.) at the same time by adding them to you scenario.

Add a Userload

There are several predefined user profiles available from the left menu.

The first User Agents gives you access to user profiles configured to use specific devices user agents and bandwidth:

  • Mobile,
  • Tablet,
  • Desktop users,
  • and even Smart TVs.

Predefined profiles

These profiles allow you to start simulating specific user agents very quickly. But, you can also define your own settings by editing the virtual user profile.

The second menu Virtual Users contains User Profiles pre-configured to run a specific Virtual User.

The following procedure explains how to add a user execution profile to your scenario:

  1. Hover the User Agents menu or the Virtual Users menu in the left sidebar,
  2. Drag'n drop the wanted profile under the scenario in the gray box.

Tip

You can also add a default "As recorded" user profile by clicking on the big + button bellow the list of profiles.

Select Userload

To select a Virtual User profile (a userload), simply click on it. A blue border appears on the selection: User profile

You can use the following keyboard shortcuts to interact with the selection:

Shortcut Button Description
del Removes the selected profile.
Selects the previous profile.
Selects the next profile.
ctrl + ↑ Moves the selected profile up.
ctrl + ↓ Moves the selected profile down.

The buttons are visible when you place your mouse hover a userload, on its top right corner.

Note

The 'Shift+?' shortcut shows / hides a list of available shortcuts.

Duplicate a Userload

The following procedure explains how to duplicate a user runtime profile in your scenario:

  1. Hover an existing user profile, below the scenario chart,
  2. The Duplicate button appears,
  3. Click on it,
  4. A new identical user profile is inserted.

Apply all

The Apply All button lets you apply one or more properties of a user profile to the whole scenario. For example you may want all your user execution profiles to have the same duration.

Apply All

The following procedure explains how to apply all settings of a user profile to your scenario:

  1. Hover an existing user profile, below the scenario chart,
  2. The Apply All button appears,
  3. Click on it,
  4. The properties selection dialog appears,
  5. Select which properties you want to apply to other user profiles,
  6. Click on the blue Apply All button at the bottom of the dialog box.

All the user profiles of your scenario now shared the same configuration.

Remove a Userload

To remove a userload from your scenario:

  1. Place your mouse over the userload you want to remove,
  2. Click on the Delete button appearing in the right top corner.

Custom Settings

Let's see the virtual user profile settings one by one:

Setting Description Example
Strategy Lets you define user load strategy (number of concurrent users and duration). 1000 vus, 1h
VU Defines the virtual user profile attached to this userload. Buying Customer
Location Geo location where the virtual users are running. It can be a Cloud or On-Premise location. EU West (Ireland)
Device Defines the virtual user's browser settings (like cookies and cache management) and the network simulation settings. As Recorded
DNS Domain Name System settings. Custom DNS
JTL JMeter Test Logs file generation settings. Elapsed Time, Latency, etc.
Memory Memory override settings. 8Mb

The following sections are describing the sub-settings for each of the setting above.

Strategy

The default user load strategy is a ramp-up followed by a peak of concurrent users. You can edit the maximum number of simulated users by editing the Maximum concurrent users field and change the test duration by updating the field Total duration.

Scale User load

Note

Changing these values will update all the steps. The load curve will remain identical but aligned with the new max duration and max concurrent user settings.

Strategy Curve Edition

You can also change the shape of the curve (to create multiple steps or add a starting delay) by:

Add a point

Add point

Note

You cannot drag a point before its predecessor or after its successor. But you place arbitrarily change its value by clicking on it.

Move a point

You can drag and drop points to quickly edit the user load curve.

Move point

Edit a point

When you click on a point, the Step edition panel is displayed:

Edit point

The current step position can be updated by changing the values of the Concurrent users and Time fields.

The button lets you go back to the user load scale configuration.

The Previous and Next button allows you to quickly navigate through the steps of the user load strategy.

The Delete button removes the current step. The first step (at time 0) cannot be removed. You curve must also have at least two points.

Tip

Press CTRL and click on a point to quickly remove the associated step.

Advanced configuration

User load advanced configuration

Several options are available by clicking on the Advanced Configuration button:

On Sample Error policy

The error policy defines what to do when an error occurs (i.e. an assertion failed):

  • Continue: the error is ignored and the test goes on.
  • Start next VU Iteration: the current Virtual User execution is stopped. And the test starts with a fresh VU.
  • Stop VU: The current Virtual User execution is completely stopped.
  • Stop Test: The all load test is stopped gracefully.
  • Stop Test Now: The all load test is stopped immediately.
Iterations

The iterations field lets you limit the Virtual Users execution to a number of iterations.

You still need to set up a duration for the User Profile, so make sure that the execution of all iterations wil not take more time than the specified duration.

Think times

You can also override think times for all actions of the selected Virtual User in this tab:

Click on the Override all HTTP Request actions think times toggle to activate the think times override.

Then you can switch between 3 different modes by clicking on the left green arrow button:

  • Constant: All think times have the same value.

Think Time Constant

  • Range: Think times vary between a lower bound and an upper bound.

Think Time Range

  • Hits: Think times are set in order to execute a certain number of requests (hits) per second.

Think Time Hits

  • Text: All think times have the same value defined in milliseconds.

Think Time Text

Location

User Profile Location

The location where the virtual user profile instances are being executed. You can select the Provider/Region using both the select box and the the map.

Note

The GPS coordinates of on-premise regions are configured on the private hosts page. You must accept to be geo-localized by OctoPerf's application if you want your location to be displayed on the map.

Europe, USA and more are available by default using our Amazon and Digital Ocean public Cloud providers.

AWS regions:

  • Canada (Montreal),
  • US East (Virginia),
  • US West (North California),
  • US West (Oregon),
  • EU West (Ireland),
  • EU West (London),
  • EU Central (Frankfurt),
  • South America (Sao Paulo),
  • Asia Pacific (Singapore),
  • Australia (Sydney),
  • Japan (Tokyo),
  • South Korea (Seoul),
  • India (Mumbai),
  • US East (Ohio).

DO regions:

  • Canada (Toronto),
  • US (New York),
  • US (San Francisco),
  • EU (Amsterdam),
  • EU (Frankfurt),
  • EU (London),
  • Asia Pacific (Singapore),
  • India (Bangalore).

Device

User Profile Device

The device settings allow to define the behavior of the virtual user profile simulated browser:

  • Device: select the user agent to simulate. Example: Chrome on Galaxy Tab,
  • Clear Cookies on each iteration: cleans the browser cookies each time a virtual user instance restarts from the beginning,
  • Cache size: defines the maximum number of elements the browser cache should hold,
  • Clear cache on each iteration: like cookies, clear cache when starting a new iteration,
  • Use HTTP cache control headers: whenever to take into account HTTP headers like Expires or Cache-Control.

These are advanced settings which should be handled with care.

Note

We recommend to leave the default settings.

Bandwidth

Defines the network simulation. Network simulation allows to simulate similar connection conditions as a real user would experience. The settings are:

  • Bandwidth: defines the upload and download bandwidth.

Info

The bandwidth setting apply to each virtual user instance.

DNS

User Profile DNS

Defines the DNS (Domain Name System) settings. The settings are:

  • Clear dns cache on each iteration: when true, the virtual user resolves the server IP behind the domain on each iteration,
  • Custom servers: when any defined, overrides the system default DNS server.
  • Static Hosts: to override IPs (right column) associated to host names (left column).

When load testing a website behind a CDN (Content Delivery Network) or several load balancers (like AWS Elastic Load Balancer), we recommend to clear dns cache on each iteration.

Info

If you need to override IPs associated to host names you can also upload a hosts file in the files manager

JTL

User Profile JTL

Defines the JMeter Test Log file generation settings. The settings are:

  • Export JTL files?: Toggle ON/OFF the JTL file generation,
  • Results Log Level: Log all results, only errors or only successes,
  • Save Settings: defines what information must be saved in the generated JTL files.

Note

JTL files are available when the test ends in the result logs panel.

JTL files can be used for reporting in the Jenkins CI integration.

Memory

User Profile Memory

Defines the Memory settings for this user profile. This setting allows you to use more or less memory than the default 8Mb to run each instance of the selected virtual user.

It may be useful in order to execute particularly complex Virtual Users.

Warning

Modifying these settings may cause your load tests to crash. Please contact our support if you think you need to change it.

WebDriver UserLoad

User Profile WebDriver

When selecting a Webdriver virtual user, the number of concurrent users is limited for performance reasons. Real-browser simulation consumes a lot more resources than regular JMeter load tests.

Launching a test

To launch a test, click on the Launch xx VUs button in the top right corner. The launch test wizard appears:

Virtual Users validation

The first step is to validate that none of your Virtual Users contains critical errors. Such errors, like a missing CSV variable file, would prevent the test from running.

You need to fix them (if any error) before continuing to the next step.

Monitoring connections validation

The second step is the monitoring connections validation. If you configured monitoring connections, OctoPerf automatically check every enabled connection before launching the test.

Subscription validation

The final step allows to select the subscription to use. Click on the Launch button, the test startup procedure is shown within seconds.

Please allow up to a few minutes for the test to start. The system may need to start machines before it can run your test. A Bench report is created when launching a test. You can always go back to your test by clicking on the corresponding bench report on the project homepage.