A scenario is a collection of user runtime profiles with settings for each. This section explains in depth how to configure and launch a scenario.
Scenarios allow you to easily compose a test with virtual users having:
- Different behavior,
- Different test locations (cloud and on premise),
- Expected network conditions (like 3G or DSL).
User Runtime Profiles¶
A scenario can contain one or more user runtime profiles, each of them having their own settings. The maximum number of profiles is defined by your subscription plan.
You can run multiple Virtual Users, using different configurations (geographical locations, user agents, bandwidth, etc.) at the same time by adding them to you scenario.
Add a Userload¶
There are several predefined user profiles available from the left menu.
The first User Agents gives you access to user profiles configured to use specific devices user agents and bandwidth:
- Desktop users,
- and even Smart TVs.
These profiles allow you to start simulating specific user agents very quickly. But, you can also define your own settings by editing the virtual user profile.
The second menu Virtual Users contains User Profiles pre-configured to run a specific Virtual User.
The following procedure explains how to add a user execution profile to your scenario:
- Hover the User Agents menu or the Virtual Users menu in the left sidebar,
- Drag'n drop the wanted profile under the scenario in the gray box.
You can also add a default "As recorded" user profile by clicking on the big + button bellow the list of profiles.
To select a Virtual User profile (a userload), simply click on it. A blue border appears on the selection:
You can use the following keyboard shortcuts to interact with the selection:
|del||Removes the selected profile.|
|↑||Selects the previous profile.|
|↓||Selects the next profile.|
|ctrl + ↑||Moves the selected profile up.|
|ctrl + ↓||Moves the selected profile down.|
The buttons are visible when you place your mouse hover a userload, on its top right corner.
The 'Shift+?' shortcut shows / hides a list of available shortcuts.
Duplicate a Userload¶
The following procedure explains how to duplicate a user runtime profile in your scenario:
- Hover an existing user profile, below the scenario chart,
- The Duplicate button appears,
- Click on it,
- A new identical user profile is inserted.
The Apply All button lets you apply one or more properties of a user profile to the whole scenario. For example you may want all your user execution profiles to have the same duration.
The following procedure explains how to apply all settings of a user profile to your scenario:
- Hover an existing user profile, below the scenario chart,
- The Apply All button appears,
- Click on it,
- The properties selection dialog appears,
- Select which properties you want to apply to other user profiles,
- Click on the blue Apply All button at the bottom of the dialog box.
All the user profiles of your scenario now shared the same configuration.
Remove a Userload¶
To remove a userload from your scenario:
- Place your mouse over the userload you want to remove,
- Click on the Delete button appearing in the right top corner.
Let's see the virtual user profile settings one by one:
|Strategy||Lets you define user load strategy (number of concurrent users and duration).||1000 vus, 1h|
|VU||Defines the virtual user profile attached to this userload.||Buying Customer|
|Location||Geo location where the virtual users are running. It can be a Cloud or On-Premise location.||EU West (Ireland)|
|Device||Defines the virtual user's browser settings (like cookies and cache management) and the network simulation settings.||As Recorded|
|DNS||Domain Name System settings.||Custom DNS|
|JTL||JMeter Test Logs file generation settings.||Elapsed Time, Latency, etc.|
|Memory||Memory override settings.||8Mb|
The following sections are describing the sub-settings for each of the setting above.
The default user load strategy is a ramp-up followed by a peak of concurrent users. You can edit the maximum number of simulated users by editing the Maximum concurrent users field and change the test duration by updating the field Total duration.
Changing these values will update all the steps. The load curve will remain identical but aligned with the new max duration and max concurrent user settings.
Strategy Curve Edition¶
You can also change the shape of the curve (to create multiple steps or add a starting delay) by:
- Clicking on the graph to add a point/step,
- Dragging and dropping a point across the chart,
- Clicking on a point to edit its values.
Add a point¶
You cannot drag a point before its predecessor or after its successor. But you place arbitrarily change its value by clicking on it.
Move a point¶
You can drag and drop points to quickly edit the user load curve.
Edit a point¶
When you click on a point, the Step edition panel is displayed:
The current step position can be updated by changing the values of the Concurrent users and Time fields.
The button lets you go back to the user load scale configuration.
The Previous and Next button allows you to quickly navigate through the steps of the user load strategy.
The Delete button removes the current step. The first step (at time 0) cannot be removed. You curve must also have at least two points.
Press CTRL and click on a point to quickly remove the associated step.
Several options are available by clicking on the Advanced Configuration button:
On Sample Error policy¶
The error policy defines what to do when an error occurs (i.e. an assertion failed):
- Continue: the error is ignored and the test goes on.
- Start next VU Iteration: the current Virtual User execution is stopped. And the test starts with a fresh VU.
- Stop VU: The current Virtual User execution is completely stopped.
- Stop Test: The all load test is stopped gracefully.
- Stop Test Now: The all load test is stopped immediately.
The iterations field lets you limit the Virtual Users execution to a number of iterations.
You still need to set up a duration for the User Profile, so make sure that the execution of all iterations wil not take more time than the specified duration.
You can also override think times for all actions of the selected Virtual User in this tab:
Click on the Override all HTTP Request actions think times toggle to activate the think times override.
Then you can switch between 3 different modes by clicking on the left green arrow button:
- Constant: All think times have the same value.
- Range: Think times vary between a lower bound and an upper bound.
- Hits: Think times are set in order to execute a certain number of requests (hits) per second.
- Text: All think times have the same value defined in milliseconds.
The location where the virtual user profile instances are being executed. You can select the Provider/Region using both the select box and the the map.
The GPS coordinates of on-premise regions are configured on the private hosts page. You must accept to be geo-localized by OctoPerf's application if you want your location to be displayed on the map.
Europe, USA and more are available by default using our Amazon and Digital Ocean public Cloud providers.
- Canada (Montreal),
- US East (Virginia),
- US West (North California),
- US West (Oregon),
- EU West (Ireland),
- EU West (London),
- EU Central (Frankfurt),
- South America (Sao Paulo),
- Asia Pacific (Singapore),
- Australia (Sydney),
- Japan (Tokyo),
- South Korea (Seoul),
- India (Mumbai),
- US East (Ohio).
- Canada (Toronto),
- US (New York),
- US (San Francisco),
- EU (Amsterdam),
- EU (Frankfurt),
- EU (London),
- Asia Pacific (Singapore),
- India (Bangalore).
The device settings allow to define the behavior of the virtual user profile simulated browser:
- Device: select the user agent to simulate. Example: Chrome on Galaxy Tab,
- Clear Cookies on each iteration: cleans the browser cookies each time a virtual user instance restarts from the beginning,
- Cache size: defines the maximum number of elements the browser cache should hold,
- Clear cache on each iteration: like cookies, clear cache when starting a new iteration,
- Use HTTP cache control headers: whenever to take into account HTTP headers like Expires or Cache-Control.
These are advanced settings which should be handled with care.
We recommend to leave the default settings.
Defines the network simulation. Network simulation allows to simulate similar connection conditions as a real user would experience. The settings are:
- Bandwidth: defines the upload and download bandwidth.
The bandwidth setting apply to each virtual user instance.
Defines the DNS (Domain Name System) settings. The settings are:
- Clear dns cache on each iteration: when true, the virtual user resolves the server IP behind the domain on each iteration,
- Custom servers: when any defined, overrides the system default DNS server.
- Static Hosts: to override IPs (right column) associated to host names (left column).
If you need to override IPs associated to host names you can also upload a
hosts file in the files manager
Defines the JMeter Test Log file generation settings. The settings are:
- Export JTL files?: Toggle ON/OFF the JTL file generation,
- Results Log Level: Log all results, only errors or only successes,
- Save Settings: defines what information must be saved in the generated JTL files.
JTL files are available when the test ends in the result logs panel.
JTL files can be used for reporting in the Jenkins CI integration.
Defines the Memory settings for this user profile. This setting allows you to use more or less memory than the default 8Mb to run each instance of the selected virtual user.
It may be useful in order to execute particularly complex Virtual Users.
Modifying these settings may cause your load tests to crash. Please contact our support if you think you need to change it.
When selecting a Webdriver virtual user, the number of concurrent users is limited for performance reasons. Real-browser simulation consumes a lot more resources than regular JMeter load tests.
Launching a test¶
To launch a test, click on the Launch xx VUs button in the top right corner. The launch test wizard appears:
The first step is to validate that none of your Virtual Users contains critical errors. Such errors, like a missing CSV variable file, would prevent the test from running.
You need to fix them (if any error) before continuing to the next step.
The second step is the monitoring connections validation. If you configured monitoring connections, OctoPerf automatically check every enabled connection before launching the test.
The final step allows to select the subscription to use. Click on the Launch button, the test startup procedure is shown within seconds.
Please allow up to a few minutes for the test to start. The system may need to start machines before it can run your test. A Bench report is created when launching a test. You can always go back to your test by clicking on the corresponding bench report on the project homepage.