Skip to content

Elasticsearch

Live metrics can be sent to an external Elasticsearch while your scenario is running.

image

Each metric corresponds to a JMeter Sample which will be sent as a JSON document to Elasticsearch. The fields of this JSON document are configurable with:

  • Standard fields: a list of 35 fields are available
  • Custom fields: you can add your own fields as key/value pairs

Example of metric sent to Elasticsearch:

{
  "ScenarioName" : "Queen Parrotfish",
  "SampleLabel" : "request_main_page",
  "URL" : "https://octoperf.com/",
  "ThreadName" : "01 - ThreadGroup 1-1",
  "ContentType" : "text/html; charset=utf-8",
  "AllThreads" : 1,
  "TestStartTime" : 1655972857482,
  "SampleStartTime" : "2022-06-23T10:27:38.549+0200",
  "SampleEndTime" : "2022-06-23T10:27:38.770+0200",
  "SentBytes" : 167,
  ...
  "ResponseTime" : 221,
  "Bytes" : 9959,
  "BodySize" : 9460,
  "Latency" : 221,
  "InjectorId" : "turquoise_donovan_howell"
  "InjectorHostname" : "legion",
  "LocationName" : "default",
  "VirtualUserName" : "octoperf.com",
  "ElapsedTime" : 1642892401000,
  "Timestamp" : 1655972858549,
  "ErrorCount" : 0
}

Requirements

  • A running Elasticsearch accessible from the OctoPerf agent where the performance test will be executed:
  • If you are running the test on our cloud agents your Elasticsearch must be accessible publicly or through a dedicated IP,
  • If you are running the test from an on premise agent, your Elasticsearch must be accessible from this agent.

Octoperf configuration

Connection

All fields are pre-filled with default values. In most cases, you only need to fill hostname and username / password. (if Basic Authentication is required). Example with Authentication enabled for an on premise Elasticsearch:

configure-connection

Name Description
Scheme HTTP or HTTPS. In case your Elasticsearch is accessible via HTTPS and its certificate is self-signed, see SSL verification parameter below.
Hostname The hostname or IP address of your Elasticsearch.
Port The port of your Elasticsearch (Default port is 9200).
Index The Elasticsearch index where data will be pushed. The index is supposed to have been correctly configured (mapping of fields to be queryable). Note that if the index doesn't exist, it will be automatically created (without any mapping) when the test starts.
Authentication Enable Basic authentication. This is when your Elasticsearch is configured with HTTP basic authentication headers.

Data sending

This section let's customize what data has to be sent to Elasticsearch. Example with default data sent and 2 custom fields added:

data-sent

Name Description
Verbosity This parameter defines which sample results will be sent to Elasticsearch during the test. Default level is INFO. Check the documentation for more details.
Standard fields There are 35 fields available with the default verbosity. OctoPerf adds the following on top of what the JMeter plugin provides: ScenarioName, LocationName, InjectorId, VirtualUserName.
Custom fields Allows to specify custom data to be sent to Elasticsearch as Key/Value pairs.

Example

Let's say we have configured 2 custom fields like this:

custom-fields

In Elasticsearch we will have these fields:

{
  ...
  "CustomFieldKey1" : "CustomFieldValue1",
  "CustomFieldKey2" : "CustomFieldValue2",
 ...
}

Advanced configuration

This section is composed of advanced options regarding data sent customization: Filtering Samples by label, Send All Request or Response headers as fields or even changing some formatted date fields format.

Example with default configuration:

data-sent

Name Description
Bulk size Data is sent to Elasticsearch index through groups of JSON documents. This setting defines the number of JSON documents being sent at once. Default value is 100 and should be fine in most cases. You can increase this value if the latency/bandwidth between the location (injector) and your Elasticsearch is really good.
Socket timeout in ms The time waiting for data – after establishing the connection; maximum time of inactivity between two data packets. Default value is 200 ms. You can increase this value if the network has degraded performance.

Samples filter

Allows filtering of specific Samples only (filtering based on Sample labels).

actions

Let's say we have these actions in our Virtual User and we only want to have results for:

  • 01 - Container
  • request_jmeter_page

With a filter like this: 01;jmeter, only Samples having a label containing 01 or jmeter will be sent to Elasticsearch.

Timestamp format

The date format to be used. This is based on Java Simple Date Format.

Headers sent as fields

This option sends Request/Response headers as fields to Elasticsearch. For example in our VU we have this GET request which has 2 headers: headers

If we activate Request headers sent as fields, in Elasticsearch we will have these fields:

{
  ...
  "headerKey1" : "headerValue1",
  "headerKey2" : "headerValue2",
 ...
}

Elasticsearch index

To make fields pushed to your Elasticsearch searchable, the index mapping must be configured.

Example using Kibana dev console:

kibana

Depending on the standard/custom fields you have selected, configure your index mapping for the fields you want to make queryable.

Below a mapping configuration with all standard fields:

{
  "mappings": {
    "properties": {
      "AllThreads": {
        "type": "long"
      },
      "BodySize": {
        "type": "long"
      },
      "Bytes": {
        "type": "long"
      },
      "ConnectTime": {
        "type": "long"
      },
      "ContentType": {
        "type": "text"
      },
      "DataType": {
        "type": "text"
      },
      "ElapsedTime": {
        "type": "date"
      },
      "ErrorCount": {
        "type": "long"
      },
      "GrpThreads": {
        "type": "long"
      },
      "IdleTime": {
        "type": "long"
      },
       "InjectorHostname": {
        "type": "keyword"
      },
      "Latency": {
        "type": "long"
      },
      "ResponseTime": {
        "type": "long"
      },
      "SampleCount": {
        "type": "long"
      },
      "SampleLabel": {
        "type": "keyword"
      },
      "ThreadName": {
        "type": "keyword"
      },
      "ResponseCode": {
        "type": "text"
      },
      "TestStartTime": {
        "type": "date"
      },
      "SampleStartTime": {
        "type": "date",
        "format": "date_optional_time"
      },
      "SampleEndTime": {
        "type": "date",
        "format": "date_optional_time"
      },
      "SentBytes": {
        "type": "long"
      },
      "Timestamp": {
        "type": "date"
      },
      "URL": {
        "type": "keyword"
      },
      "AssertionResults": {
        "properties": {
          "failure": {
            "type": "boolean"
          },
          "failureMessage": {
            "type": "text"
          },
          "name": {
            "type": "text"
          }
        }
      },
      "FailureMessage": {
        "type": "text"
      },
      "Success": {
        "type": "boolean"
      },
      "RequestHeaders": {
        "type": "text"
      },
      "RequestBody": {
        "type": "text"
      },
      "ResponseHeaders": {
        "type": "text"
      },
      "ResponseBody": {
        "type": "text"
      },
      "ResponseMessage": {
        "type": "text"
      },
      "ScenarioName": {
        "type": "keyword"
      },
      "LocationName": {
        "type": "keyword"
      },
      "InjectorId": {
        "type": "keyword"
      },
      "VirtualUserName": {
        "type": "keyword"
      }
    }
  }
}

In the example of Data sent section, there are 2 custom fields. If we want them to be searchable:

"CustomFieldKey1": {
  "type": "keyword"
},
"CustomFieldKey2": {
  "type": "keyword"
}