Skip to content

Data Persistence

OctoPerf Enterprise-Edition (EE) entirely runs as Docker containers. Containers are inherently volatile: they can be created and destroyed easily. How can we persist data in this context, given the containers may be destroyed and re-created any time? Docker volumes. OctoPerf relies on Docker to let you manage how you want to persist your data.

Beware that any change in data persistence settings may delete your previously existing data. Always make a backup of your data before any manipulation.

Datas

The datas which need to be persisted are:

  • Elasticsearch: Elasticsearch stores the virtual users, scenarios and test results within the Elasticsearch container on path /usr/share/elasticsearch/data,
  • OctoPerf Resources: This includes recorded requests and responses, validation requests and responses, JMeter logs and JMeter JTLs. This is saved inside OctoPerf Enterprise-Edition container inside /root/data.

By default, when using OctoPerf-Catalog to setup the Enterprise-Edition, Named volumes are configured:

  • elasticsearch-data: contains the whole database including users, scenarios, test results and more,
  • octoperf-data: contains the license, the backend configuration file, the recorded requests and responses and JMeter log files.

Volumes Mapping

Configuring volume mapping

  • Open Rancher Web UI,
  • Go to Stacks,
  • Click on Upgrade button on the relevant service, Example: Elasticsearch,
  • Click on Volumes Tab,
  • Enter the appropriate configuration,
  • Click on Upgrade.

There are many different ways to persist data using Docker container. Some of them are exposed below. Please see the Docker documentation for more information.

Persistence Strategies

Container

This strategy is only suitable for testing purpose. All the data is wiped if a container is upgraded or re-created.

In this case, the data is saved inside the docker container itself. There is no real advantage to use this method. The major drawback is OctoPerf EE upgrade may entirely wipe off all your data.

Named Volumes

When setting up OctoPerf Enterprise-Edition using our Rancher Catalog, data is persisted using Local Docker volumes. When upgrading Elasticsearch or OctoPerf itself, the data is kept safe in a separate volume on the local machine.

This strategy requires no configuration and can be easily setup.

Named Volumes Mapping Mapping a named volume to container directories

However, the major drawback is it's difficult to move the data from one physical machine to another.

Local Directory

Local directory volume mapping allows to map a local directory on your computer to a directory inside a docker container. For example, a Host Directory (Mount a host directory as a data volume) can be mounted directly as a volume. In this case, a local directory on your machine is mapped to the directory inside the container.

This setup makes it easy to move data from one physical machine to another because the data is accessible from the host machine.

Data Storage Settings

Rancher Install OctoPerf OctoPerf Installation Settings.

Data Storage Settings:

  • OctoPerf Data Location: octoperf-data local volume by default. Defines the volume where backend stores data like JMeter Logs and recorded responses. Example: /home/ubuntu/octoperf/data, which is the local path on the machine running the containers.
  • ElasticSearch Data Location: elasticsearch-data local volume by default. Defines the volume where backend stores data like JMeter Logs and recorded responses. Example: /home/ubuntu/octoperf/elasticsearch. Make sure to chmod -R 777 (recursively) the folder otherwise Elasticsearch won't be able to write in it.

Warning: any change in Storage Settings may result in data loss.

Backend Configuration Settings:

  • OctoPerf License Location: octoperf-data local volume by default. Defines the volume mapping where the backend looks up the license.l4j file. Example: /home/ubuntu/octoperf/license, which is the local path on the machine running the containers.
  • OctoPerf Config Location: octoperf-data local volume by default. Defines the volume mapping where the backend looks up the application.yml config file. Example: /home/ubuntu/octoperf/config, which is the local path on the machine running the containers.

Customizing Storage Settings

octoperf-data is great to try OctoPerf EE without messing with Docker Volume Mapping.

You can list the existing Docker volumes by running docker volume ls

local               enterprise-edition_elasticsearch-data_9a8a2
local               enterprise-edition_octoperf-data_842b2

You can also inspect a single Docker Volume by name:

ubuntu@desktop:~$ docker inspect volume enterprise-edition_elasticsearch-data_9a8a2
[
    {
        "CreatedAt": "2017-10-19T11:01:12+02:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/enterprise-edition_elasticsearch-data_9a8a2/_data",
        "Name": "enterprise-edition_elasticsearch-data_9a8a2",
        "Options": {},
        "Scope": "local"
    }
]
Error: No such object: volume

It displays where the volume has been stored along with other information.

You can configure OctoPerf EE to use a specific path to store the datas, instead of the default octoperf-data volume. You can specify the absolute path on the docker host machine, for Example /home/ubuntu/octoperf/elasticsearch for Elasticsearch Data Location.

Use Local Docker Host Path

  • SSH on the Docker host running OctoPerf EE,
  • Create the folder in the wanted location, for Example /home/ubuntu/octoperf/elasticsearch,
  • chmod -R 777: recursively allow write permission on the folder for anyone,
  • Specify the absolute path during OctoPerf EE configuration in the wanted setting. Example: Elasticsearch Data Location.

If you already had data stored in octoperf-data volume:

  • Stop All OctoPerf EE Services,
  • SSH onto the Docker host,
  • List all volumes by running docker volume ls. There should be at least enterprise-edition_elasticsearch-data_xxxx and enterprise-edition_octoperf-data_xxxx volumes,
  • Inspect the volume containing the existing data. Example: docker inspect volume enterprise-edition_elasticsearch-data_9a8a2,
  • Copy recursively the files and folders from Mountpoint folder to the new target folder by running:

cp -rp MOUNT_POINT NEW_PATH, where MOUNT_POINT is the existing volume folder and NEW_PATH the absolute path to the new folder.

  • Open Rancher UI,
  • Upgrade OctoPerf EE Stack (even if already up-to-date) to change the volume settings, Example Elasticsearch Data Location changed from octoperf-data to /home/ubuntu/octoperf/elasticsearch,
  • Start OctoPerf EE.

Your data should still be there, but now it's being stored in another location on the Docker Host.