ElasTest provides a simple way of execute same TJob with different configurations, compare several executions and improve visualization with matrices and graphs (Basic Orchestration).

For this it is necessary to activate the "multi" option in the TJob creation page:

A Multi-Configuration field is formed by a name and one or more values. Each name-value pair represents an environment variable with its name and its value.

Add new Multi-Configurations to a TJob when creating/editing it, typing the name of the configuration and adding as many values as you want to test:

Save the TJob and you can run it.

ElasTest provides an example TJob Multi-Configuration within the WebApp project, called Multi WebApp. This TJob has a single configuration named BROWSER with two values: chrome and firefox.

Therefore there will be two executions, one for each value, in which the corresponding browser will be used. During the execution of the Multi-Configuration tJob the following screen will be displayed, in which you can see information on the execution of each configuration (in this case two):

By clicking on each of them you can see the execution page of that configuration:

Once the executions are finished, the results screen will be displayed, in which you can view general information of all the executions presented in multiple ways: You can see it as a list or table of configurations executions and as a test table.

The charts will show a line for each configuration, to be able to compare quickly:

Advanced Configuration

You can configure your tests to send marks to ElasTest, so you can make better comparisons in the charts later. To send a mark to ElasTest, you must print a log line for each mark in your test with the following exact structure: ##elastest-monitor-mark: id=MARKID, value=MARKVALUE replacing MARKID with the common id of the mark and MARKVALUE for the value of the mark at that time. There will be a different chart display for each mark id.

In the TJob "Multi WebApp" test several marks are sent with id=action, therefore, in the test results page we will have the option to visualize the charts as "action" Mark View or as Time View (default). By clicking on "action" Mark View button we will have the following view, in which each point of the chart corresponds to each mark sent: