ElasTest has a TJob Execution comparator which allows you to see the differences between two or more executions at a glance.

Accessing Executions Comparator

To access the run comparator you must first navigate to the corresponding TJob section.

Once there you will have to select two or more finished executions from the table and finally press the Compare executions button.

Using the Executions Comparator

Once inside the execution comparator the following information will be displayed:

Executions to compare

A list with information about each execution will be displayed here. You will be able to click on each one of them to navigate to the page of that execution.

Test Cases Comparison

Here you can see a table with the results of each test case in each of the executions.

Monitoring Traces Comparison

On the other hand, you can also make a comparison of the metrics and logs of the executions. To do this, you can select the logs and/or metrics that you want to compare in the Monitoring Configuration window (by pressing the nut-button on the upper right-hand side) as you would on the screen of a tjob run. Once added they will be shown as follows:

Log Comparator

To compare logs you must first select the logs you want to compare:

Once added you will see the following card:

In this card there will be a tab for each log selected and if you select more than one there will be another tab called AIO that mixes all the logs selected.

In addition, within each of these tabs there will be one tab for each pair of selected executions (in this example we have only selected two, so there will only be one tab).

There are several possible filters that are divided into two categories:

• View:
  • Complete logs: compare all available logs. (Default)
  • Test Logs: compare only the logs of the test cases.
  • Failed Tests: compare only the logs of the failed test cases.
• Comparison:
  • Complete: displays the complete message as it is saved.
  • No timestamp: if the message contains timestamp, removes it. (Default)
  • Time diff: includes the time diff between traces in the logs comparison.

To start the check you should press the button Start Comparison:

The comparison will be shown:

If we select the Test Logs view:

Metrics Comparator

To compare metrics you must first select the metrics you want to compare:

Once added you will see the following card:

Advanced Metrics Configuration

You can configure your tests to send marks to ElasTest, so you can make better comparisons in the charts later. To send a mark to ElasTest, you must print a log line for each mark in your test with the following exact structure: ##elastest-monitor-mark: id=MARKID, value=MARKVALUE replacing MARKID with the common id of the mark and MARKVALUE for the value of the mark at that time. There will be a different chart display for each mark id.

In this test example several marks are sent with id=action, therefore, in the test results page we will have the option to visualize the charts as "action" Mark View or as Time View (default). By clicking on "action" Mark View button we will have the following view, in which each point of the chart corresponds to each mark sent: