Integration with Azure DevOps

Instructions

You can use the Endtest API and a Shell Script task to integrate Endtest with Azure DevOps.

YAML Snippet

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: Bash@3
  inputs:
    filePath: 'test.sh'
    arguments: 'app-id app-code "api-request-to-start-test-execution" number-of-loops'

Create test.sh at the root of your repo. We recommend creating this file from a Linux environment (such as a real Linux machine or Windows Subsystem for Linux) so that line endings are correct. Also, don’t forget to chmod +x test.sh before you commit it. test.sh

#!/bin/bash
hash=$(curl -X GET --header "Accept: */*" "${3}")
for run in {1.."${4}"}
do
  sleep 30
  result=$(curl -X GET --header "Accept: */*" "https://app.endtest.io/api.php?action=getResults&appId=${1}&appCode=${2}&hash=${hash}&format=json")
  if [ "$result" == "Test is still running." ]
  then
    status=$result
    # Don't print anything
  elif [ "$result" == "Processing video recording." ]
  then
    status=$result
    # Don't print anything
  elif [ "$result" == "Stopping." ]
  then
    status=$result
  elif [ "$result" == "Erred." ]
  then
    status=$result
    echo $status
  elif [ "$result" == "" ]
  then
    status=$result
    # Don't print anything
  else
     echo "$result" | jq
     exit 0
  fi
done
exit

This sample script would require the following arguments:

The App ID for your account.
The App Code for your account.
The API Request for starting the test execution.
The number of times the API request for fetching the results will be sent once every 30 seconds.

For example, if you know that your test execution usually takes 3 minutes, you should use the value 7. 7 x 30 seconds = 210 seconds > 3 minutes Don’t forget to make your Shell script executable by running the following command:

sudo chmod +x test.sh

And you would also need to install the jq package:

sudo apt-get install jq

The command for running the test.sh script locally would have the following format:

./test.sh app-id app-code api-request-to-start-test-execution number-of-loops

Let’s pretend we have the following values:

App ID = 44233559

App Code = 22381137

Endtest API Request to start test execution = "https://app.endtest.io/api.php?action=runWeb&appId=44233559&appCode=22381137&suite=380667&platform=windows&os=windows10&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes="

Number of loops = 10

The command for running the test.sh script locally would look like this:

./test.sh 44233559 22381137 "https://app.endtest.io/api.php?action=runWeb&appId=44233559&appCode=22381137&suite=380667&platform=windows&os=windows10&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=" 10

And the YAML file in Azure DevOps would look like this:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: Bash@3
  inputs:
    filePath: 'test.sh'
    arguments: '44233559 22381137 "https://app.endtest.io/api.php?action=runWeb&appId=44233559&appCode=22381137&suite=380667&platform=windows&os=windows10&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=" 10'

Since you will be integrating Endtest into your current Azure DevOps Pipeline, your actual YAML file will also contain existing instructions that are not related to Endtest. And it will also contain instructions to read the JSON output from the Endtest API response in order to execute a certain logic in the pipeline (such as marking a build as Passed or Failed).

The JSON with the results contains the following keys:

test_suite_name - The name of the test suite.
configuration - The configuration of the machine or mobile device on which the test was executed.
test_cases - The number of test cases.
passed - The number of assertions that have passed.
failed - The number of assertions that have failed.
errors - The number of errors that have been encountered.
start_time - The timestamp for the start of the test execution.
end_time - The timestamp for the end of the test execution.
detailed_logs - The detailed logs for the test execution.
screenshots_and_video - The URLs for the screenshots and the video recording of the test execution.
test_case_management - The name, status and external IDs for each test case.

You can also use the value of the hash variable to generate the link to the Results page for that test execution:

results=https://app.endtest.io/results?hash="$hash"

More details about our API are available in the Endtest API chapter.

Documentation

​Instructions

Instructions