Parallelisation Of Test Cases : Cypress Cloud and CI/CD GitHub actions

In today’s world where time equals money, it is essential to lower the execution time for various actions. When doing testing, the same holds true.

Parallelization of test cases is a useful technique to speed up the execution of test suites and reduce the overall testing time. Both Cypress Cloud and GitHub Actions support parallel execution of test cases, allowing you to run multiple tests simultaneously across multiple environments or machines.

In this blog we are going to cover:

1. How We Set up /Integration with Cypress Cloud
2. Cypress cloud GitHub actions Integration
3. Parallel execution In CI/CD GitHub Action
4. In the end, you will see the comparison report in time taken by test cases in execution in Cypress Cloud with different sets of machine

Let’s walk through Parallel Test Execution with Cypress first before learning how to run tests in parallel using Cypress.

This blog was originally published at https://qaautomationlabs.com/test-case-parallelly-in-cypress-cloud-and-ci-cd-github-actions/

For demo purposes, I am using Cypress Latest Version 12.1.0

What is Parallel Test Execution?

In parallel test execution, Several test cases are executed concurrently rather than sequentially. Parallel test execution has gained popularity as a result of the growing demand for cross-browser testing in the current industry. Testers can execute test cases simultaneously across numerous browsers thanks to it. As a result, the testers will save a lot of time and effort.

Parallel Test execution with Cypress

Cypress makes it simple to run tests concurrently by running your test cases on CI Tools. On the local device, the tester can also run the test cases concurrently. However, it is not advised because it will use up more resources and slow the CPU. Running it while configuring parallel execution on the CI tools is always advantageous.

What is Cypress Cloud?

Cypress Cloud is an enterprise-ready, web-based companion to the Cypress app. Using Cypress cloud we can online access the recorded test results, orchestrates test runs across multiple machines, provides rich analytics and diagnostics, and integrates those insights with your favorite tools.

Benefits of Cypress Cloud

Analyze and diagnose

Store the full history of your test results, with video clips, screenshots, and full stack traces. On the Latest Runs page, you can quickly examine the current condition of your program. Rich Analytics helps you spot troubling trends, and Flaky Test Management helps you discover unreliable tests. Associate related tests with grouping to see results broken down by browser and OS.

Run tests in parallel, in priority order, or not at all

With Smart Orchestration features, you can run tests across a swarm of machines simultaneously while Cypress Cloud coordinates runners and balances test loads — no setup required! You can prioritize recently failed specs to surface problems earlier and cancel whole test runs on failure to save on resource usage. You can also cancel in-progress runs manually from Cypress Cloud if you need to.

Integrate with source control providers

Keep failing code out of your GitHub, GitLab, and Bitbucket repositories with status checks that prevent commits from being merged until your Cypress tests are successful to ensure rock-solid reliability. Surface test results directly in your PRs with pull request comments that include test run statistics, specific failure details, and deep links to results in Cypress Cloud for fast debugging.

Collaborate and organize

Cypress can integrate with Slack and Jira, two of the most widely used collaboration systems. Deliver test results along with helpful supporting information straight to a specialized Slack channel. You may also immediately create bidirectionally linked Jira tickets from specific test failures if you have the Team, Business, or Enterprise plans.

Before Setup the Cypress Cloud we need to create Cypress Project that we can link with Cypress Cloud

Integration with Cypress Cloud

Before doing the setup of Cypress Cloud let's create Cypress Project first

Create Cypress Project

Pre-request

Cypress latest version is installed use the command “npm install cypress” to install the latest version

• Create a Cypress Project let’s say “cypress_github_Action”
• Create two .spec files to run the Test cases in Cypress Cloud

The Package.json file is attached below

Set up Cypress Cloud For Parallel Test Execution

Step 1

Open the URL https://cloud.cypress.io/login

Step 2

Log in with any of the above options

Step 3

In My case I am log-in with GitHub and Project CypressV12_GitHub is already created

Step 4

Once you set up your project to record, generated unique projectId for your project and automatically insert it into your Cypress configuration file.

Or We Can Copy/Paste Project Id from the below screen as well.

Once all the above steps are done integration with Cypress Cloud Is finished

What are GitHub Actions

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production.

GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository. For example, you can run a workflow to automatically add the appropriate labels whenever someone creates a new issue in your repository. GitHub provides Linux, Windows, and macOS virtual machines to run your workflows, or you can host your own self-hosted runners in your own data center or cloud infrastructure.

Set Up CI/CD GitHub Action For Parallel Test Execution

Step 1

Create a repository in my case let’s say “cypress_github_Action”.Initially, there is no code in created repo

Step 2

Push the code to the newly created repo. Below are the test cases that we are using to run in GitHub Action and Cypress Cloud

/// <reference types="cypress" />

describe("UI QAAutomationLabs.com", { testIsolation: false }, () => {
  it("Open URL", () => {
    cy.visit("https://qaautomationlabs.com/");
  });
  it("Click on Read More ", () => {
    cy.get(".staticslider-button").click();
  });
  it("Verify Particular Blog ", () => {
    cy.contains(
      "Running End-to-End Cypress Test cases In Google Cloud Build Pipeline"
    );
  });
  it("Click on Blogs", () => {
    cy.contains("Blog").scrollIntoView().click({ force: true });
  });
  it("Search the datas", () => {
    cy.get('[id="wp-block-search__input-2"]').scrollIntoView();
    cy.get('[id="wp-block-search__input-2"]')
      .click({ force: true })
      .type("cypress");
    cy.get('[id="search-icon"]').click({ force: true });
    cy.contains("Search Results for: cypress");
  });
});

describe("API QAAutomationLabs.com", { testIsolation: false }, () => {
  it("GET API testing Using Cypress API Plugin", () => {
    cy.request("GET", "https://reqres.in/api/users?page=2").should(
      (response) => {
        expect(response.status).to.eq(200);
      }
    );
  });

  it("POST API testing Using Cypress API Plugin", () => {
    cy.request("POST", "https://reqres.in/api/users", {
      name: "morpheus",
      job: "leader",
    }).should((response) => {
      expect(response.status).to.eq(201);
    });
  });

  it("PUT API testing Using Flip Plugin", () => {
    cy.request("PUT", "https://reqres.in/api/users/2", {
      name: "morpheus",
      job: "zion resident",
    }).should((response) => {
      expect(response.status).to.eq(200);
    });
  });

  it("DELETE API testing Using Cypress API Plugin", () => {
    cy.request("DELETE", "https://reqres.in/api/users/2").should((response) => {
      expect(response.status).to.eq(204);
    });
  });
});

Step 3

After pushing the code click on the “Actions” Tab in GitHub

Step 4

After clicking on Actions Tab go to the bottom and click on Configure under Automation -> Manual workflow Section (Highlighted below)

What is Workflow

A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file checked into your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule. Workflows are defined in the .github/workflows directory in a repository,

Step 5

Under project/.github/workflow/manual.yml created.

Update the .yml file for parallel execution of Test cases in GitHub Action

In the below .yml, we can see that we have passed record:true and parallel:true

So as we push the code. Test case start execute parallelly in two palace

1. In CI/CD GitHub action with provided container
2. Also, the Test case start executing in parallel in TWO machines in Cypress Cloud

Both the test case start parallelly in GitHub Action and Cypress Cloud

name: QAAutomationLabs.com Parallel Cypress Tests

on: [push]

jobs:
  test:
    name: Cypress Cloud Tests
    runs-on: ubuntu-latest
    container: cypress/browsers:node18.12.0-chrome107
    strategy:
      # when one test fails, DO NOT cancel the other
      # containers, because this will kill Cypress processes
      # leaving Cypress Cloud hanging ...
      # https://github.com/cypress-io/github-action/issues/48
      fail-fast: false
      matrix:
        # run 2 copies of the current job in parallel
        containers: [1, 2]
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      # because of "record" and "parallel" parameters
      # these containers will load balance all found tests among themselves
      - name: QAAutomationLabs.com Tests
        uses: cypress-io/github-action@v4
        with:
          record: true
          parallel: true
          group: 'QAAutomationLabs'
        env:
          # pass the Cypress Cloud record key as an environment variable
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # pass the GitHub token lets this action correctly
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

One Improtant setting that we have to do for to set the environment variable “CYPRESS_RECORD_KEY”

Open the particular repository in GitHub, In my case its cypress_github_Action

1. Click on Setting

3. Click on Secrets

4. We have to Add the “Repository secrets” by entering Name (e.g CYPRESS_RECORD_KEY)

Enter Secret Key “XXXXXXXXXXX” which can be copied from Cypress Cloud Setting screenshot attached below

Test Case Execution In GitHub and Cypress Cloud

As we push the code test cases to start executing automatically in both palace

GitHub

In GitHub as code is pushed both test cases to start executing parallelly at the same time. We can see in below screenshot Under Jobs

Checking the history of both jobs, we can see that qaautomationlabs_Flow1.cy.js has been executed on job 1 and qaautomationlabs_Flow2.cy.js has been executed on job 2.

Cypress Cloud

In Cypress Cloud, we can see that both test cases are executed successfully. In below screenshot, we can see the test results

Both the test cases are passed in the different machines with running in parallel.

The first test case ran in Machine 1 machine and passed successfully

The second test case ran in Machine 2 and passed successfully

Comparison of Time Taken by 5 Specs In Different Machine Sets

Pre-request

1. We should have 5 .spec file
2. We can update the number of container in manual.yml file by increase the count of container

e.g

In below screenshot we have 2 container

containers: [1, 2]

In below screenshot we have 3 container

containers: [1, 2,3]

We can increase container as per the requirement.

Below is the comparison when we execute the 5 .specs file with 24 test cases (it block) by executing in different machine size

1. When execute 5 . specs In 1 Machine

In the below screenshot, you can see 5 Spec with 1 Machine with 24 Tc time taken ~1:49S

2. When execute 5 . specs In 2 Machines

In the below screenshot, you can see out of 5 Spec (3 Spec run in Machine 1 and 2 Spec in Machine 2) with 24 Tc. time taken ~0.55s.

3. When executing 5 .specs In 3Machines

In the below screenshot, you can see out of 5 Spec (3 Spec run in Machine 1 and 2 Spec in Machine 2, and 1 Spec in Machine 3) with 24 Tc. time taken ~0.42s.

4. When executing 5 . specs In 4 Machine

In the below screenshot, you can see out of 5 Spec (2 Spec run in Machine 1 and 1 Spec in Machine 2, 1 Spec in Machine 3,1 Spec in Machine 4) with 24 Tc. time taken ~0.30s.

When executing 5 .spec In 5 Machine

In the below screenshot, you can see out of 5 Spec (1 Spec run in Machine 1 and 1 Spec in Machine 2, 1 Spec in Machine 3,1 Spec in Machine 4, and 1 Spec in Machine 5) with 24 Tc take ~0.27s.

Conclusion

In addition to integrating with GitHub Actions, Cypress also works with GitLab, and CircleCI, Cypress also offers parallelization and continuous testing.

Cypress’s use of parallel execution is highly advantageous because it speeds up the continuous testing process. It helps execute a lot of test cases in a short amount of time.

Reference

https://docs.cypress.io/