How to reduce flakiness/Save the time of Test case execution using the “test retries” feature of Cypress

Problem Statement

In cypress before the “5.0.0” version, we don’t have the option to retries the failed test cases. If test cases are failed due to (e.g., temporary outages, API response delay, random network errors, etc.) and due to the flaky nature of test cases in such cases we have to run the test cases multiple times and which is time-consuming and very painful.

Solution

Cypress.io team introduces “test retries” in version 5.0.0 which helps us to save our time for running the failed test cases multiple times.

Benefits

With test retries, Cypress is able to retry failed tests to help reduce test flakiness and continuous integration (CI) build failures. By doing so, will save the team valuable time.

E.g There is scenario where we are verifying after doing some action certain text/field should be display but due to network issue,loading issue ,API delay responce, data is not loading on time.After introducing test retries option cypress will re-run the test cases till as per the given retry attempt in cypress.json.

How It Works

By default, tests will not retry when they fail. We will need to enable test retries in our configuration to use this feature.

Once test retries are enabled, tests can be configured to have an X number of retry attempts. For example, if test retries have been configured with 2 retry attempts, Cypress will retry tests up to 2 additional times (for a total of 3 attempts) before potentially being marked as a failed test.

Configure Test Retries

1. Global configuration
2. Custom configuration

1.Global configuration: We can configure this in configuration file cypress.json bypassing the retries option an object with the following options:

• runMode allows you to define the number of test retries when running cypress run
• openMode allows you to define the number of test retries when running cypress open

In the above case, the cypress will retry 2 times if required data is not found and the test case Is failed.

If the test case passed in 1st-time cypress moves to other test cases. if the test case is failed will try 1st retry Attempt

If 1st retry attempt failed will try 2nd retry option.

If 2nd retry attempt failed Cypress will mark the test as failed and then move on to run any remaining tests.

So total 3 attempts (*One when test case run the first time and other 2 attempts for retry )

2.Custom configuration:

In Custom configuration we can configure in Individual Test Level and Test Suite Level.

Individual Test Level (*In the It block)

Test Suite Level (In describe block)

Conclusion

Cypress test retries option really helpful when we have flakiness test cases.

Cypress test retries option saves time when we have to re-run the failed test case again and again.