Revolutionizing Microsoft Playwright Testing with the Power of AI

Software testing industry is open and constantly developing, so the speed, reliability, and expandability play a significant role in software testing to help provide applications of the highest quality. While developers and QA engineers work towards achieving these goals, AI integrated into automation tools has become the new frontier. Microsoft Playwright is already a well respected and powerful end-to-end testing tool widely recognized for its flexibility and speed.

The combination of Playwright commands and ZeroStep — an AI-based testing library brings a powerful synergy to test automation, allowing teams to streamline workflows and enhance productivity.

This blog seeks to discuss how ZeroStep puts AI into Microsoft Playwright to transform test automation and why it has become a necessity among testing teams today.Also explain the limitation of using AI based tool

My New Book On Playwright

Web Automation Testing Using Playwright: End-to-end, API, accessibility, and visual testing using…

Why Microsoft Playwright?

Microsoft Playwright has become a popular choice for end-to-end testing due to its robust features:

Architecture and Design

• Auto-waiting mechanism that reduces flaky tests by intelligently waiting for elements to be actionable
• Browser context isolation that provides a clean testing environment for each test
• Built-in tracing capabilities that capture videos, screenshots, and network logs for debugging

Cross-browser Support

• Uses browser-specific protocols to communicate directly with Chromium, Firefox, and WebKit
• Single API works consistently across all major browser engines
• Maintains separate browser-specific drivers that optimize performance for each browser

Advanced Features

• Network interception and modification capabilities
• Powerful selectors including text, CSS, and XPath with built-in best practices
• Native handling of modern web features like iframe and shadow DOM
• Mobile device emulation and responsive testing support

Performance

• Parallel test execution out of the box
• Smart handling of timeouts and retries
• Efficient resource management through browser contexts

Despite its strengths, even Playwright can benefit from AI integration to overcome limitations such as optimizing execution, and generating insights from test runs.

What And Why is Zerostep?

Zerostep is a revolutionary AI library for Playwright that brings natural language processing to test automation. It allows testers to write test cases in plain English, which are then automatically converted into robust Playwright test scripts. This approach dramatically reduces the technical barrier to entry while maintaining the power and flexibility of Playwright.

ZeroStep revolutionizes the end-to-end (E2E) testing process by addressing the common pain points of traditional approaches. Lets see the some points that highlights its strengths:

The Conventional Approach:

1. Selectors are tightly coupled to the application’s markup: Traditional E2E tests rely on particular selectors (CSS, XPath, Ids): those are very sensitive to UI changes and the tests fail often.
2. Slow implementation: E2E tests involve coding of each action separately which takes time as compared to unit or integration tests.
3. Difficulty with complex scenarios: When it comes to the automation of complex user interactions flows tend to produce rigid tests, which, over time, require frequent adjustments.
4. Delayed testing: Testing can only start after the feature is fully developed, which can delay feedback and the identification of issues.

Testing with ZeroStep:

1. No selectors are used, ever: ZeroStep’s AI-based assistant analyzes the application and takes the necessary actions at runtime, eliminating the need for brittle selectors.
2. Quick test creation: Test creation is simplified into plain-text instructions, making it faster and easier to build tests without writing extensive code.
3. Easy automation: ZeroStep allows easy automation by enabling testers to simply describe the actions they want to test in plain language, even for complex scenarios.
4. Test-Driven Development (TDD) for E2E: With tests independent of implementation details, ZeroStep promotes a TDD approach, allowing tests to be written before the functionality is implemented. This leads to better design, faster feedback, and early bug detection.

In the next section you will see how to set-up Playwright and ZeroStep

Set-up and Installation

Install Playwright

Execute the below command to install Playwright

npm init playwright@latest

Install ZeroStep

Execute the below command to install ZeroStep

npm i @zerostep/playwright -D

Get ZeroStep API Key

To get the API we have to Login into ZeroStep https://app.zerostep.com and either create a free account or Login existing Google account Or Sign in with Github and Copy the Key see below screen. You will get 500 requests free in your account.

Create ZeroStep config file

Create zerostep.config.json file in the Playwright project root folder and paste the above key. In the below screenshot you can see it’s highlighted in the left side and the key is added into the .json file.

{
 "TOKEN": "0step:xx-00c8–421d-xx-xxxxxxx"
}

We have installed Playwright and set-up ZeroStep. Now we have to see how we can leverage AI using the ZeroStep library in our automation to make test scripts in plain English without writing much code.

Use Case For Automation

Let’s take a use case for automation, first we will do it with a traditional approach then we will see how we can automate using ZeroStep library.

For Example purpose I am taking the example of http://www.rahulshettyacademy.com/seleniumPractise/#/offers

Use case

1. Open the url http://www.rahulshettyacademy.com/seleniumPractise/#/offers
2. Get the price of Wheat from Table
3. Get the price of Tomato from table
4. Find the difference between the price of Wheat and Tomato
5. Verify the difference between Wheat and Tomato should be correct

Lets first automate the above use from a traditional approach. Below the code for the above use

Traditional approach Using Playwright and JavaScript

test("Green Kart Application -Traditional Approch ", async ({ page }) => {
 await page.goto('http://www.rahulshettyacademy.com/seleniumPractise/#/offers');
 await page.waitForSelector('table');
 const wheatPrice = await page.locator('//tr[contains(., "Wheat")]/td[2]').textContent()
 .then(text => parseInt(text));
 const tomatoPrice = await page.locator('//tr[contains(., "Tomato")]/td[2]').textContent()
 .then(text => parseInt(text));
 const priceDifference = Math.abs(wheatPrice - tomatoPrice);
 console.log('Price Difference Using Traditonal approch', priceDifference);
 expect(priceDifference).toBe(30);
 });

Here’s a breakdown of above code :

Test Initialization:

• The test is named “Green Kart Application -Traditional Approach” and uses an asynchronous function with the Playwright page object.

Navigate to the Application:

• The browser navigates to the URL: http://www.rahulshettyacademy.com/seleniumPractise/#/offers.

Wait for the Table to Load:

• The test ensures the table element (where the price data is displayed) is present using page.waitForSelector(‘table’).

Extract Prices for Wheat and Tomato:

• The price for “Wheat” is retrieved from the second column (td[2]) of the table row that contains the text “Wheat.”
• Similarly, the price for “Tomato” is extracted.

Calculate Price Difference:

• The difference between the prices of “Wheat” and “Tomato” is calculated using Math.abs() to ensure a positive value.

Assertion:

• The test uses expect(priceDifference).toBe(30) to validate that the difference between the two prices matches the expected value (30).

Implement above Use Case Using ZeroStep Library

test("Green Kart Application - With ZeroStep Library", async ({ page }) => {
 await page.goto("http://www.rahulshettyacademy.com/seleniumPractise/#/offers");
 const diffrence_price = await ai("What is value difference between Price of Wheat and Tomato ", { page, test });
 expect(diffrence_price).toEqual("30");
 });

Above code is a Playwright test that uses the ZeroStep library to simplify testing logic and reduce the number of lines of code.

Here’s a brief explanation:

Test Initialization:

• The test is named “Green Kart Application — With ZeroStep Library” and runs asynchronously with the Playwright page object.

Navigate to the Application:

• The browser navigates to the given URL: http://www.rahulshettyacademy.com/seleniumPractise/#/offers.

Using ZeroStep AI:

• The ai() function from the ZeroStep library is used to query the value difference between the prices of “Wheat” and “Tomato.”

Assertion:

• The test validates that the retrieved price difference equals “30” using expect(diffrence_price).toEqual(“30”)

In the above code you can see with the combination of Playwright command and ZeroStep library how easy it is to automate the test scenario when compared to the traditional approach where we have to write a lot of code.

Lets see comparison between Traditional Approach and using ZeroStep Library Approach

Comparison of Traditional Approach and ZeroStep Library Approach

Limitations of ZeroStep:

The ZeroStep library offers several significant advantages but also comes with a few limitations:

1. Intermittent Test Flakiness: Tests may occasionally become unstable compared to those using traditional locators.
2. Challenges with Complex or Dynamic UIs: Managing intricate or highly dynamic interfaces can be difficult.
3. Cost Involvement : Licensing or subscription costs associated with ZeroStep could be a limitation for developers, through 500 ai() requests free in your account when you sign up.

ZeroStep is a useful library for web page automation and there is space for enhancement. As the application of the technology continues to improve, it is more likely to become an even more efficient and effective tool for automation processes.

Conclusion

ZeroStep requires improvement in areas like handling complex UIs, reducing test flakiness.

Despite these challenges, the combination of Playwright and ZeroStep enables faster, more adaptive test automation, making it a valuable asset for teams looking to optimize their testing workflows.