Elevating Your Testing with AI-driven testing tool “Applitools Autonomous” : A Step-by-Step Approach

Test Your Applications with AI-driven tool “Applitools Autonomous”

This blog originally published under https://applitools.com/blog/comprehensive-testing-with-applitools-autonomous/

Nowadays companies are investing a lot of money on QA efforts to attain quality of the software products being released. However, the traditional QA method is not only time-consuming and ineffective, but also often disappointing in its results, which in turn triggers inefficiency, additional expenses and ultimately, higher risk for delivering the end product with visual or functional issues. The pressure to release frequently and stay ahead of the competition can lead to insufficient testing, which may result in bugs and other issues that negatively affect the user experience.

Today’s users require software that not only has the functional capabilities but also one with better graphics and is more easy to use. To meet these expectations, a lot of testing needs to be done for both the visuals and the user interface of the application. This is because as the software development life cycle progresses, the testing teams are put under pressure by the fast pace of software development. Development teams are likely to release products faster to counter competition hence flooding the market.

In this blog, we will explore the challenges of testing modern applications and how these challenges can be addressed by utilizing AI-powered testing tool Applitools Autonomous.

You will see how different types of testing like functional, visual, and API testing are addressed by tool Applitools Autonomous with detailed examples.

Challenges In Testing Modern applications

Below are some of the challenges that we normally face In testing modern applications.

Too Many Scenarios:

Modern applications are highly complex, with hundreds of end-to-end scenarios that need to be tested across a vast array of screens, devices, and browsers.

Too Many Screens & Devices:

In Modern applications, users access digital content on a wide variety of devices, browsers, and screen sizes, each with its own unique characteristics. Manually verifying that an application looks and functions correctly across all these environments is nearly impossible without automated assistance. Traditional testing methods cannot scale to meet this demand, often resulting in inconsistent user experiences and missed visual bugs.

Constant UI Changes:

In dynamic development environments, user interfaces (UIs) are constantly evolving. Even minor changes to the UI can require updates to test scripts. This constant maintenance burden can slow down the development process and lead to outdated or ineffective tests.

Different Skill Sets:

Teams often consist of members with varying technical skills, leading to the use of different tools and approaches to testing. This creates silos within the organization, where engineers and non-technical testers might not collaborate effectively. So far we have seen the challenges in testing modern applications.

In the next section let’s see how we can leverage AI to take over the heavy lifting in testing using Applitools Autonomous.

Types of Testing Using Applitools Autonomous

Applitools Autonomous, powered by Applitools’ proven Visual AI, offers a game-changing solution that allows teams to automate and streamline testing across various dimensions. In this blog, we’ll explore the types of testing you can perform using Applitools Autonomous and how it simplifies the testing process.

1. Full Website Testing

One of the key features of Applitools Autonomous is its ability to conduct a Full Website Test with minimal effort. Simply by entering your website’s URL, Autonomous crawls the sitemap and automatically creates a test suite that covers every page of your web application.

Benefits of Full Website Testing:

• Comprehensive Test Coverage: Automatically creates tests that span the entire site.
• Visual AI Validation: Screenshots are taken and compared in real-time to identify visual discrepancies.
• Technical Issues Detection: Automatically detects issues like HTTP errors, timeouts, and structural changes (added/removed pages).
• Cross-Environment Consistency: Ensures your website renders consistently across all environments.

With just a URL input, Applitools Autonomous handles the heavy lifting of testing, giving teams complete coverage across all environments.

2. URL List Testing

If you don’t need to test your entire website, or if you want to focus on specific pages, Applitools offers URL List Testing. This allows you to upload a list of important URLs that you want to validate across different browsers and devices.

Benefits of URL List Testing:

• Custom Test Plans: Create targeted test plans for critical or specific parts of your website.
• Visual and Functional Validation: Detect visual differences, server errors, and loading failures.
• Quick Setup: Simply upload a CSV, XLS, or sitemap file to initiate the testing process.

This method gives you granular control over which parts of your web application are tested, ensuring the most important areas receive detailed attention.

3. Custom Flow Testing

For complex workflows, Applitools Autonomous offers Custom Flow Testing, which allows you to simulate end-user actions across your site. For example, you can test user journeys like logging in, adding items to a cart, and checking out — with every action captured step by step.

Benefits of Custom Flow Testing:

• Workflow Validation: Ensure that user flows, from login to checkout, function correctly.
• Real-Time Interaction: Record steps in an interactive browser, which are automatically converted into plain English for easy review.
• Dynamic Content Testing: Easily handle personalized content and dynamic data using Visual AI, ensuring accuracy even with fluctuating data.

Custom Flow Testing is ideal for complex, multi-step processes, such as eCommerce workflows or user registration flows.

4. Cross-Browser and Device Testing

Cross-browser testing can be a significant bottleneck in the development process, but Applitools Autonomous eliminates this pain point. It replicates browsers in seconds, cutting down the time required for Cross-Browser and Device Testing by up to 90%.

Benefits of Cross-Browser Testing:

• Fast Replication: Save time by reducing cross-browser testing time by up to 90%.
• Consistent Testing: Validate how your site performs across multiple browsers and devices with minimal effort.

By reducing the time spent on browser replication, teams can focus on identifying and resolving any browser-specific issues faster.

5. API Testing

Applitools Autonomous doesn’t just stop at visual testing — it also enables API Testing. You can initiate HTTP requests directly in your tests, manage application states, and store responses for use in later test steps.

Benefits of API Testing:

• Seamless Integration: Initiate API calls and validate responses as part of your testing workflow.
• State Management: Manage application states before or after the UI flow without complex logic.

API Testing combined with visual validation ensures that your web app’s functionality works correctly and efficiently across the board.

Example of Testing Using Applitools Autonomous

In the below section you will see the different types of testing you can perform using Applitools Autonomous.

Full Website Testing

In full website testing, users need to provide the URL they want to scan and specify the test environments where they want to execute the scan. Additionally, users can set optional settings such as a maximum number of URLs and select a custom flow to run before the test. Once these steps are completed, the website is ready to be scanned.

For example purpose we are taking the example of site https://dan.sandbox.applitools.com/e-commerce

Steps to Create a Full Website Test

1.Navigate to Test Management:

• In the navigation bar, go to Test management > Tests.
• Click on Create a new test and then select Create a full website test.

2. Enter the URL:

• In the URL field, input the homepage of the website in the format https://www.example.com/.
• Alternatively, you can enter the full path to the sitemap or a sub-directory, such as https://dan.sandbox.applitools.com/e-commerce/type/books
• If a sub-directory is specified, Autonomous will use the sitemap in the root directory to identify URLs within the sub-directory. If the sub-directory has its own sitemap, provide the full path to that sitemap.

3. Default Test Environments:

• The test will use the following environments by default, each defining a browser, operating system, and screen size:
• Chromium, Linux, 1440 x 900
• Chromium, Linux, 360 x 780
• Microsoft Edge, Windows, 1080 x 1920
• You can click on an environment to remove it from the test. After creating the test, you can add or modify environments in the test plan.

4. Optional Settings:

Set the Limit

• To limit the number of URLs in the initial test run, click on Optional settings and select Set a max. number of URLs to include in the test, then enter the desired number of URLs. This helps to quickly run the first test and get preliminary results. This setting only applies to the first run; future runs will include all URLs in the sitemap, with new URLs marked as “New” in the results.
• If you have a trial account, note that there is a limit to the number of tests you can include in a full website test.

Custom Flow and Authentication:

• To enter basic authentication or run a custom flow before each URL opens, click on Optional settings and choose Select a custom flow to run before this test. You can select:
• Basic authentication and enter a username and password. This adds an authentication encoded string in each URL, ensuring the username and password are included.

5. Click on Next Button

Once the above steps are completed, your website is ready to be scanned.

Once you click on the Next button autonomous is capable of developing baseline data at every screen on the website. These are the initial test images which are in most cases used to identify changes in subsequent test iterations. The maximum count of baselines to be generated is obtained by multiplying the count of URLs in the test by count of environments.

In the below screenshot you can see baseline data is created for every screen.

6.Execute the Test

Once the baselines have been created, click Run Test Autonomous, run the test to create a checkpoint for each screen, then compare the checkpoint to the baseline, determine the issues and the screen has changed.

7.Execution report

Once the execution is complete , click Review Results to view the complete results in the Results page.

By clicking on the Results tab on the left side you can see the details of the Pass and Fail test case. In the above screenshot we can see one test case has failed when we try to find out the reason it was found that this test case was failing because of the 404 error screenshot attached below.

This way we can automatically find any technical issues,detect pages that were added or removed from the website and visual disparities and unexpected alterations on any of the pages.

URL List Test

A URL list test allows users to create a list of URLs with combinations that will be used for testing. This is especially useful when you have no sitemap, want to crawl specific URLs in your domain, or need to crawl URLs of other domains too.

It is recommended that while setting up your URL list test you should create a file with the URL to be tested and check if all of them are properly loaded. The list should therefore be in a format such as CSV, XLS or TXT, or in sitemap format. xml file. The sitemap. xml should only contain the URLs you are going to test and not all the URL of your site.

Steps to Create a URL List Test in Applitools Autonomous:

1.Navigate to Test Management:

• In the navigation bar, go to Test Management > Tests.

2.Create a New Test:

3. Click on Create a new test and select the URL list from the dropdown.

4.Enter Test Details:

• Test Name: Enter a name for the test. This will appear in the list of tests.
• Description: (Optional) Add a description of the test.
• Application: Select the application that the test should be associated with. Every test must be linked to an application. If you have an existing full website test or another test linked to the same application, associate this test accordingly. For more details, refer to the “Applications” section in the platform documentation.
• Plan: Choose the associated plans. Plans define the schedule, environment, and test parameters. A test can belong to multiple plans, and the list of available plans will depend on the application associated with the test. For more details, refer to the “Plans” section.

5.Run a Flow Before Each Step (Optional):

• If you need to perform an action (e.g., entering authentication credentials or closing a popup) before each step, click Select a flow to run before the screen URLs. For more details, see the Flow to Run Before Steps section.

6.Import URLs:

• Click Import URL list to upload a comma-separated text file, CSV, or XML file with the list of URLs you want to test.
• Alternatively, enter a single URL and click Add URL to manually add URLs one by one.

7.Set Match Level:

• The default match level for all URLs is set to Strict. If you want to change the match level for a particular URL, click the area to the right of the URL and select the desired match level from the dropdown.

8.Advanced Settings:

• To configure additional advanced settings for the test or individual URLs, refer to the Advanced Test Settings section for further customization options.

9.Finalize and Run:

• Once the configuration is complete, choose one of the following actions: -
• Create (or Update): Saves the test to the associated plans without running it.
• Create & run > Create & run test: Saves and runs the test as a single instance without executing any other tests in the plan. You will be asked to choose a plan to determine the test environments. You can view the results on the Results page, and the test will be listed as “Run without a plan.”
• Create & run > Create test & run plan: Saves the test and executes all associated plans, including all tests within those plans. Results can be viewed on the Results or Plans pages.

Custom Flow Testing

The Autonomous Applitools Custom Flow Test is an intelligent and streamlined way to automate end-to-end testing of specific workflows on a website. This feature enables testers to design a sequence of actions or user scenarios, for instance, login, adding items to a shopping cart, and navigating to a checkout page while at the same time, every action is recorded for evaluation purposes.

Key components of Autonomous Custom Flow Test:

1.Interactive Browser:

— Autonomous provides an interactive Chrome-based browser that allows users to manually interact with the application. Testers can perform actions like clicking buttons, filling in forms, scrolling, or navigating between pages.

— The interactive browser records every action in real-time and automatically translates them into test steps, simplifying the test creation process.

2. User Action Steps:

Navigate to Test Management:

In the navigation bar, go to Test Management > Tests

Click on Create a new test and select the → Custom Flow from the dropdown.

Below screens is open when user click on Custom flow

In the below screen you can see the recorded steps:

1. Actions performed in the browser, such as clicking or typing, are captured as plain English commands. These steps can be easily added, modified, or deleted. For instance, a step might read: “Enter “standard_user” as username” and Enter “secret_sauce” as password and “Click on the Login button.”
2. Autonomous offers auto-suggestions to ensure that the commands are valid, reducing the risk of errors and speeding up the creation of test scripts.
3. UI Validation Steps:

— Visual validation is a core feature of the Autonomous test. At key points in the workflow, Autonomous takes screenshots to validate the user interface. These visual checkpoints help ensure that there are no unexpected visual changes (known as visual diffs) between test runs.

— Screenshots act as baselines and provide insights into the accuracy of the UI across different devices and resolutions.

In the below screenshot you can see that after login into the application we are visually verifying the page.

Use Cases of Custom Flow Testing

Let’s take the example of https://www.saucedemo.com/ where user perform below actions

• Login into the application with Username and Password
• Click on Add to cart against the product “Sauce Labs Backpack”
• After adding into cart palace the order
• Finally verify the confirmation message.

For the given use case, we have two options to write the test script:

• Manually add user actions: This involves explicitly defining each user action in the script or code.
• Record the steps: This option allows for capturing the user’s interactions automatically by recording the steps as they are performed.

Both approaches can be utilized based on the specific requirement or convenience. In the below screenshot you can see the above use case is recorded.

• Debug or Modify a Test

In the below screenshot you can see we can perform various actions by clicking on any exciting action. The user has clicked on an action (likely Step 7, clicking a button with a CSS selector in above screenshot), revealing a dropdown menu that provides options such as:

• Run next steps: Executes the remaining steps in the test from the selected point onward.
• Set as current step: Marks the selected step as the starting point for future test execution.
• Run until here: Runs the test from the beginning up to the selected step, stopping just before it.
• View execution result: Displays the outcome of the selected step, including logs or screenshots, if applicable.
• Edit step: Allows modification of the selected step, such as changing input values or actions.
• Duplicate: Creates a copy of the selected step, useful for repeating actions with minor adjustments.

• Previewing Test Steps in an Interactive Browser

This approach saves time during the test construction process because it enables previewing of the test steps as one proceeds with the development of the test script in an interactive browser.

When in an interactive browser running a test there is an option where you get to see the screen as it was before each step was performed and after.

How to Preview a Test Step:

Place the mouse pointer over an icon before one or several particular test steps, and you will be provided with a tiny image of the screen. For a more detailed view of the screen, navigate through the icon on the toolbar in order to open up the full-size picture preview.

On the top of the preview window you can find information such as the number of the step, command run and its status — pass or fail.

Use the icons behind the worksample: Click on Action to see the screen before the step and Outcome, to see the screen after the step. Action window For when viewing the action window the element clicked on during the step is highlighted in pink.

Actions (Screen before the step)

Outcome (To See the screen after the step)

• Running the Test from the Start

To start the test from the first step you can just press the “Run” button or “Rerun test from start” which is above the list of the steps. While the test is running you can click or to pause or stop the test.

Once started, Autonomous will go through each step one by one while taking a screenshot of the program at every step. If the browser is not already connected, then the Run will also create the connection before the test is carried out. This way it can be ensured all the steps are performed correctly and also shows the status of a particular step.

• A ✅ icon indicates that the step was successful.
• A ❌ icon shows that Autonomous couldn’t process the step, and the test will stop at that point without continuing further.

During the test, you can “pause” or “stop” it at any time by clicking the Pause or Stop buttons.

API Test

Applitools Autonomous is not limited to functional and visual testing; it can also be used for API automation. Supporting all CRUD operations, it allows you to execute standard HTTP actions like GET, POST, PUT, and DELETE in your tests. This adds versatility to its API testing capabilities. It also enables you to validate test operations that cannot be confirmed via the UI, such as verifying form submissions through backend records. Additionally, it ensures backend APIs expose the correct endpoints, function as expected, and return proper responses, headers, and cookies

GET Requests

For GET request lest tale an example of https://gorest.co.in/

Navigate to Test Management:

In the navigation bar, go to Test Management > Tests.

Click on Create a new test and select the → Custom Flow from the dropdown.

Use Case 1: API Response With Positive Validation

Objective:
Hit the API endpoint and verify the following:

• response.status (HTTP status code)
• response.body[0].title (Title of the first object in the response body)
• response.body[0].id (ID of the first object in the response body)

In the below screenshot you can see all the above validation are passed

Copying Response Values:

Hover over a response value to view options to copy the following information to the clipboard. See in the below screenshot

• Copy Property Value: Copies the actual value from the response, e.g.,{“id”:157394,”user_id”:7436337,”title”:”Vulgus enim alius comis et officia.”,”body”:”Tametsi suscipit reiciendis. “}
• Copy Property Accessor: Copies the accessor to retrieve the value programmatically, e.g., {response.body.0}
• Copy Property Checkpoints: Copies a formatted checkpoint to use in a test, e.g.,

verify that {response.body.0.id} is 157394 and verify that {response.body.0.user_id} is 7436337

Use Case 2: API Response With Negative Validation

Objective:
Hit the API endpoint and verify the following:

• response.status (HTTP status code)
• response.body[0].title (Title of the first object in the response body)
• response.body[0].id (ID of the first object in the response body)

In the below screenshot you can see title validation is failing when we are verifying the title of first object

POST Requests

For POST request let’s take an example of the site https://reqres.in/api/users

Use Case 1: API Response With Creating User

• Objective:
  Hit the API endpoint https://reqres.in/api/users
• Pass the body in the request

{
"name": "morpheus",
"job": "leader"
}

POST https://reqres.in/api/users with header "Content-Type: applitcation/json" and header "Accept: application/json" and body '{ "name": "morpheus", "job": "leader" }'

• Verify the status 201

In the below screenshot you can see test case is passed and new record is created with status “201”

Use Case 2: API Response With Registering User — successful

• Objective:
  Hit the API endpoint https://reqres.inhttps//reqres.in/api/register
• Pass the body in the request

{
"email": "eve.holt@reqres.in",
"password": "pistol"
}

POST to https://reqres.in/api/register with header "Content-Type: application/json" and body '{ "email": "eve.holt@reqres.in", "password": "pistol" }'

• Verify the status 200
• Verify ‘token’ is generated

Summary

In the rapidly evolving software industry, companies are pouring resources into quality assurance to ensure smooth product launches. However, traditional QA methods are often too slow, costly, and ineffective, which can lead to bugs, poor user experiences, and product defects.However, AI-powered tools like Applitools Autonomous offer a powerful solution to these challenges. By automating functional, visual, and API testing, Applitools enhances testing efficiency and effectiveness, ensuring higher quality releases while reducing the risks of user dissatisfaction.