Insightful stories that revolve around technology, culture, and design

A sitemap is a comprehensive list of URLs on a website that provides information about the location and content of the website. It includes pages, videos, images, and files, along with their relationships to each other. This serves as a roadmap for search engines to navigate and understand the website's content.

There are different sitemap extensions, such as:

• XML Sitemap
• Mobile Sitemap
• Video Sitemap
• Image Sitemap

XML sitemap

An XML sitemap is a file that enumerates all the pages and other content present on a website, accompanied by metadata. It is aimed at assisting search engines in comprehending the site's structure. Google introduced the XML sitemap format in 2005 to streamline the process of crawling and indexing websites. XML sitemaps encompass metadata regarding URLs, which includes:

• The date of the last update
• The frequency of changes
• Its priority compared to other pages

Mobile sitemap

A mobile sitemap is tailored for mobile devices. It comprises a catalog of URLs from a website, optimized for seamless mobile viewing. This version may also include extra metadata about each URL, encompassing details like the last updated date, the frequency of changes, and its priority in relation to other pages.

It is important to note that a distinct mobile sitemap is required only if the website includes a feature phone (WAP/WML) version. For the majority of websites, a unified sitemap is adequate for both desktop and mobile URLs.

Video sitemap

A video sitemap is a specialized form of a sitemap that contains additional information about videos hosted on a website. It offers essential metadata about the video content, including the title, description, thumbnail URL, and runtime.

This information aids search engines like Google in comprehending and indexing video content, thereby simplifying users' direct discovery and access to videos from search results.

Through the creation of a video sitemap, website owners can enhance the visibility and discoverability of the video content, potentially resulting in increased traffic, engagement, and conversions.

The namespace that defines the tags for video sitemaps is as follows:

Image sitemap

An image sitemap provides search engines with information about the images on a website. This assists search engines in crawling and indexing images, potentially boosting visibility and attracting traffic from image search results.

The image sitemap includes metadata about each image, such as its URL, type, and caption. Using an image sitemap aids search engines in comprehending a website's content, contributing to improved overall search engine optimization.

The namespace establishes the image tags utilized in the image sitemap to relay metadata about website images to search engines like Google. For instance:

Advantages of sitemaps

SEO ranking

Sitemaps offer a clear overview of a website's structure, simplifying user navigation and content discovery. A properly organized sitemap can significantly impact website ranking. It contains a list of the most crucial pages on a site, ensuring search engines can locate and crawl them effectively.

User-friendly navigation

Sitemaps enhance website navigation for both search engine crawlers and human users by presenting an organized and easily-followed layout of website content.

Improved crawling

Without a sitemap, search engines might struggle to crawl and index all pages on the site, potentially affecting the site's rankings. The term "improved crawling" refers to how search engine crawlers navigate and index a website more efficiently and accurately when a properly organized sitemap is present, instead of relying solely on the site's architecture.

Notifying about updates

By including metadata in sitemaps — such as the last modification date and different language versions of a page — one can effectively communicate updates to search engines and provide precise information about a page's content.

Checklist for testing sitemaps

Manually check the location of the sitemap

The most common (and simple) way to locate the XML sitemap of a website is to check manually. To initiate the process, input the website URL into the browser and explore various iterations, experimenting with different variations such as:

http://www.{yourwebsiteurl}.com/sitemap.xml

Here are a few additional quick tests to attempt:

• /sitemap-index.xml
• /sitemap.php
• /sitemap.txt
• /sitemap.xml.gz
• /sitemap/
• /sitemap/sitemap.xml
• /sitemapindex.xml
• /sitemap/index.xml
• /sitemap1.xml

Best practices to follow

• The best practice for a sitemap is to place it in the root directory Baseurl/sitemap.xml
• The sitemap protocol format consists of XML tags. It is important to ensure that all data values in a sitemap are properly entity-escaped and the file is encoded in UTF-8 format. It will help ensure that search engines can correctly interpret the data in the sitemap, and accurately index the website.

Guidelines to follow

Format the sitemap correctly to ensure search engines can read and interpret it properly, and follow the steps below:

• Start the sitemap with an opening <urlset> tag and ending it with a closing </urlset> tag. Adhere to the proper protocol standard specified within the <urlset> tag.
• Specify the namespace (protocol standard) within the <urlset> tag.
• Add each URL as a parent XML tag <url>.
• Include a <loc> child entry for each <url> parent tag to specify the URL.

Check Robots.txt file

A robots.txt file is a text document that guides web robots or crawlers on which parts of a website they can or cannot access. It prevents the site from getting overwhelmed by requests.

Crawlers usually check the robots.txt file first when they visit a site. So, it's important to include a sitemap path there. This tells search engines about the pages you want them to find. To see if it's set up correctly, enter your site's URL followed by '/robots.txt' in your browser, for example, https://www.qed42.com/robots.txt.

For more details, check The Importance of Robots.txt.

Test mobile XML sitemap

To test a mobile XML sitemap, utilize online tools to check for validity and identify any errors or warnings. One useful tool is Google Search Console, offering a Mobile Usability report to pinpoint mobile compatibility issues on site.

Another option is the W3C mobileOK Checker, which assesses mobile sitemap and offers feedback on areas for enhancement. This tool measures the site's mobile usability against industry standards, suggesting improvements to enhance mobile compatibility.

Alternate language directive for XML sitemap

If there are different language versions of the website, use the "hreflang" tag in the XML sitemap to indicate the relationship between pages in various languages. This can help search engines understand which version of the website to display in search results based on user language and location preferences. Additionally, including this information in the XML sitemap can help search engines properly index the website for international SEO.

Pages added to optimize crawling and indexing

The sitemap should not include all  pages. If everything is included, it may lead to poorly optimized crawling, which means crawling low-quality pages. This can hinder the indexing of high-quality pages on the site because search engines might not have the resources to crawl them

Ensure that the pages included in a sitemap:

• Respond with a 200 code
• Are not blocked by robots.txt
• Don't include a no-index meta robots tag
• Have canonical versions

Make sure that the sitemap is broken into smaller sections if it contains more than 50,000 URLs. The maximum size of the sitemap should be 10 MB.

Check uniformity and completeness of URLs within the sitemap by confirming the presence of the HTTP/HTTPS protocol and "www" in all URLs.

Utilize sitemap extensions for additional media content such as images, videos, and news.

Conclusion

In this blog, we've covered various sitemap types: XML, mobile, video, and image sitemaps. We've looked at how they can benefit SEO ranking, improve navigation, and enhance crawling. We've stressed the importance of correctly formatting and validating sitemaps to ensure search engines interpret and index the site accurately.

Don't forget to use the checklist we provided to test the sitemap! We've emphasized testing mobile XML sitemaps and using alternate language directions to enhance crawling and indexing.

In the 'A Beginner's Guide to Sitemap Testing: Part 2' blog post, we delve into the tools that assist with sitemap testing. These tools include Google Search Console, XML Sitemap Validator, and SEOptimer. Additionally, we demonstrate real-time examples using Screamingfrog to test a sitemap.

Remember, a well-tested and properly optimized sitemap is a valuable asset for improving your website's search engine visibility and user experience.

‍

Headway is an open-source automation framework by QED42 to build test scripts and generate reports for web applications. It primarily focuses on minimizing initial framework setup and coding efforts increasing automation efficiency. It supports a data-driven framework built using Selenium, TestNG and Java libraries. 

Headway is the answer to

• Implementing automation from scratch in an existing project
• A cost-friendly framework - use of open-source libraries
• Reduce dependency on highly-skilled and technical automation QAs

How does it work?

Headway is a data-driven framework consisting of various components, such as Selenium, Java Language, TestNG framework, Maven dependency management tool, and utility libraries. It supports the Page Object Model pattern, where we have separate packages for pages and tests. 

Configuration files are used to store environment and application generic data that remains static throughout the framework. All the test data can be stored in Excel or JSON files, and we can handle the data using appropriate libraries. At the end of the test suite execution, report and log files are generated with screenshots. 

Headway architecture can be reused, and libraries can be extended for test automation. Utilities of Headway can be used directly or exported as jar files in your automation project.

Features of Headway

1. Data-driven testing framework to run tests multiple times over a set of test data.
2. Multiple browser testing on Chrome, Firefox, and Safari across various platforms.
3. Supports sequential and parallel testing across multiple browsers to reduce overall execution time.
4. Provides a detailed and graphical report dashboard and generates HTML reports with screenshots.
5. In-house support to read and write various file types, such as Excel, JSON, and properties.
6. Easily integrates with CI and 3rd party tools.

Benefits of Headway

1. Framework development time is reduced by 40%.
2. Reduced costs as the framework is built using open-source libraries and tools.
3. Increases ROI by 30% due to automated framework, parallel testing, and shorter test cycles.
4. Reduced dependency on QA experts, which provides flexibility in resources.
5. Less execution time with parallel testing.
6. QA resources can focus on test scripts and extending the framework.
7. Increase in reliability and quality, accelerating validation and release of web applications.
8. Easy to use and understand the utilities by less-skilled QA resources.

What is the wait for…

Try Headway and implement automation in your project with ease! The how-to-use guide will help you try Headway effortlessly!

Artificial Intelligence (AI) refers to the development of computer systems that can perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. AI systems use algorithms, statistical models, and computational power to analyze data and make predictions or decisions based on that data. 

Machine Learning (ML) is a subfield of AI that involves teaching machines to learn from data and improve their performance without being explicitly programmed. Machine learning algorithms can analyze large datasets, identify patterns, and make predictions or decisions based on that data. 

Artificial intelligence and machine learning (AI/ML) have become increasingly important in testing software, resulting in increased automation and more lifelike results. AI/ML testing is essential to ensure that AI/ML models produce accurate and reliable results.

AI/ML models are trained using large datasets, and it can be challenging to ensure that they are working correctly and producing the expected results in different scenarios. The testing process helps identify errors, biases, and other issues in the model, making it more reliable and effective. Additionally, AI/ML testing helps improve the model's transparency and interpretability, making it more trustworthy and easier to use. 

Imperatives for AI system testing

The global AI market is expected to reach nearly $126 billion by 2025, up from just $10.1 billion in 2016. With AI becoming more prevalent, it's crucial to ensure that these systems are tested thoroughly. Originally, AI was conceived as a technological concept to replicate human intelligence. It was mostly researched and developed within the confines of large technology companies.

However, in recent times, AI is transformed into an indispensable resource for every type of organization, thanks to significant advancements in data collection, processing, and computing power. In essence, AI has become the "new electricity" for businesses of all kinds. 

Challenges of AI and ML testing

Non-Deterministic

AI and ML systems are non-deterministic. This means that they tend to show different behaviors for the same input. 

Adequate and Accurate Training Data

One of the biggest challenges of AI and ML testing is the lack of test data. ML algorithms rely on vast amounts of data to learn, but it can be challenging to obtain sufficient data that accurately represents the real-world scenarios that the system will encounter. This can make it tough to test the system thoroughly and accurately. 

Bias

Testing for bias can be challenging, as it requires a thorough understanding of the training data and the potential sources of bias. 

Interpretability

There is a massive level of difficulty when extracting specific attributes. For example, finding what caused a system to recognize an image of a coupe as a sedan wrongly may not be possible. 

Sustained Testing

After testing and validating traditional systems, we do not need to retest them until we modify the software. On the other hand, it is imperative to know that AI/ML systems constantly learn, train, and adjust to new data and input. 

Common obstacles in AI application testing

Massive sensor data pose storage and analytics issues and result in noisy datasets. Here are some common obstacles faced while testing AI systems/applications: 

• Data obtained during unplanned events is exceedingly challenging to aggregate, offering training problems to AI systems.
• Human bias is typically present in training and testing datasets. It should be identified and removed in AI model test situations.
• AI works well with sophisticated input models. If the inputs are not up to the mark, the defects are more complex, and a lot of time and effort is consumed to resolve them.
• AI is a complex system, and even minor defects are greatly amplified. The difficulty in this problem resolution also increases.

Challenges in testing ML applications

Testing machine learning applications are also difficult and pose several self-assessment challenges: 

• The data, code, curricula, and frameworks that support ML development must be thoroughly tested.
• Traditional testing methods, such as test coverage, are often ineffective when testing machine learning applications.
• The behavior of your ML model may change each time the data training is updated.
• As domain-specific information is necessary, creating a test or test (e.g., labeling data) costs time and money.
• As it is challenging to identify trustworthy carpenters, ML testing frequently indicates false positives in defect reports.

Key factors to consider while testing AI-based solutions

When evaluating AI-based solutions, it's important to remember that data is the new code. For a well-functioning system, these solutions must be tested for any change in input data. This is similar to the classic testing approach, in which any changes in the code cause the improved code to be tested. 

Several steps must be taken to create effective and accurate machine learning models, such as: 

Semi-Automated Curated Training Data Sets

For this step, the input data and intended output are essential. We need to analyze data dependencies statically to annotate data sources and features. This analysis is crucial for migration and deletion. 

Test Data Sets

Test data sets are created to determine the efficacy of trained models. These data sets are logically constructed to test all possible combinations and permutations. The model gets refined during training as the number of iterations and data richness increase. 

System Validation Test Suites

Algorithms and test data sets are used to create system validation test suites. These test suites must include various test scenarios, such as risk profiling of patients for the disease in question, patient demography, and patient therapy, for a system designed to predict patient outcomes based on pathology or diagnostic data. 

Reporting Test Findings

Test results must be presented in statistics since machine learning algorithm validation yields range-based accuracy or confidence scores rather than expected outcomes. For each development, testers must specify confidence criteria within a given range. 

Taking note of fundamental biases

Fundamental biases are essential to take note of during AI/ML testing. For modern enterprises, developing unbiased systems has become critical. Supervised learning techniques, which make up more than 70% of AI use cases today, often rely on labeled data prone to human judgment and biases. 

It creates a double-edged sword for measuring the bias-free quotient of the input training data sets. We miss out on experiential information if we don't factor the human experience into labeled data. And even if we do, data biases are likely to emerge. 

Data Skewness

Data skewness is a common problem in machine learning, especially in sentiment analysis. Most data sets do not have equal or sufficient data points for different sentiments, leading to skewed training data. 

Prediction Bias

In a well-functioning system, the distribution of predicted labels should match the distribution of observed tags. This diagnostic step is crucial to detect problems, such as sudden changes in behavior. If the training distributions based on historical data are no longer accurate, it can lead to prediction bias. 

Relational Bias

Users' understanding of how to solve a data pattern or problem set is often constrained and biased by their knowledge of relational mapping, leading them to favor more familiar or simple solutions. This bias can result in a solution that avoids complex or unfamiliar alternatives. 

Critical aspects of AI systems testing

Data Curation & Validation 

The efficiency of AI systems is based on the quality of training data, including aspects like bias and variety. Car navigation systems and phone voice assistants find it quite troublesome to understand different accents. It proves that data training is critical for AI systems to get the correct input. 

Algorithm Testing

Algorithms, which process data and provide insights, are at the heart of AI systems. This approach's primary advantages include model validation, learnability, algorithm efficiency, and empathy. 

• Natural Language Processing
• Image Processing
• Machine Learning
• Deep Learning

Performance and Security Testing

AI systems require extensive performance and security testing. Aspects such as regulatory compliance are also included. 

• Smart Interaction Testing
• Devices (Siri, Alexa, etc.)
• AR/VR
• Drones
• Driverless cars

The potential of AI

AI research used to be confined to large technology companies, and it was envisioned as a technical idea that could emulate human intelligence. On the other hand, AI has become the new electricity for every organization, thanks to significant breakthroughs in data collecting, processing, and computation power. 

The AI sector has exploded in the last several years, with applications covering multiple industries. The widespread adoption of AI is expected to help uncover its full potential and increase efficiencies in various industries in the coming years. 

Integration Testing

AI systems are created to work with other systems and tackle specific challenges. This necessitates a comprehensive analysis of AI systems. Integration testing is critical when numerous AI systems with competing agendas are deployed together. 

Real-life Testing

AI systems are created to work with other systems and tackle specific challenges. This necessitates a comprehensive analysis of AI systems. According to Gartner, the worldwide business value of AI was expected to exceed $1.2 trillion in 2018, up 70% from 2017. By 2022, this market was estimated to reach $3.9 trillion. With more and more systems incorporating AI features, they must be adequately evaluated. 

Non-Functional Testing

Semi-automated curated training data sets include input data and intended output. Static data dependency analysis is necessary to enable the annotation of data sources and features, a key aspect for migration and deletion. 

Black Box and White Box testing

Like traditional test methods, black box and white box testing are used for ML models. Obtaining training data sets that are large and thorough enough to suit the objectives of ML testing is a significant difficulty. 

Data scientists test the model's performance during the development phase by comparing the model outputs (predicted values) to the actual values. The following are some of the strategies used to do black box testing on ML models: 

Model Performance Testing

It entails comparing the model's performance in terms of precision-recall, F-score, and confusion matrix (False and True positives, False and True negatives) to that of a predetermined accuracy with which the model was previously constructed and placed into production. 

Metamorphic Testing

It attempts to solve the problem of the test oracle. A test oracle is a method that allows a tester to assess whether a system is functioning correctly. It is challenging to determine the expected outcomes of selected test cases or know if the actual output matches the expected results.

Dual Coding/Algorithm Ensemble

Given the same input data set, multiple models utilizing various algorithms are created, and predictions from each one are compared. Numerous methods, such as Random Forest or a neural network like LSTM, could design a typical model to address classification difficulties. However, the model that gives the most expected outcomes is ultimately chosen as the default. 

Coverage Data

Data fed into the ML models are designed to verify all feature activations using guided fuzzing. Test data sets that result in the activation of each of the neural network's neurons/nodes, for example, are required for a model produced with neural networks.

Model Backtesting

A predictive model based on historical data is known as backtesting. This method is widely used in the financial sector to estimate the performance of previous models, particularly in trading, investment, fraud detection, and credit risk evaluations. 

Testing for Non-Functional Requirements (NFR)

A representative sample view of things and the deployment approach must be considered while evaluating ML Models with performance and security testing. AI systems require extensive performance and security testing. Aspects such as regulatory compliance are also included. 

HSBC's Voice Recognition System was recently hacked by a customer's non-identical twin, who gained access to balances, recent transactions, and the ability to transfer money across accounts. Chatbots can be influenced into providing business-sensitive information without proper testing. 

AI/M-based tools for testing

There are many AI-based QA products on the market, each with its own set of features. Here's a quick rundown of the three most commonly used AI tools in software quality assurance. 

Applitools

Applitools is a visual UI testing and monitoring program powered by artificial intelligence. It is a visual AI-powered end-to-end software testing platform that can be used by engineers and manual QA, as well as test automation, DevOps, and Digital Transformation, teams. Furthermore, the AI and ML algorithm is completely adaptive - it scans and analyses the app displays just like a human eye and brain would but with the power of a computer. 

Testim

It's an AI and ML-based automated functional testing platform that speeds up automated tests' creation, execution, and management. Chrome, Firefox, Edge, IE, Safari, and Android are among the browsers and operating systems that can use the tool. This AI-powered software testing platform lets customers develop robust end-to-end tests that can be programmed or left codeless, or both. Testim's original cycle model is responsible for its success and popularity. 

Sauce Labs

Another popular cloud-based test automation solution that uses ML and AI is Sauce Labs. It supports a wide range of browsers, operating systems, mobile emulators, simulators, and mobile devices. It works at the pace that its consumers require. It also claims to be the world's largest continuous testing cloud, with over 800 browser and operating system combinations, 200 mobile emulators and simulators, and hundreds of genuine devices available. 

Summing Up

As AI and ML become more prevalent in our lives, it's crucial to ensure these systems are thoroughly tested to work as intended. The regularity with which the AI model is tested for accuracy affects the previous 'test once and deploy forever' strategy.

As businesses increasingly use AI to construct systems and applications, testing approaches and procedures will evolve and improve over the next few years, eventually approaching the maturity and standardization of traditional testing methods. 

There is always a world of difference between the test environment conditions and real environmental conditions. For example, we cannot trust the efficiency of a robot, car or any other equipment that is tested only in laboratories (test environments). And when these are brought to the real world or tested under real-world scenarios, there are 90-95% chances of failure.

The efforts of repairing this damage or loss are very high as compared to the efforts required to repair if the same issue were found in the earlier testing stage. 

You can come across a similar case in software development, even after testing the software in all possible ways there are chances that some bugs escape to User Acceptance Testing (UAT).

What is User Acceptance Testing?

User Acceptance Testing (UAT) is the testing phase conducted by the client to validate the developed software system before its deployment to the production environment. This testing phase occurs during the final stage of the testing process, following the completion of functional, UI, and integration tasks.

Why is User Acceptance Testing necessary?

As all know User Acceptance Testing occurs in the final stage of testing. Its purpose is to ensure that the developed software system meets the client's and end-users real-time requirements and acceptance criteria. Involving the client/stakeholders in testing the system against the specified requirements, can identify issues or areas of poor usability in the testing stage before production deployment, and it helps to save the cost and time of issue fixing. This collaboration allows the development team to make necessary improvements based on the UAT feedback, ensuring that the final product meets the highest standards before its successful launch into production.

Here are a few cases that might introduce bugs in User Acceptance Testing and how to tackle them:

Case 1: Acceptance criteria not understood

• Acceptance criteria should be understood from the customer's perspective. For this QAs should be part of the Sprint 0 activities.
• The customer requirement must be clear, and accordingly need to work on Acceptance criteria.
• If the requirements provided to the user for testing are incomplete or unclear, it can result in bugs or missing functionalities.
• To tackle this, clear communication channels should be established between the development team/ team representative and the customer, allowing any questions or clarifications.

Case 2: Testing "n" number of times is not sufficient

• For overcoming this, the project management team should be proactive and make an effective plan. QA timelines should be considered.  
• QA timelines should be considered.  

Case 3: Negative test cases suite is missing

• Test cases suite should be included in the negative test cases. These test cases that are designed for validating the software behaviour against unacceptable, abnormal, or unexpected condition data input. 
• Need to cover negative test cases to ensure that the developed software/Application output response should be expected when providing an invalid or random input.
• Eg. A software development team has built an e-commerce platform which has Login functionality with Mail ID and Password but while testing they miss the negative test case like “if the mail Id/ Password fields are empty and try to the Login, what is the response”,  “if both the mail Id/ Password fields are empty and try to the Login, what is the response”,  “if enter invalid/wrong mail Id”. Aforementioned scenarios, the response should be an error message.

Case 4: Collaboration between QAs and other team members

• QAs don't collaborate equally in the team and they end up working either completely or partially on their own. QAs should always communicate clearly about the testing scenarios, testing strategies with other team members and also share their research work and ideas that can be helpful for the project.
• Always keep visibility in QA  activity so other team members know about testing strategies.
• As a QA if you have any suggestions for improvement of the project functionality/usability, it should be discussed with team members.
• Eg. All sites have a Header section, when the user scrolls down, and then the wants to check the header section then at that time the user needs to scroll up the page; QA can give a suggestion to add a “Back to Top” button on the screen after one scroll of a page.

Case 5: Participate in all project meetings

• QAs should participate in all meetings, this allows them to acquire in-depth knowledge of the project from an end-user perspective and also develop a collegial network with other stakeholders on the project.
• They can ensure that the requirements are clear, testable, and aligned with the overall goals of the project.
• QA plays the role of a gatekeeper, responsible for understanding all customer and stakeholder requirements. Knowing the requirements empowers to raise qualitative issues during testing, ensuring the product meets the highest standards of quality.
• QA involvement in project meetings allows them to track the progress of development and testing activities. They can provide updates on the status of the deliverables, identify any issues, and suggest any new ideas for improvement.
• Overall when QA participates in Projects meetings, it helps in Project quality management, raising issues early and timely resolutions of issues, resulting in the delivery of a high-quality end product by the team.

Case 6: Checklists for common scenarios are missing

• Always keep test cases or checklists handy for common scenarios that a user will come across during the testing. These checks should cover all the scenarios across that feature.
• Eg. For every website, the Logo of the site must be clickable and navigate the user to the homepage. Though this scenario is common, it can sometimes be overlooked during testing. However, by including it in the QA checklist, the chances of missing this scenario is reduced.

Case 7: Test data is not sufficient

• Seldom the test data created by QAs is not sufficient. Try to test by restoring production databases i.e with real-time data. 
• Test data needs to be precise to uncover the defects.
• Test data helps QA to conclude whether the developed system is relatable or not with the expected output.
• If a QA has sufficient test data before they begin testing activities, then the quality of the product will be up to the mark and makes sure all scenarios are covered.
• Eg. During the testing of an E-commerce website, the checkout flow represents an important scenario that requires examination. As part of this process, testing the "Payment" step is crucial. However, QA may encounter challenges if they lack the necessary testing data, such as card details, to proceed with the payment verification and to complete checkout.

Case 8: Poor site performance

• Try to increase the load on the system by setting up performance tests and by sending multiple requests in parallel. 
• QA needs to test site performance for heavy traffic, for different networks, and for site response time.
• Eg. If the developed site/application performance is high, then the site can handle heavy traffic.

Case 9: UAT and QA environment metrics differ

• QAs and customers should follow the same environment and device metrics. This should be defined and approved as a part of the test plan.

Case 10: Regression tests are missing

• Extensive regression testing before UAT is very important. By chance if any issue is missed while testing or any functionality is not working as per requirement that will be found while doing regression testing.
• Regression testing should be performed to reduce the risk of poor user acceptance.
• Eg. In the Checkout process, there is a mandatory "Checkbox" that users must check. Despite conducting functionality testing for all positive and negative scenarios, one crucial test case was overlooked: verifying the scenario when the 'checkbox' is unchecked, and the user attempts to checkout, the User is able to do the checkout process. Unfortunately, it resulted in a test case failure.
• To enhance the overall quality of the end product, it is important for QA to perform regression testing, identifying and resolving such issues. Doing so can significantly reduce the risk of poor User Acceptance.

We should always focus on breaking the system in all possible ways to reduce the count of User Acceptance Testing bugs. 

Happy Testing! 

You know exactly how this goes. You write a test. It passes once, then fails the next morning.
‍

TimeoutError: Waiting for selector ".submit-btn" failed

You add sleep(1000). It passes. You push it. Fails on a slow machine. You bump it to sleep(3000). Now your suite takes 12 minutes. Still fails on staging when the server is having a bad day. You bump it again.

I've been in that loop. The problem isn't the numbers. The root cause is a mismatch: your test runs at code speed, but your UI moves at human speed. Buttons appear after API responses. Modals open at varying times. Form fields stay disabled until validation passes. Every one of those gaps is waiting to break your test.

The playwright's answer isn't a bigger sleep. It's auto-waiting and once you understand how it actually works, you'll know what to do the next time a test breaks instead of just guessing at the number.

By the end of this, you'll know exactly which of five checks failed when Playwright times out and the fix will almost never be a bigger sleep.

TLDR

Playwright checks five things before every action  attached to DOM, visible, stable, enabled, not covered  automatically, on a retry loop every ~100ms up to 30 seconds. When something fails, the error tells you which check timed out. Read it. The fix is almost always one line.

What auto-waiting actually is

When you write this:
‍

await page.getByRole('button', { name: 'Submit' }).click();

‍

The playwright doesn't fire a click event straight away. It runs a polling loop checking every ~100ms, up to 30 seconds and only fires when the element passes all five actionability checks.

It's not magic. It's a retry loop with smart conditions built in. Every action — click, fill, check, hover, selectOption, dragTo — runs this loop automatically.

The default timeout is 30 seconds. You can override it per-action or globally:
‍

// Per-action
await page.getByRole('button').click({ timeout: 5000 });


// Global config
export default defineConfig({
use: { actionTimeout: 10_000 },
});

What each actionability check actually does

Let's go through each check, because knowing them is what turns a confusing timeout into a one-line fix.

1. Attached to the DOM

The element must exist in the document  not hidden via CSS, but actually present in the DOM tree.

// Waits until the element is rendered into the DOM
await page.getByTestId('user-card').click();

‍

This handles lazy-loaded components, skeleton loaders replacing real content, and anything injected by JavaScript after the page loads.

2. Visible

The element must be visible. Playwright checks:

• Display is not none
• Visibility is not hidden
• Opacity is not 0
• The element has a non-zero bounding box (width > 0, height > 0)
  

// Fails if the modal is hidden; waits if it's animating in
await page.getByRole('dialog').getByRole('button', { name: 'Confirm' }).click();

3. Stable (not animating)

If an element is sliding in, fading up, or otherwise moving, Playwright waits for it to stop. It does this by comparing the element's bounding box across two consecutive frames. If the position changed, it's unstable keep waiting.

// The dropdown animates open. Playwright waits for it to settle.
await page.getByRole('listbox').getByText('India').click();

‍

In practice, this is the check that trips people up most on component libraries with heavy CSS transitions. The element is visible but the playwright won't touch it because it's still moving.

4. Enabled

Form elements input, button, select, textarea must not be disabled.

// Waits until the submit button becomes enabled after form validation passes
await page.getByRole('button', { name: 'Submit' }).click();

// Waits until the submit button becomes enabled after form validation passes

await page.getByRole('button', { name: 'Submit' }).click();

5. Editable (for fill and type)

For input actions, Playwright also checks that the element is not readonly and not disabled.

// Waits for the OTP field to become editable after the SMS is sent
await page.getByLabel('Enter OTP').fill('123456');

How Playwright compares to Selenium and Cypress

Selenium makes you write every wait yourself:
‍

await driver.wait(until.elementLocated(By.css('.submit-btn')), 10000);
await driver.wait(until.elementIsVisible(
driver.findElement(By.css('.submit-btn'))), 5000);
await driver.wait(until.elementIsEnabled(
driver.findElement(By.css('.submit-btn'))), 5000);
const btn = await driver.findElement(By.css('.submit-btn'));
await btn.click();

‍

Cypress is better , it retries get() automatically but you still declare the conditions yourself:
‍

cy.get('.submit-btn', { timeout: 10000 })
.should('be.visible')
.should('not.be.disabled')
.click();

Playwright:
‍

await page.getByRole('button', { name: 'Submit' }).click();

One line. All five checks happen automatically.

When auto-waiting saves you without extra code

Scenario 1: Button appears after an API response
‍

await page.getByRole('button', { name: 'Load More' }).click();
// Waits until the new items actually appear in the DOM
await expect(page.getByTestId('product-card')).toHaveCount(20);

No sleep. No explicit wait. The assertion retries until it's true.

Scenario 2: Multi-step form with dependent fields
‍

// Country field enables State field
await page.getByLabel('Country').selectOption('India');
// State dropdown is disabled until country is selected
// Playwright waits for it to become enabled automatically
await page.getByLabel('State').selectOption('Karnataka');
// Submit is disabled until both are filled
await page.getByRole('button', { name: 'Continue' }).click();

Where auto-waiting won't save you

These are the three cases where you need to help Playwright along.

Case 1: Element is visible but covered by an overlay

The cookie banner and chat widget problem. The element is in the DOM and visible but something is sitting on top of it.
‍

Error: locator.click: Element is not visible
Locator: getByRole('button', { name: 'Sign Up' })
Element: visible
Covered by: <div id="chat-widget" />

‍

await page.locator('#chat-widget .close-btn').click();
await page.getByRole('button', { name: 'Sign Up' }).click();

Case 2: Element that never becomes stable

Some animations loop indefinitely spinners, pulsing indicators. Playwright waits for stability and eventually times out.
‍

// Clicking inside a looping spinner will always timeout
await page.locator('.loading-spinner .inner-text').click();



// Wait for the spinner to disappear first
await expect(page.locator('.loading-spinner')).not.toBeVisible();
await page.locator('.inner-text').click();

Case 3: Page navigations need explicit waits

After a form submit, Playwright doesn't automatically wait for the page to fully navigate before running the next action.
‍

// This might proceed before the page fully navigates
await page.getByRole('button', { name: 'Login' }).click();
await page.getByRole('heading', { name: 'Dashboard' }).waitFor(); // might fail



// Better: wait for the URL change explicitly
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('**/dashboard');
await page.getByRole('heading', { name: 'Dashboard' }).waitFor();

‍

Note: waitForNavigation is soft-deprecated in Playwright v1.26+. Use waitForURL() or waitForLoadState() instead.

How to debug a failed actionability check

When Playwright times out, here's the actual workflow:

Step 1 - Read the error

Playwright's error messages are specific. "Element is not visible", "Element is covered by", "Element is not stable" each one points to a different check. Don't jump to a fix before you read which check failed.

Step 2 - Run with --headed and --slowmo

Watch what actually happens in the browser. Nine times out of ten, you'll see exactly what's blocking the element.
‍

npx playwright test --headed --project=chromium -- test-name

Step 3 -  Fix the root cause, not the symptom

If the element is covered, dismiss the overlay. If it's not stable, wait for the animation to finish. If it's disabled, wait for the condition that enables it. Never reach for sleep() to patch a timing issue you don't understand yet.

FAQ

Does auto-waiting work with iframes and shadow DOM?

Playwright's locators handle shadow DOM natively. For iframes, use frameLocator() to scope your locator auto-waiting then applies to the element within that frame.

What happens when the 30-second timeout expires?

Playwright throws a TimeoutError and the test fails at that step. To retry the whole test, configure retries in playwright.config.ts.

Can I disable auto-waiting for a specific action?

Not directly. You can set a very low timeout to fail fast, but there's no flag to skip actionability checks entirely.

How is expect().toBeVisible() different from the auto-waiting visibility check?

The actionability check runs internally before actions like click and fill. expect().toBeVisible() is an explicit assertion that also retries use it when you need to assert state without triggering an action.

Why do my tests pass locally but fail in CI?

Usually two causes: CI machines are slower and hit the default 30s timeout, or overlays (cookie banners, chat widgets) appear in the test environment but not locally. Check which actionability check failed that tells you which environment difference to investigate.

Does auto-waiting apply to page.evaluate() calls?

No. page.evaluate() runs JavaScript directly in the browser and bypasses all actionability checks.

Conclusion

The next time a test fails and someone reaches for sleep(3000), stop. Read the error. Playwright is telling you exactly which check failed attached, visible, stable, enabled, or covered. The fix is almost always one line that addresses the actual condition, not a number that masks it.
sleep() was always a guess. Auto-waiting is Playwright already knowing how to wait. Let it.

‍

In today’s digitally connected world, accessibility is not a “nice to have,” but a legal and moral imperative. The Web Content Accessibility Guidelines (WCAG) offer a set of guidelines to assist in making your website accessible and inclusive for all users, including those with disabilities.

In our earlier blogs, we learned how accessibility laws across the countries comply with the WCAG and how it helps in selecting the correct WCAG level (A, AA, or AAA) for your website.

But how do we handle situations where new versions are released, which might have new features to comply with?

If we have a website that has already been developed according to WCAG 2.0, it is essential to understand how WCAG 2.1 and 2.2 have updated their features compared to their earlier version.

Accessibility core principles and conformance level

Accessibility principles ensure content is perceivable, operable, understandable, and robust so all users can access and use it.

WCAG defines three levels: Level A (basic requirements that remove major barriers), Level AA (standard level with better usability, like good contrast and readable text), and Level AAA (highest level with advanced accessibility features, though not always practical for all content).

WCAG 2.0: the foundation of web accessibility (2008)

WCAG 2.0 acts as the base with 61 success criteria covering technology support, cognitive, motor skill, hearing, and visual disabilities. It ensures the content is accessible in various ways, navigation is possible with different input devices, the information is understandable, and the website works with different browsers.

WCAG 2.1: accessibility for mobile and low vision (2018)

Smartphones and tablets have transformed the way users interact with the internet, especially with the introduction of WCAG 2.1, which is an extension of WCAG 2.0, with 17 new success criteria that focus on mobile devices, low vision, and cognition. It ensures that using a website is touch-friendly, content is legible even if it is zoomed, and it is simple to use.

WCAG 2.2: latest update (2023)

WCAG 2.2, published in October 2023, introduces 9 new success criteria for enhanced cognitive accessibility, keyboard operation, and user support. It has been improved for easier task completion, with all functionality being easily accessible with the keyboard and help functionality being easy to find.

‍

Comparison between WCAG 2.0, 2.1, and 2.2 (examples)

‍

Legal compliances

Accessibility compliance ensures that digital content is compliant with existing laws and regulations that protect the rights of people living with disabilities, which are mostly aligned with WCAG guidelines at varying levels. Most compliance is aligned with WCAG 2.1 AA, but now many organizations are moving towards WCAG 2.2 AA as a best practice.

Why upgrade? the benefits are clear

Making your website more accessible is a good idea for many reasons. It means your website will be accessible to more people, it will help you comply with the law, it will improve the overall usability of your website, and it will help create a more inclusive design.

How much effort does it take to upgrade from WCAG 2.0 to 2.1 or 2.2?

If your site meets WCAG 2.0 Level AA, upgrading to 2.1 or 2.2 is easier than starting over, but it still takes planning and time. Here’s what to expect:

Step 1: Understand the new guidelines

Start by reviewing the detailed new guidelines in WCAG 2.1 and 2.2 to understand how they apply to your site.

Step 2: Run automated accessibility tests (medium effort)

Use tools like Pa11y, Axe, or Lighthouse (set to WCAG 2.1/2.2) to spot common issues, but remember they catch only 30–50% of actual problems. 

Step 3: Start manual testing (this is where the main effort begins)

• Mobile-friendly design: Ensure buttons and links are large enough, and gestures work easily with one finger.
  
  
• Cognitive-friendly interfaces:  Ensure forms, instructions, and time limits are clear, avoiding complex language or confusing layouts.
  
  
• Zoom and visibility: Zoom to 400% to ensure content stays accessible, and check contrast on icons, buttons, and inputs for visibility.
  
  
• Keyboard use: Navigate the site with just a keyboard, ensuring all elements are accessible and focus moves logically.
• Assistive Technology: Ensure your website is compatible with assistive devices such as screen readers, voice control, etc. This is to ensure that all users can use your website effectively.

Step 4: Document what’s missing

Group issues by the WCAG 2.1 or 2.2 guideline that they violate to prioritise and fix them efficiently

Step 5: Fix and improve (this part varies a lot)

• Development fixes: Some fixes, like tweaking the CSS for button sizes, may be easy and take a short time. However, other fixes, like implementing custom gestures and handling complex forms, may take a while.
  
  
• Design updates: Design teams may need to adjust components, colours, and layouts to enhance mobile usability and visual accessibility.
  
  
• QA & retesting: Once the fixes have been implemented, thorough testing should be carried out, including the use of automated tools and manual testing, to ensure that the fixes have worked and there are no new issues created in the process.
  
  
• Tooling: Accessibility tools should be integrated or updated within the development process, as well as within the QA process, including integration with CI/CD pipelines.

Factors that influence your upgrade effort

The effort needed to upgrade to WCAG 2.1 or 2.2 isn’t just about the guidelines. It also depends on how your application is built and how your team works. Here are some factors that can greatly affect the workload:

• App complexity: Upgrading a dynamic website is more complex than a static one. This is because of the complex interactions in a dynamic website.
  
  
• Code quality: Good, clean HTML and CSS semantics help with upgrades. Bad code may need fundamental changes before new guidelines can be applied.
  
  
• Team experience: Teams familiar with accessibility, ARIA, keyboard navigation, and contrast can upgrade faster, while inexperienced teams need more time to learn and implement.
  
  
• Testing automation: Adding automated accessibility tests to your CI/CD catches issues early and prevents regressions, speeding up the process.
  
  
• Design system maturity: Applications with more mature and accessible design systems can upgrade the entire application by updating a few key components.

A strong foundation makes upgrades easier, while disorganised code, little accessibility knowledge, or no testing framework will require more time to fix.

Conclusion: accessibility as an ongoing journey

The shift from WCAG 2.0 to 2.1 and now 2.2 shows that accessibility is not a one-time task. It’s an ongoing effort. This effort changes as technology advances and as we learn more about how people use digital content.

The best approach is to combine smart tools with human judgment. Use automated checkers to catch common issues. And support them with thorough manual testing. Most importantly, create a culture where accessibility is part of your team’s mindset and daily work.

Ultimately, accessibility is more than just meeting compliance rules. It’s about building digital experiences that are inclusive, easy to use, and empowering for everyone.

‍

Have you ever wondered how companies make sure their products or systems don’t shatter the rules? That’s where Compliance testing is like the last check of the rulebook. 

Compliance testing acts as the final safeguard, ensuring your product not only works but also meets the legal, security, and accessibility standards that protect businesses from costly risk. This procedure is all about making sure that everything follows the correct rules or industry standards, whether it's making sure that an app keeps your data safe under laws like GDPR or making sure that new software satisfies safety standards. 

Keeping things safe, equitable, and reliable is more important than just checking boxes. Here are some reasons why creating safe and reliable digital products requires compliance testing.

Let's explore what compliance testing is, the requirements you must be aware of, how QA teams approach it, and how it could prevent some quite serious circumstances.

What is Compliance testing in web applications?

Compliance testing is the process of verifying that a web application complies with industry rules, external laws, internal security guidelines, and accessibility standards

It verifies that your application:

• Protects personal data under regulations like GDPR/CCPA.
• Validates accessibility for users with disabilities (WCAG).
• Prevents security risks by addressing known vulnerabilities.
• Aligns with regional laws (e.g., EU Cookie Law)..

QA teams do this important testing to ensure functional, security, and performance testing to make sure nothing is overlooked before a product launches.

Types of Compliance standards you should know

Every industry and location has different requirements for compliance. These are a few of the more popular ones:

• GDPR (General Data Protection Regulation): EU law governs the processing of personal information for EU citizens.Avoids heavy fines in the EU (up to 4% of global turnover) for mishandling personal data. Read More 
• CCPA (California Consumer Privacy Act): California law giving consumers rights over personal data collection and usage. Read More
• PCI-DSS (Payment Card Industry Data Security Standard):  Safeguards credit card data, preventing breach costs and customer churn.. Read More
• HIPAA (Health Insurance Portability and Accountability Act):  Regulates healthcare data in the United States. Read More 
• WCAG (Web Content Accessibility Guidelines): Ensures digital content is accessible to individuals with disabilities. Read More 
• SOX (Sarbanes-Oxley Act): Enforces financial data transparency, protecting against fraud and legal penalties.
• ISO 27001: Globally recognised information security framework that enhances credibility and client confidence. Read More

Key areas QA teams cover in Compliance testing

QA teams should mainly focus on applications that don't expose the business to legal or compliance risks.

 Key focus areas include:

• Data privacy: Verifying sensitive user data is encrypted and never exposed.
• Consent management: Making sure user consent is properly obtained and recorded (e.g., cookie banners).
• Access controls: Verifying that only authorised users can access restricted features or data.
• Audit trails: Checking that system logs record all relevant actions for traceability.
• Accessibility: Confirming the application is usable for disabled users.
• Localisation compliance: Ensures adherence to regional laws (e.g., EU, U.S., APAC).

By integrating these tests at an early stage of development, QA prevents major issues from making it into production.

Tools for Compliance testing

In the market, there are many tools available that can be used for compliance Testing.

Some popular tools used by QA teams include:

• Accessibility: Axe DevTools, WAVE, Screen Readers, Keyboard Navigation.
• Security & privacy: OWASP ZAP (security), Postman (API security headers), Cookiebot (consent).
• Performance & SEO: Google Lighthouse (accessibility, performance, SEO compliance).

QA workflow 

Real-world examples of Compliance testing in action

Here are a few examples of how compliance-focused testing can be done and has saved companies from major trouble:

1. Accessibility Compliance (WCAG 2.1)

   Test case:

• Test: Verify that images on the homepage have alternative text (alt attribute).
  
• Steps:
  - Use a tool like Axe DevTools, WAVE, or Lighthouse.

Observation: Images have alt text.

Risk: Missing alt text can lead to non-compliance, accessibility lawsuits, and the exclusion of users

2. GDPR and Cookie Consent Compliance

 Test case:

• Test: Verify that the site presents a cookie consent banner on the first visit.
  
• Steps:
  - Open the website in incognito mode.
  - Observe whether any tracking cookies (e.g., Google Analytics) are set before the user accepts consent.
  

Observation:

• A cookie banner appears at the bottom of the screen.
  
• Use the browser DevTools > Application tab > Cookies to check if third-party cookies are dropped before consent is given.
  

Bug:

• If Google Analytics cookies (like _ga) are placed on a user’s device before they click “Accept,” it goes against GDPR rules.

Risk: GDPR violation with potential fines up to €20M.

Screenshot:

 3. Privacy Policy & Form Data Handling

 Test case:

• Test: Verify if the forms (e.g., newsletter signup or contact form) link to the Privacy Policy and collect only necessary data.
  
• Steps:
  - Go to Get Involved.
  - Fill and submit a form.
  

Observation:

• Check if there’s a visible privacy notice or checkbox indicating user consent.
• Use browser DevTools > Network to confirm that submitted data is encrypted (HTTPS) and goes to the right domain.
  

Bug:

• Form lacks an explicit consent checkbox.
• No immediate privacy policy link near the form.

Risk: The absence of a consent checkbox and policy link can lead to data misuse, potential lawsuits, and loss of user trust.

4. Keyboard Navigation and Screen Reader Support

 Test case:

• Test: Verify that users can navigate menus and buttons using just the keyboard.
  
• Steps:
  - Use Tab/Shift+Tab to move through menus and buttons.
  - Observe if focus states are visible.
  

Observation:

• Tab navigation works with elements like buttons and links.

Risk: Ineffective navigation may result in accessibility violations and exclude users with disabilities.

 5. Security Headers & HTTPS

Test case:

• Test: Verify that the website uses HTTPS and includes security headers.
  
• Steps:
  - Use tools like SecurityHeaders.com or SSL Labs.
  

Observation:

• Site uses HTTPS.

Risk: The absence of security headers may expose the application to vulnerabilities, including XSS and man-in-the-middle attacks

Screenshot:

‍

 6. Language Declaration & HTML Validation

 Test case:

• Test: Verify that the correct language is declared in the HTML tag.
  
• Steps:
  - View page source and check for <html lang="en">.
  

Observation:

• Language is properly declared — good for screen readers and SEO.

Risk: Missing Language attributes can lead to poor SEO and screen reader misinterpretation 

Screenshot:

Best practices for Compliance testing

These are best practices if you want to make compliance testing an efficient part of the QA process.

• Start testing for compliance at the very beginning of the development cycle.
• Use tools to quickly catch high-risk issues.
• Use checklists (GDPR, WCAG, PCI-DSS, HIPAA) to guide testing.
• QA should collaborate closely with legal teams, product owners, and developers.
• Stay Updated: Compliance rules are always changing, so stay up to date or participate in pertinent forums.
• Maintain Records: Keep Documents Up to date. For upcoming audits, preserve thorough logs, reports, and screenshots of test results.

Conclusion

In today’s digital economy, compliance is not optional — it’s a competitive advantage.

Products that prioritise compliance:

• Win user trust.
  
• Reduce legal and financial risks.
  
• Stay resilient and future-proof.

Compliance is ultimately about trust. User data is extremely valuable in today's world, and people are concerned about how their personal information is handled. 

Compliance testing is crucial for this reason: it protects your platform from potential legal problems and ensures that users can trust it.

Make compliance testing an integral part of your QA strategy, and you’re not just following rules; you’re building trust and protecting your business.
‍

‍

‍

AI systems are changing at lightning speed, constantly learning and evolving. Testing them with our traditional software testing methods is no longer enough. These AI models produce different outputs every time, never the same and verifying them each time is nearly impossible.

Yet testing AI systems for reliability, accuracy, and consistency is critical, especially as they are increasingly embedded into decision-making processes.

It’s a challenging task since it generates non-deterministic outputs (unclear), and manual verification and test case maintenance every time isn’t possible. To overcome this, metamorphic testing has come into the picture as a powerful strategy in AI testing

What is Metamorphic testing? 

Metamorphic testing is used when the output of a system is not consistent over time. MT maps relationships between input and output, and these relationships are referred to as Metamorphic Relations (MRs). 

Metamorphic Relations define how output should change (or stay the same) when the input is transformed in a specific, meaningful way. Instead of verifying the same answers, it validates output differences logically.

So instead of saying “ Input A should give Output X ”, we say “ If I change input A in a specific way, the output should increase/decrease/stay the same.”

To illustrate this behaviour, let me give an example : 

We have an LLM bot, and let's check the output consistency for the same question answer but with differently phrased input prompts : 

All of these essentially ask the same question, just worded differently. The model must have Delhi in its answers. The goal isn’t to get the same sentence back, but to see whether the core answer remains logically consistent.

So if I use different synonyms or just change my sentence, but the meaning remains the same, the AI model must understand this and change the outputs relatively

Let's check with another example :

Imagine you are making lemonade for 2 people, you know exactly how to make it, but now if you have to make lemonade for 4 people, you will simply double the ingredients. We may not know exactly how many glasses will be produced, but we expect more lemonade because the input has increased predictably. That’s how metamorphic relations work; it’s not about the exact output, but about how the output should change with the input.

Understanding metamorphic relations 

1. Increase

When you increase or add something important to the input, the output should also increase.

This is similar to the above lemonade example

Consider an example of Loan return Risk analysis AI of a Bank, the model must predict the risk 

Person A has a Salary of 80000 with no debts 

Person B, with a Salary of 80000, has 3 loans.

Expected output: Adding loans should make the person riskier, so the score should increase.

‍

‍

If the model does not give appropriate risk scores, then it is behaving incorrectly.

2. Decrease

When you reduce something important or add a positive signal, the output should go down. 

Consider a healthcare AI model that predicts the risk of developing diabetes.

So the risk factor can be based on Healthier habits being Lower risk, and Unhealthy habits being Higher risk.

Person X is eating Healthy food with regular exercise. 

The risk score must decrease

We don’t need to know what the exact risk score should be; we just know that eating healthy and exercising should lower it.
That’s the beauty of metamorphic testing: it checks whether your AI behaves logically, even when the “correct” answer is unknown.

3. Invariance

When you change the input in a way that does not affect its meaning, the output should stay the same.

Examples of Search engines

If you search : 

"Give me Chemist shops near me"
Or
"Give me nearby pharmacies"

‍

Expected Output : 

Both must give the same search result. 

How Metamorphic testing works

Here’s how the testing process works in practice. We can consider a Movie Review AI model to understand each step clearly. 

1. Identify critical properties of your model

This phase is for understanding what key behaviours of your model are. 

Our Movie Review AI model will have the following behaviours :

• More positive reviews (with more praise) should get a higher rating
• A review with added negative words should get a lower rating
• Reviews with rephrased language should get the same rating
  

2. Define your Metamorphic relations (MRs)
‍

State how input-output should behave under changes.

Example :
‍