Negative Testing in Playwright with TypeScript

Using Try/Catch

One of the key ways of building a champion automated test suite is negative testing - making sure that your application gracefully handles errors, exceptions, and bad user behavior. This is a good way for automation to add value with testing as negative testing isn't something that manual QA does on a regular bases.

When working with Playwright and TypeScript, there isn't a direct equivalent to pytest.raises() in Python for expecting exceptions. But that doesn't mean you're out of luck.

With TypeScript, the tried-and-true try/catch pattern becomes a powerful way to validate that your code throws errors when it's supposed to.

Scenario: Navigating to an Invalid URL

Say you want to verify that navigating to a malformed URL triggers the appropriate error. Here's how you can do it using Playwright with TypeScript:

import { test, expect } from '@playwright/test';
test('should throw an error when navigating to an invalid URL', async ({ playwright }) => {
  let errorCaught = false;
  let errorMessage = '';
  try {
    // Attempt to navigate to an invalid URL, which should throw an error
    await playwright.chromium.launch().then(async browser => {
      const page = await browser.newPage();
      await page.goto('httpy://www.whitehouse.gov'); // Intentionally bad protocol
      await browser.close();
    });
  } catch (error) {
    errorCaught = true;
    errorMessage = error instanceof Error ? error.message : String(error);
  }
  // Assert that an error was caught
  expect(errorCaught).toBe(true);
  // Optionally, check the error message contains expected content
  expect(errorMessage).toContain('net::ERR_ABORTED');
});

Why Use try/catch Instead of Custom Matchers?

Playwright doesn't provide a built-in matcher for error expectations like pytest.raises. However, try/catch blocks are fully supported and give you total control over the error message, how it's caught, and what behavior you want to assert afterward.

You can go beyond just checking that an error occurred-you can validate specific error types, compare error messages, and decide whether to re-throw or suppress the error.

Tips for Cleaner Code

If you find yourself doing this often, you can extract the logic into a helper function:

async function expectError(fn: () => Promise<void>, expectedMessagePart: string) {
  let errorCaught = false;
  let errorMessage = '';
  try {
    await fn();
  } catch (error) {
    errorCaught = true;
    errorMessage = error instanceof Error ? error.message : String(error);
  }
  expect(errorCaught).toBe(true);
  expect(errorMessage).toContain(expectedMessagePart);
}

And use it like this:

test('should throw an error for bad URL', async ({ playwright }) => {
  await expectError(async () => {
    const browser = await playwright.chromium.launch();
    const page = await browser.newPage();
    await page.goto('httpy://www.whitehouse.gov');
    await browser.close();
  }, 'net::ERR_ABORTED');
});

Final Thoughts

try/catch isn't just a fallback-it's a flexible and explicit way to perform negative testing in Playwright. It gives you visibility into what exactly went wrong and control over how to validate it. Whether you're validating error messages, failed network calls, or unhandled exceptions, try/catch should be part of your Playwright testing toolkit.

In Automation Consistency Is Key

PlayWright Flaky Test Check

In Automation Consistency Is Key

Tests that pass reliably instill confidence in your codebase, while unpredictable ones can erode trust and waste valuable time. Enter flaky tests - those pesky tests that don't always pass on the first try, even when the underlying code seems fine.

Fortunately, Playwright, a powerful browser automation framework, offers a handy tool to help QA automation teams tackle this challenge head-on: the --fail-on-flaky-tests option.

What Are Flaky Tests?

Flaky tests are the wild cards of the testing world. They might pass on one run and fail on the next, often due to timing issues, external dependencies, or subtle environmental differences.

While they don't necessarily indicate a bug in the application, they do signal a problem in the test itself - something that might need a closer look to ensure reliability.

For QA teams, flaky tests can be a headache. They slow down workflows, muddy the waters of test reporting, and make it harder to trust the results of a test suite. That's where Playwright steps in with a feature designed to shine a spotlight on these troublemakers.

Playwright's Flaky Test Detection

Playwright provides a built-in mechanism to identify and flag flaky tests, helping teams address them proactively. By enabling certain configurations, Playwright can detect when a test exhibits inconsistent behavior - passing sometimes and failing others - and mark it as flaky.

This doesn't just sweep the issue under the rug; it gives teams actionable insights to refine their tests.

One particularly useful option in Playwright's arsenal is --fail-on-flaky-tests. By default, this option is set to false, meaning Playwright will note flaky tests but won't treat them as failures in the overall test run.

However, when you flip this switch to true, Playwright takes a stricter stance: if any test is flagged as flaky, the entire test suite fails. This forces the team to confront the issue immediately rather than letting it linger.

How to Use --fail-on-flaky-tests

To leverage this feature, simply add the --fail-on-flaky-tests flag when running your Playwright tests from the command line. For example:

            
                npx playwright test --fail-on-flaky-tests
            
        

PlayWrite Date format

Typescript vs Python

Amazing how easier it is to get the date format for saving screenshots. Why is it so complicated in Typescript?

Here's getting the date in YYYY-MM-DD-HH-MM format:

Typescript

function getCurrentDateTimeFormatted(): string {
  const now = new Date();
  const year = now.getFullYear();
  const month = String(now.getMonth() + 1).padStart(2, '0'); // Months are 0-indexed
  const day = String(now.getDate()).padStart(2, '0');
  const hours = String(now.getHours());
  const minutes = String(now.getMinutes());
  return `${year}-${month}-${day}-${hours}-${minutes}`;
}
// Example usage
const formattedDateTime = getCurrentDateTimeFormatted();

Python

from datetime import datetime
datefile = datetime.now().strftime("%Y-%m-%d-%H-%M")

Parametrization in PlayWright

Test Uptime Status of Multiple Sites

Yesterday, I showed how to use Parametrization in Pytest. Here's an example of how you would run that same code in PlayWright with TypeScript:

This code checks to make sure the four websites are up and running. This is just a quick sanity test, it doesn't do any critical path testing.

import { test, expect } from '@playwright/test';
// List of websites to test
const WEBSITES = [
  "https://www.company.com",
  "https://qa1.company.com",
  "https://qa2.company.com",
  "https://stage.company.com",
];
// Configure Playwright to run in headless mode globally
test.use({ headless: true });
test(`Check Websites Status`, async ({ page }) => {
// Iterate over websites to create a test for each
    for (const website of WEBSITES) {
        test(`Check if ${website} is up and running`, async ({ page }) => {
            try {
                // Attempt to load the website
                await page.goto(website, { waitUntil: 'domcontentloaded' });
                // Check if page title exists and is not empty
                const title = await page.title();
                expect(title).not.toBe('');
                // Check if body element exists
                const body = page.locator('body');
                await expect(body).toBeVisible();
                // Log success
                console.log(`✓ ${website} is up and running (Title: ${title})`);
            } catch (error) {
                // Oh the Horror: Fail the test with a detailed message
                throw new Error(`Website ${website} failed: ${error.message}`);
            }
        });
    }
})

Get Random Line from a File

Add some variety to your PlayWright tests

In PlayWright, you can easily get the contents of a file to include in a form. This is the getRandomLineFromFile function that I use to open up a local file and get a random line:

async function getRandomLineFromFile(filePath: string): Promise {
    try {
        // Read file content and split into lines
        const fileContent = await fs.readFile(filePath, 'utf8');
        const lines = fileContent.split('n').filter(line => line.trim() !== ''); // Remove empty lines
        
        if (lines.length === 0) {
            throw new Error('File is empty');
        }
        
        // Get random line
        const randomIndex = Math.floor(Math.random() * lines.length);
        return lines[randomIndex].trim();
    } catch (error) {
        console.error(`Error reading file: ${error}`);
        throw error;
    }
}

I would use this to open up a file that has a random sentence to fill in a feedback form. Here's an example PlayWright with TypeScript entry that I would use:

test('Random Feedback Form Test', async ({ page }) => {
    const filePath = '/Users/dict/srand.txt';
    const randomLine = await getRandomLineFromFile(filePath);
    await page.goto('https://www....');
    await page.waitForLoadState("networkidle");
    await page.getByLabel('name').fill('Chris Ryan');
    await page.getByLabel('comment').fill(testText);
....
})

You could also do this to randomize names, locations etc. This is just handy to have when you want to add some variety to a test run.

XPath with Playwright page.locator

A Practical Guide

Playwright page.locator method is designed to find elements dynamically, with built-in support for multiple selector types - including CSS, text, and yes, XPath. While CSS selectors are great for straightforward queries, XPath shines when you need more flexibility or when dealing with complex DOM structures.

Here's why XPath might be your go-to:

• Structural Navigation: XPath lets you traverse the DOM based on relationships (e.g., parent, sibling, child) rather than just classes or IDs.

• Attribute Precision: Target elements by any attribute, not just class or id.

• Text-Based Selection: Easily find elements containing specific text, even partial matches.

• Dynamic Pages: XPath can handle scenarios where CSS selectors falter, like when class names are auto-generated or unpredictable.

Playwright's page.locator makes XPath a first-class citizen, so let's see it in action.

Getting Started with page.locator and XPath

The syntax for using XPath in page.locator is simple: prefix your XPath expression with xpath= or use the double-slash shorthand //. Here's the basic structure:

await page.locator('xpath=//tag[@attribute="value"]').click();

Playwright will evaluate the XPath expression and return a Locator object, which you can then interact with (e.g., click(), fill(), textContent()).

Practical Example

Let's walk through a real-world scenario where XPath and page.locator save the day.

Targeting an Element by Attribute

Imagine a login form with a button lacking a unique ID or class:

<button type="submit" data-test="login-btn">Sign In</button>

With XPath, you can target it by its data-test attribute:

const { test } = require('@playwright/test');
test('click login button', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.locator('xpath=//button[@data-test="login-btn"]').click();
});

The //button[@data-test="login-btn"] means "find any <button> element with a data-test attribute equal to login-btn."

When to Avoid XPath

While XPath is powerful, it’s not always the best choice:

• Simple Selectors: Use CSS for #id or .class - it’s faster and more readable.
• Dynamic IDs: If attributes change frequently, text-based or role-based selectors (role=) might be more stable.
• Maintenance: Complex XPath expressions can become brittle if the DOM structure shifts.

PlayWright URL Scraping

Sample Code to get all URLs

While experimenting with Playwright this week, I put together a script that grabs all the URLs from a website and writes them to a file. Here's the code that I finally came up with:

test('Extract and save my URLs from cryan.com', async ({ page }) => {
  // Navigate to the target URL
  await page.goto('https://www.cryan.com');
  // Extract all  tags with href attributes
  const links = await page.$$eval('a[href]', (anchors) => anchors.map((a) => a.getAttribute('href')));
  // Remove any relative URLs or empty strings
  const filteredLinks = links.filter((link) => link?.startsWith('http') && link.trim() !== '');
  // Save the unique URLs to a file
  const uniqueLinks = [...new Set(filteredLinks)];
  await fs.promises.writeFile('/Users/cryan/Desktop/url.txt', uniqueLinks.join('n'), 'utf8');
  // Assertions to validate extracted URLs
  expect(uniqueLinks.length).toBeGreaterThan(0); // Assert at least one URL found
});