Playwright steps and attachments

The steps and files of every attempt go with its result in Probara. Write steps with Playwright’s own test.step, turn on screenshots, videos and traces in use as usual, and the reporter sends them: nothing needs to be turned on in the reporter.

Steps

The steps of a test show under its result in Probara, nested as they ran, each with its status, duration and error.

import { expect, test } from '@playwright/test';

test('checks out', async ({ page }) => {
  await test.step('Open the cart', async () => {
    await page.goto('https://shop.example.com/cart');
  });
  await test.step('Pay', async () => {
    await test.step('Enter the card', async () => {
      await page.getByLabel('Card number').fill('4242 4242 4242 4242');
    });
    await test.step('Confirm', async () => {
      expect(await Promise.resolve('declined')).toBe('paid');
    });
  });
});

[
  {
    "status": "failed",
    "steps": [
      { "action": "Open the cart", "status": "passed" },
      {
        "action": "Pay",
        "status": "failed",
        "steps": [
          { "action": "Enter the card", "status": "passed" },
          { "action": "Confirm", "status": "failed" }
        ]
      }
    ]
  }
]

What ran	In the result
A `test.step`	A step, with its children
A hook (`beforeEach`, `afterEach`, `Before Hooks`…) that ran a `test.step`	A step named after the hook, holding those steps
A hook or a fixture without a `test.step`	Nothing: its `test.step` children, if any, move up
`expect` calls, Playwright API calls (`page.click`…), fixtures	Nothing: they are Playwright’s internals, not your steps

A step’s status is failed when it threw (its parents fail with it), skipped for test.step.skip(), and passed otherwise. Its error (message and stack) goes with it, and its duration is Playwright’s.

Hooks

A hook that runs steps shows as a step of its own, so setup and teardown stay apart from the test:

import { test } from '@playwright/test';

test.beforeEach(async () => {
  await test.step('Sign in', async () => {});
});

test('lists the orders', async () => {
  await test.step('Open the orders', async () => {});
});

[
  {
    "steps": [
      {
        "action": "Before Hooks",
        "steps": [{ "action": "beforeEach hook", "steps": [{ "action": "Sign in" }] }]
      },
      { "action": "Open the orders" }
    ]
  }
]

Steps of a new case: `probara.step()`

A result’s steps record what ran. A case’s steps are its specification: an action, the expected result and the test data. Declare them with probara.step(action, expected, data) as the title of a test.step; the step runs as usual, and the case the report creates takes it:

import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('refunds an order', async () => {
  await test.step(
    probara.step('Open the order', 'The order is shown', 'order=1042'),
    async () => {},
  );
  await test.step(probara.step('Refund it', 'The order is refunded'), async () => {
    await test.step('Confirm the dialog', async () => {});
  });
});

[
  {
    "steps": [
      { "action": "Open the order", "expected": "The order is shown", "data": "order=1042" },
      {
        "action": "Refund it",
        "expected": "The order is refunded",
        "steps": [{ "action": "Confirm the dialog" }]
      }
    ],
    "case": {
      "steps": [
        { "action": "Open the order", "expected": "The order is shown", "data": "order=1042" },
        { "action": "Refund it", "expected": "The order is refunded" }
      ]
    }
  }
]

probara.step('Pay') returns Pay [probara:1]: Playwright’s own reports show that title, and the reporter reads the short reference and sends Pay.
The case steps are the probara.step() steps that ran and are not inside another one, in the order they started. A plain test.step is a step of the result only: nested helpers such as Confirm the dialog are how a test works, not what it specifies.
Like every case field, they apply only when the report creates the case (created cases only).

Attachments

Every file of an attempt goes to its result: what Playwright records on its own, and what a test attaches.

What Playwright records

Turn them on in use, as usual; the reporter uploads what they produce:

use: {
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
  trace: 'retain-on-failure',
},

Playwright	The file in Probara
`screenshot`	`test-failed-1.png` (and the others Playwright keeps)
`video`	The `.webm` video of the attempt
`trace`	`trace.zip`: open it with `npx playwright show-trace`
A failed attempt (Playwright 1.51 and later)	`error-context.md`: the page snapshot at the failure
`toHaveScreenshot()` failures	The expected, actual and diff images

Each attempt has its own files, so a retried test uploads the files of every attempt, each to its own result.

`testInfo.attach()`

Playwright’s own API works as it is, with a file or a body:

import { test } from '@playwright/test';

test('creates an order', async ({ page }, testInfo) => {
  await testInfo.attach('order', {
    body: JSON.stringify({ id: 1042, total: 25 }),
    contentType: 'application/json',
  });
  await testInfo.attach('checkout', { body: await page.screenshot(), contentType: 'image/png' });
});

The files are named after the attachment, with the extension of their content type:

order.json application/json
checkout.png image/png

`probara.attach()`

probara.attach() does the same from anywhere a test runs (a helper, a fixture, a page object), without passing testInfo around. Give it a file on disk or a body, and await it:

import { writeFile } from 'node:fs/promises';
import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('prints the receipt', async () => {
  const path = test.info().outputPath('receipt.txt');
  await writeFile(path, 'Total: 25.00');
  await probara.attach({ name: 'receipt.txt', path });
  await probara.attach({ name: 'log', body: 'paid in 1.2 s', contentType: 'text/plain' });
});

receipt.txt text/plain
log.txt text/plain

Field	With `path`	With `body`
`name`	Required: the name in Probara	Required: the name in Probara
`path`	The file to attach (Playwright copies it)	—
`body`	—	A string, a `Buffer` or a `Uint8Array`
`contentType`	Optional: guessed from the file name	Required, such as `text/plain`

A string body is sent as it is; pass a Buffer for binary content. A file that cannot be attached (it does not exist, say) is a [probara] warning on the test’s stderr, and the test goes on.

Files of a step

A file attached while a test.step runs (testInfo.attach() or probara.attach()) goes to that step, and Probara shows it under the step, where it happened:

import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('uploads an avatar', async ({ page }) => {
  await test.step('Choose the file', async () => {
    await probara.attach({
      name: 'avatar',
      body: await page.screenshot(),
      contentType: 'image/png',
    });
  });
});

A file attached inside a hook or a fixture that is not a test.step goes to the nearest test.step around it, or to the result.

File names

A file keeps its own name (test-failed-1.png, trace.zip). Playwright stores what testInfo.attach() copies under a hashed name (receipt-<sha1>.txt), and a merged blob report stores every file that way; such a file is named after its attachment instead, with the file’s extension when the name has none: receipt.txt, avatar.png.

Test output

With captureOutput: true (PROBARA_CAPTURE_OUTPUT=true), each attempt’s stdout and stderr are attached as stdout.log and stderr.log, when there is any:

reporter: [['@probara/playwright-reporter', { captureOutput: true }]],

import { test } from '@playwright/test';

test('prints what it does', async () => {
  console.log('Opening the cart');
  console.error('Slow response: 2.1 s');
});

stdout.log text/plain
stderr.log text/plain

Limits and what is skipped

Limit	What happens
32 MiB per file	A larger file is skipped with a warning
20 files per result	The files after the first 20 are skipped, with one warning
Images over 10 MiB or 8192 px wide or tall	Skipped with a warning (Probara converts images to WebP)
Executables and scripts	Skipped: Probara refuses their content types
A missing, unreadable or empty file	Skipped with a warning
A result that recorded nothing	Its files are not uploaded

Warnings name the file, and the log ends with how many files were attached, skipped and failed. uploadAttachments: false (PROBARA_UPLOAD_ATTACHMENTS=false) uploads none. Every result with files costs two more requests against the organization’s rate limit.

Files upload after their report is recorded, while the run is still open, then the run is closed. Keep Playwright’s output folder (test-results/) until the reporter is done: it uploads from there.

Playwright versions

Playwright	What changes
1.51 and later	A failed attempt also uploads `error-context.md`
1.50 and later	Files go to the step they were attached in; `test.step.skip()` is a `skipped` step
1.42 to 1.49	Playwright does not say which step a file belongs to: every file stays with the result; `test.step.skip()` does not exist

Nothing fails on an older Playwright of the supported range: those details are just not there.

Common problems

A step is missing. Only test.step makes steps. A helper function that is not wrapped in test.step shows nothing; neither do expect calls.
Steps are flattened. A step started without await ends before its children: await every test.step.
The case has no steps. Only probara.step() titles declare case steps, and only for a case the report creates. An existing case keeps its own.
An attachment is missing. See troubleshooting.

Next: results and retries.