# Playwright steps and attachments

> How @probara/playwright-reporter sends test.step trees, hooks, declared case steps, screenshots, videos, traces and your own attachments to Probara.

The steps and files of every attempt go with its result in Probara. Write steps with Playwright's
own `test.step`, turn on screenshots, videos and traces in `use` as usual, and the reporter sends
them: nothing needs to be turned on in the reporter.

## Steps

The steps of a test show under its result in Probara, nested as they ran, each with its status,
duration and error.

```ts
// tests/checkout.spec.ts
import { expect, test } from '@playwright/test';

test('checks out', async ({ page }) => {
  await test.step('Open the cart', async () => {
    await page.goto('https://shop.example.com/cart');
  });
  await test.step('Pay', async () => {
    await test.step('Enter the card', async () => {
      await page.getByLabel('Card number').fill('4242 4242 4242 4242');
    });
    await test.step('Confirm', async () => {
      expect(await Promise.resolve('declined')).toBe('paid');
    });
  });
});
```

```json
[
  {
    "status": "failed",
    "steps": [
      { "action": "Open the cart", "status": "passed" },
      {
        "action": "Pay",
        "status": "failed",
        "steps": [
          { "action": "Enter the card", "status": "passed" },
          { "action": "Confirm", "status": "failed" }
        ]
      }
    ]
  }
]
```

| What ran                                                                     | In the result                                            |
| ---------------------------------------------------------------------------- | -------------------------------------------------------- |
| A `test.step`                                                                | A step, with its children                                |
| A hook (`beforeEach`, `afterEach`, `Before Hooks`...) that ran a `test.step` | A step named after the hook, holding those steps         |
| A hook or a fixture without a `test.step`                                    | Nothing: its `test.step` children, if any, move up       |
| `expect` calls, Playwright API calls (`page.click`...), fixtures             | Nothing: they are Playwright's internals, not your steps |

A step's status is `failed` when it threw (its parents fail with it), `skipped` for
`test.step.skip()`, and `passed` otherwise. Its error (message and stack) goes with it, and its
duration is Playwright's.

### Hooks

A hook that runs steps shows as a step of its own, so setup and teardown stay apart from the test:

```ts
// tests/orders.spec.ts
import { test } from '@playwright/test';

test.beforeEach(async () => {
  await test.step('Sign in', async () => {});
});

test('lists the orders', async () => {
  await test.step('Open the orders', async () => {});
});
```

```json
[
  {
    "steps": [
      {
        "action": "Before Hooks",
        "steps": [{ "action": "beforeEach hook", "steps": [{ "action": "Sign in" }] }]
      },
      { "action": "Open the orders" }
    ]
  }
]
```

### Steps of a new case: `probara.step()`

A result's steps record what ran. A **case's** steps are its specification: an action, the
expected result and the test data. Declare them with `probara.step(action, expected, data)` as the
title of a `test.step`; the step runs as usual, and the case the report creates takes it:

```ts
// tests/refund.spec.ts
import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('refunds an order', async () => {
  await test.step(
    probara.step('Open the order', 'The order is shown', 'order=1042'),
    async () => {},
  );
  await test.step(probara.step('Refund it', 'The order is refunded'), async () => {
    await test.step('Confirm the dialog', async () => {});
  });
});
```

```json
[
  {
    "steps": [
      { "action": "Open the order", "expected": "The order is shown", "data": "order=1042" },
      {
        "action": "Refund it",
        "expected": "The order is refunded",
        "steps": [{ "action": "Confirm the dialog" }]
      }
    ],
    "case": {
      "steps": [
        { "action": "Open the order", "expected": "The order is shown", "data": "order=1042" },
        { "action": "Refund it", "expected": "The order is refunded" }
      ]
    }
  }
]
```

- `probara.step('Pay')` returns `Pay [probara:1]`: Playwright's own reports show that title, and
  the reporter reads the short reference and sends `Pay`.
- The case steps are the `probara.step()` steps that ran and are not inside another one, in the
  order they started. A plain `test.step` is a step of the result only: nested helpers such as
  `Confirm the dialog` are how a test works, not what it specifies.
- Like every case field, they apply only when the report creates the case
  ([created cases only](https://docs.probara.net/en/developers/ci-integrations/playwright/metadata/#created-cases-only)).

## Attachments

Every file of an attempt goes to its result: what Playwright records on its own, and what a test
attaches.

### What Playwright records

Turn them on in `use`, as usual; the reporter uploads what they produce:

```ts
use: {
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
  trace: 'retain-on-failure',
},
```

| Playwright                                   | The file in Probara                                   |
| -------------------------------------------- | ----------------------------------------------------- |
| `screenshot`                                 | `test-failed-1.png` (and the others Playwright keeps) |
| `video`                                      | The `.webm` video of the attempt                      |
| `trace`                                      | `trace.zip`: open it with `npx playwright show-trace` |
| A failed attempt (Playwright 1.51 and later) | `error-context.md`: the page snapshot at the failure  |
| `toHaveScreenshot()` failures                | The expected, actual and diff images                  |

Each attempt has its own files, so a retried test uploads the files of every attempt, each to its
own result.

### `testInfo.attach()`

Playwright's own API works as it is, with a file or a body:

```ts
// tests/api.spec.ts
import { test } from '@playwright/test';

test('creates an order', async ({ page }, testInfo) => {
  await testInfo.attach('order', {
    body: JSON.stringify({ id: 1042, total: 25 }),
    contentType: 'application/json',
  });
  await testInfo.attach('checkout', { body: await page.screenshot(), contentType: 'image/png' });
});
```

The files are named after the attachment, with the extension of their content type:

```text
order.json application/json
checkout.png image/png
```

### `probara.attach()`

`probara.attach()` does the same from anywhere a test runs (a helper, a fixture, a page object),
without passing `testInfo` around. Give it a file on disk or a body, and await it:

```ts
// tests/receipt.spec.ts
import { writeFile } from 'node:fs/promises';
import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('prints the receipt', async () => {
  const path = test.info().outputPath('receipt.txt');
  await writeFile(path, 'Total: 25.00');
  await probara.attach({ name: 'receipt.txt', path });
  await probara.attach({ name: 'log', body: 'paid in 1.2 s', contentType: 'text/plain' });
});
```

```text
receipt.txt text/plain
log.txt text/plain
```

| Field         | With `path`                               | With `body`                            |
| ------------- | ----------------------------------------- | -------------------------------------- |
| `name`        | Required: the name in Probara             | Required: the name in Probara          |
| `path`        | The file to attach (Playwright copies it) | —                                      |
| `body`        | —                                         | A string, a `Buffer` or a `Uint8Array` |
| `contentType` | Optional: guessed from the file name      | Required, such as `text/plain`         |

A string body is sent as it is; pass a `Buffer` for binary content. A file that cannot be attached
(it does not exist, say) is a `[probara]` warning on the test's stderr, and the test goes on.

### Files of a step

A file attached while a `test.step` runs (`testInfo.attach()` or `probara.attach()`) goes to that
step, and Probara shows it under the step, where it happened:

```ts
// tests/upload.spec.ts
import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test('uploads an avatar', async ({ page }) => {
  await test.step('Choose the file', async () => {
    await probara.attach({
      name: 'avatar',
      body: await page.screenshot(),
      contentType: 'image/png',
    });
  });
});
```

A file attached inside a hook or a fixture that is not a `test.step` goes to the nearest
`test.step` around it, or to the result.

### File names

A file keeps its own name (`test-failed-1.png`, `trace.zip`). Playwright stores what
`testInfo.attach()` copies under a hashed name (`receipt-<sha1>.txt`), and a merged blob report
stores every file that way; such a file is named after its attachment instead, with the file's
extension when the name has none: `receipt.txt`, `avatar.png`.

### Test output

With `captureOutput: true` (`PROBARA_CAPTURE_OUTPUT=true`), each attempt's stdout and stderr are
attached as `stdout.log` and `stderr.log`, when there is any:

```ts
reporter: [['@probara/playwright-reporter', { captureOutput: true }]],
```

```ts
// tests/logs.spec.ts
import { test } from '@playwright/test';

test('prints what it does', async () => {
  console.log('Opening the cart');
  console.error('Slow response: 2.1 s');
});
```

```text
stdout.log text/plain
stderr.log text/plain
```

### Limits and what is skipped

| Limit                                      | What happens                                               |
| ------------------------------------------ | ---------------------------------------------------------- |
| 32 MiB per file                            | A larger file is skipped with a warning                    |
| 20 files per result                        | The files after the first 20 are skipped, with one warning |
| Images over 10 MiB or 8192 px wide or tall | Skipped with a warning (Probara converts images to WebP)   |
| Executables and scripts                    | Skipped: Probara refuses their content types               |
| A missing, unreadable or empty file        | Skipped with a warning                                     |
| A result that recorded nothing             | Its files are not uploaded                                 |

Warnings name the file, and the log ends with how many files were attached, skipped and failed.
`uploadAttachments: false` (`PROBARA_UPLOAD_ATTACHMENTS=false`) uploads none. Every result with
files costs two more requests against the organization's
[rate limit](https://docs.probara.net/en/developers/ci-integrations/playwright/configuration/#rate-limit).

Files upload after their report is recorded, while the run is still open, then the run is closed.
Keep Playwright's output folder (`test-results/`) until the reporter is done: it uploads from
there.

## Playwright versions

| Playwright     | What changes                                                                                                              |
| -------------- | ------------------------------------------------------------------------------------------------------------------------- |
| 1.51 and later | A failed attempt also uploads `error-context.md`                                                                          |
| 1.50 and later | Files go to the step they were attached in; `test.step.skip()` is a `skipped` step                                        |
| 1.42 to 1.49   | Playwright does not say which step a file belongs to: every file stays with the result; `test.step.skip()` does not exist |

Nothing fails on an older Playwright of the supported range: those details are just not there.

## Common problems

- **A step is missing.** Only `test.step` makes steps. A helper function that is not wrapped in
  `test.step` shows nothing; neither do `expect` calls.
- **Steps are flattened.** A step started without `await` ends before its children: await every
  `test.step`.
- **The case has no steps.** Only `probara.step()` titles declare case steps, and only for a case
  the report creates. An existing case keeps its own.
- **An attachment is missing.** See
  [troubleshooting](https://docs.probara.net/en/developers/ci-integrations/playwright/troubleshooting/#an-attachment-is-missing).

Next: [results and retries](https://docs.probara.net/en/developers/ci-integrations/playwright/results-and-retries/).
