# Reporting from CI with Playwright

> Create the app token the Probara Playwright reporter signs in with, add the reporter to your Playwright configuration, and see what it sends.

The **Playwright** app sends your Playwright test results to Probara from any CI pipeline, with the
Probara reporter for Playwright. Each Playwright run becomes an automated test run with its results,
the test cases it names and their attachments.

The reporter signs in with an **app token**: a token that belongs to the organization, not to a
person. It uses no seat, and it keeps working when the person who created it leaves. This guide
covers the card in Probara and the setup; the
[reporter's documentation](https://docs.probara.net/en/developers/ci-integrations/playwright/) covers every
[reporter option](https://docs.probara.net/en/developers/ci-integrations/playwright/configuration/), test helper and
[sharded runs](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/).

## Before you start

| You need                                              | Why                                                                                                                         |
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| A paid plan (**Pro** or **Enterprise**, or any trial) | Reporting from CI is not part of the Free plan. On Free, its dialog shows **See plans** instead of **Create an app token**. |
| A Probara **owner** or **admin**                      | Only owners and admins can create and manage app tokens. Other members see the card without its tokens.                     |
| A Playwright project that runs in CI                  | The reporter runs with your tests, in the same `npx playwright test` command.                                               |

## Set it up

1. In **Workspace › Integrations**, select the **Playwright** card. The dialog lists what the app
   can do and links to this guide.
2. Open the **Access tokens** tab and select **Create an app token**.
3. Enter a **Token name** and select **Create token**. Name it after what uses it — the repository
   or the pipeline — so you know which one to revoke later.
4. Select **Copy token** in **Copy your new token**. This is the only time Probara shows it: store it
   in your CI as a secret named `PROBARA_API_TOKEN` before you select **Done**.
5. Install the reporter in your Playwright project:

   ```bash
   npm install -D @probara/playwright-reporter
   ```

6. Add it to the reporters of `playwright.config`, next to the one you read in the terminal, and
   replace `<PROJECT_CODE>` with your project's Project ID, such as `SHOP`:

   ```ts
   // playwright.config.ts
   reporter: [
     ['list'],
     ['@probara/playwright-reporter', { projectId: '<PROJECT_CODE>' }],
   ],
   ```

7. Pass the secret to your tests as the `PROBARA_API_TOKEN` environment variable and run them. For
   example, in GitHub Actions:

   ```yaml
   - name: Run Playwright tests
     run: npx playwright test
     env:
       PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
   ```

The token never goes in `playwright.config`: the reporter reads it from the environment. The card on
the Integrations page shows how many active tokens the app has.

## What the reporter sends

| What                 | How Probara records it                                                                                                                                                                                              |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Results              | One result per attempt: a retried test sends each attempt, in order, and the run case keeps the last outcome. Every attempt stays in the case's history.                                                            |
| Steps                | The steps each attempt ran (`test.step`), as a tree with their status, duration and error, shown under the result.                                                                                                  |
| Attachments          | Screenshots, videos, traces and the other files of an attempt, on its result — or on the step they were attached in. A trace opens in the trace viewer straight from the result.                                    |
| Parameters           | The parameters an attempt ran with. A name the case declares selects its run case; any other name, such as the Playwright `project`, is shown with the result only, as context.                                     |
| New test cases       | A test with no case yet creates one, under a suite path built from its file and `describe` blocks.                                                                                                                  |
| Existing test cases  | An archived case the test reports to is restored. A case found by its automation key moves to the suite the test is in now; a case linked by its ID keeps its suite. Its title, steps and fields are never changed. |
| Fields of a new case | Its description, tags, steps and fields — system fields such as priority or severity, and custom fields by name. Only when the report creates the case: an existing case is never changed.                          |

A field or option Probara does not know, or one deleted while the report runs, is skipped and
answered as a warning, never a failure: the report still records every result. A field with an
empty value is skipped without a warning. The
[reporter's documentation](https://docs.probara.net/en/developers/ci-integrations/playwright/metadata/) shows how a test sets its
parameters, tags, fields and steps.

## Manage tokens

Under **App tokens**, the **Access tokens** tab lists each active token with who created it (or
**Created by a removed user**), when, and when it was last used (**Never** until its first report).

**Revoke** a token, and confirm, when it may have leaked or when someone who could read it leaves
your team: pipelines that use it stop reporting at once, and it cannot be undone. Create a new token
for them first. Because an app token outlives the person who created it, rotating it is your call:
Probara never revokes it on its own.

## What an app token can do

Only what the reporter does: create automated runs, send reports (which can create the test cases
and suites they name), add attachments to their results, and close runs. It can also read the case
keys of a run, which the Jest reporter uses to run only a run's tests. It cannot read anything else, cannot remove attachments, and cannot be used with [Probara MCP](https://docs.probara.net/en/developers/mcp/) or for any
other API call. For scripts and other API clients, use a personal
[API token](https://docs.probara.net/en/developers/authentication/).

Runs, activity entries and attachments that an app token created show the app's logo and name beside
the token name — for example **Playwright · e2e-ci** — in the runs list, the run and milestone
summaries, [Workspace activity](https://docs.probara.net/en/guides/workspace/activity/) and the attachments list.

## When the plan changes

If your organization moves to the Free plan, its app tokens are kept but Probara refuses them, so
pipelines stop reporting; its dialog says so and offers **See plans**. You can still see and revoke the
tokens. They work again as soon as the organization is back on a paid plan, with nothing to set up
again. Manual test runs are not affected.

## Next step

- [Reporter documentation](https://docs.probara.net/en/developers/ci-integrations/playwright/) —
  [options](https://docs.probara.net/en/developers/ci-integrations/playwright/configuration/),
  [test helpers](https://docs.probara.net/en/developers/ci-integrations/playwright/metadata/),
  [sharded runs and blob reports](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/).
- [Reports API](https://docs.probara.net/en/developers/api-reference/#reports) — the endpoint the reporter calls, for
  your own tools.