Playwright reporter

@probara/playwright-reporter is a Playwright reporter that sends every result of a playwright test run to Probara: each attempt with its status, steps, errors and attachments, linked to its test case. Cases that do not exist yet are created, and the run is closed once everything is in. You register it in playwright.config next to the reporter you read in the terminal; there is no separate upload step.

• Package: @probara/playwright-reporter on npm (current version 0.1.0).
• Source and changelog: probara-sdk on GitHub (Apache-2.0).
• Probara side: the Playwright integration guide covers the Playwright card, its app tokens and what Probara shows.

Before you start

You need	Why
Node.js 22.12 or later	The package’s engines field. Playwright’s main process runs the reporter.
@playwright/test 1.42 or later	The peer range. Files of steps need 1.50, the error context of a failed attempt 1.51.
A paid plan (Pro or Enterprise, or any trial)	Reporting from CI is not part of the Free plan. On Free, Probara answers 403 forbidden.
Someone with the Manage integrations permission	Creating an app token is part of that permission, held by owners and admins.
The Project ID of the project to report into	A capital letter followed by capital letters or digits, such as SHOP. Any other value turns reporting off.

1. Create an app token

The reporter signs in with an app token: a token that belongs to your organization, not to a person. It uses no seat and keeps working when the person who created it leaves.

1. In Workspace › Integrations, select the Playwright card.
2. Open the Access tokens tab and select Create an app token.
3. Enter a Token name after what uses it (the repository or the pipeline) and select Create token.
4. Select Copy token in Copy your new token. It starts with probara_app_ and is shown once: store it right away as a CI secret named PROBARA_API_TOKEN.

The reporter reads the token from the environment. Never put it in playwright.config, which is committed.

2. Install the reporter

npm i -D @probara/playwright-reporter

Pin the version in package.json (or keep the lockfile) so a new release never changes a pipeline by surprise: the reporter is in 0.x, where a minor version may change options (upgrading).

3. Register it in playwright.config

Add it next to the reporter you read in the terminal, with your Project ID (or set PROBARA_PROJECT in the environment instead):

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['@probara/playwright-reporter', { projectId: 'SHOP' }]],
});

Everything else has a default: the run is named after the CI build, and its branch, commit and build URL come from the CI. The package exports the type of its options, ProbaraPlaywrightOptions, if you want the options checked:

playwright.config.ts

import { defineConfig } from '@playwright/test';
import type { ProbaraPlaywrightOptions } from '@probara/playwright-reporter';

export default defineConfig({
  reporter: [
    ['list'],
    ['@probara/playwright-reporter', { projectId: 'SHOP' } satisfies ProbaraPlaywrightOptions],
  ],
});

4. Run your tests with the token

Give the step that runs npx playwright test the secret. In GitHub Actions:

- name: Run Playwright tests
  run: npx playwright test
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}

The reporter logs on stderr, in [probara] lines, and ends with the link to the run:

$ npx playwright test
[probara] Sending 2 results of 2 tests (2 passed, 0 failed, 0 skipped, 0 blocked)
[probara] Recorded 2 results (1 new case, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1

Without a token and a project, the reporter stays off and quiet: local runs, and builds that get no secrets (such as pull requests from forks on most CIs), send nothing, with no extra setting. For a complete workflow on your CI, see CI providers.

What you see in Probara

• An automated test run, named after the CI build (Playwright #42 on GitHub Actions), with the branch, the commit and a link to the build. It is closed when the reporter is done.
• One result per attempt: a retried test shows every attempt in the run case’s history, and the run case keeps the last outcome.
• Steps from test.step, nested, with their status, duration and error, and the files attached in them.
• Attachments: screenshots, videos, traces, the error context and your own testInfo.attach() files.
• New test cases for tests that match none, in suites built from the file and the describe titles.
• Attribution to the token: runs and results show the app and the token name, such as Playwright · e2e-ci, not a person.

How Playwright maps to Probara

Playwright	In Probara
The playwright test command	One automated run, named after the CI build, closed at the end
A test (per Playwright project)	A test case, matched by its automation key or linked by a case id
Each attempt (retries included)	A result in the run, with its status, duration, start time and errors
test.step and the hooks that run steps	The steps of the result, nested, with their status, duration and error
Screenshots, videos, traces, attachments	Files of the result, or of the step they were attached in
probara.* calls in the test	The title, suite, tags, fields and steps of a new case; result parameters

The same test in two Playwright projects (chromium and firefox) is two cases: the project is part of the automation key (linking).

Reporting never breaks your test run

The reporter never throws into Playwright and never changes its exit code: a failed test fails the command, a reporting problem does not. A wrong setting, Probara being unreachable or a refused token is logged on stderr, never with the token, and the tests keep their outcome. There is no option to fail the command on a reporting error; to keep what could not be sent, set a results file.

Reference

Page	What it covers
Configuration	Every option, its variable, type and default; precedence; network, proxies and retries
Linking tests to cases	Annotations, case ids in titles, probara.id(), automation keys, missing cases
Metadata	Title, suite, comment, tags, fields, parameters, ignoring an attempt
Steps and attachments	test.step, hooks, probara.step(), screenshots, videos, traces, probara.attach()
Results and retries	Run options, statuses, status mapping, retries, the results file
Sharding and several projects	One run for every shard, merge-reports, results for several Probara projects
CI providers	GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket, Buildkite
Troubleshooting	Problems and their solutions, the log, PROBARA_DEBUG, checking what would be sent
Migrating and upgrading	From Qase, from probara import junit, versioning and the automation key contract

Next: configure the reporter, or go back to the CI integrations overview. Running Jest, Cypress or Vitest instead? See the Jest reporter, the Cypress reporter or the Vitest reporter.