Link Playwright tests to cases

Each result goes to a Probara test case. A test that names its case is linked to it; a test that names none is matched by its automation key, and its case is created the first time it is reported. Link tests by id when their cases already exist in Probara, written by hand or imported.

Three ways to name a case

Way	Example	When
A `probara_case` annotation	`test('pays', { annotation: { type: 'probara_case', description: 'SHOP-12' } }, ...)`	The usual way; works for skipped tests too
A case id in the title	`test('SHOP-12 pays with a card', ...)`	Quick, and visible in every report
`probara.id()` in the test	`probara.id('SHOP-14')`	Ids known only at run time

import { test } from '@playwright/test';
import { probara } from '@probara/playwright-reporter';

test.describe('checkout', () => {
  test(
    'pays with a card',
    { annotation: { type: 'probara_case', description: 'SHOP-12' } },
    async () => {
      // ...
    },
  );

  test('SHOP-13 pays with a voucher', async () => {
    // ...
  });

  test('pays with a gift card', async () => {
    probara.id('SHOP-14');
  });

  test('shows the total', async () => {
    // No case named: matched by its key, created the first time.
  });
});

Probara receives one result per test, each with its case, and the same key it would get without the ids:

[
  {
    "caseDisplayId": "SHOP-12",
    "automationKey": "checkout.spec.ts > checkout > pays with a card",
    "title": "pays with a card"
  },
  {
    "caseDisplayId": "SHOP-13",
    "automationKey": "checkout.spec.ts > checkout > pays with a voucher",
    "title": "pays with a voucher"
  },
  {
    "caseDisplayId": "SHOP-14",
    "automationKey": "checkout.spec.ts > checkout > pays with a gift card"
  },
  { "automationKey": "checkout.spec.ts > checkout > shows the total" }
]

Annotations

An annotation takes one id or a comma list ('SHOP-12, SHOP-13'), and several annotations add up. Since Playwright 1.42 they go in the test details; a test can also push one at run time:

test.info().annotations.push({ type: 'probara_case', description: 'SHOP-12' });

Case ids in titles

A title id is an id of the reported project (or of one of projects) in a describe or test title, written SHOP-12 or SHOP_12, anywhere in the title. It is removed from the title and from the key, so adding one never changes a key. An id of a project the reporter does not know stays in the title, as text.

`probara.id()`

probara.id() takes an id or a list, and adds a probara_case annotation to the running attempt. A test skipped before it runs (test.skip('title', ...), or a skipped describe) never runs its body: link it with an annotation or a title id instead.

What an id is

The ids are the case’s display id: the Project ID, a dash and the number shown in Probara (SHOP-12). An id Probara does not know is not recorded, and the log names it as unmatched (troubleshooting).

Several cases, and several sources

Every source counts: the annotations first (in order, probara.id() among them), then the title ids, each id once. A test linked to several cases is sent once per case, with the same key, and its attachments are uploaded to each result:

import { test } from '@playwright/test';

test(
  'SHOP-21 logs in',
  { annotation: { type: 'probara_case', description: 'SHOP-20, SHOP-21' } },
  async () => {
    // ...
  },
);

[
  { "caseDisplayId": "SHOP-20", "automationKey": "login.spec.ts > logs in" },
  { "caseDisplayId": "SHOP-21", "automationKey": "login.spec.ts > logs in" }
]

Automation keys

A test that names no case is matched by its automation key: the same key finds the same case on every run, and a new key creates a new case. The key is built from:

Part	From	Example
File	The test file, relative to `rootDir` (Playwright’s, by default)	`checkout.spec.ts`
Title path	The `describe` titles, then the test title, case ids removed	`checkout > pays`
Playwright project	Not part of the key. It is sent with each result as its `project` parameter	`project=chromium`

So checkout.spec.ts > checkout > pays is the key of that test in every project. The projects share one key, so they share one case, with one result per project run. The project travels with each result as its project parameter, and a case that declares a project parameter gets one run case per project. The project is context: it is shown on the result, and it never becomes a parameter of a case a report creates.

Earlier reporter releases put the project in the key. This behavior needs the reporter release that sends parameters apart from the key; a case reported with an earlier key is not matched by it.

The algorithm is automation key v1 of @probara/core, frozen and shared by every Probara tool.

Changes the key (creates a new case)	Does not change the key
Renaming a test, a `describe` or a file	Adding or removing a case id
Moving a file	`probara.title()` and `probara.suite()`
Renaming a Playwright project	`probara.parameters()`
Changing `rootDir`, or `testDir`, which Playwright’s `rootDir` follows	The Playwright version, the CI, or the directory you run from

To keep a case through a rename, link the test by id. To check keys before a refactor, see check what would be sent.

Missing cases

A test whose key matches no case creates one, named after the test, in a suite path built from the file and the describe titles, under suiteUlid (the project root by default). probara.title() and probara.suite() choose the name and the suite of a created case.

With createMissingCases: false (PROBARA_CREATE_MISSING_CASES=false), such a test records nothing and is logged as unmatched: only existing cases get results.

The same keys as `probara import junit`

The reporter builds the keys the Probara CLI gives Playwright’s JUnit reporter (--reporter=junit), and reads the same links: probara_case annotations (which the JUnit reporter writes as properties) and case ids in titles. A project that imported Playwright’s JUnit XML with probara import junit can switch to the reporter and keep every case; the steps are in migrating from probara import junit.

Next: name the cases a report creates, or report into several projects.