Migrating to the Cypress reporter

Whatever you come from, the reporter is three registrations: the reporter of the Cypress config, its plugin in setupNodeEvents, and its support file.

const { defineConfig } = require('cypress');
const { probaraNodeEvents } = require('@probara/cypress-reporter/setup');

module.exports = defineConfig({
  e2e: {
    reporter: '@probara/cypress-reporter',
    reporterOptions: { projectId: 'SHOP' },
    setupNodeEvents(on, config) {
      return probaraNodeEvents(on, config);
    },
  },
});

import '@probara/cypress-reporter/support';

If setupNodeEvents already registers other plugins (such as the one of the tool you leave), see with other plugins.

You come from	Jump to
Cypress’s `junit` reporter or cypress-junit, `probara import junit`	The JUnit import
The Qase Cypress reporter	Qase
`allure-cypress`	Allure Cypress
JUnit XML and `trcli parse_junit`	TestRail

From the JUnit import

If your pipeline writes JUnit XML with Cypress’s built-in junit reporter or with cypress-junit, and sends it with probara import junit, the reporter can replace both steps without unlinking a single case: it gives each test the automation key the import gives it.

// cypress.config.js, before
module.exports = defineConfig({
  e2e: {
    reporter: 'junit', // or 'cypress-junit'
    reporterOptions: { mochaFile: 'reports/cypress-[hash].xml', includePending: true },
  },
});

Replace the JUnit reporter with this one, and register its plugin and support file (above). Cypress takes one reporter: to keep the XML for a while, list both in cypress-multi-reporters.
Check the keys of both on the same specs: they must be equal.
In CI, drop the import step, and give the test step PROBARA_API_TOKEN.

The JUnit report	Reporter setting
Written with the default options, the spec in the `file` of its root suite	Nothing: `keyIncludesFile: true` is the default
Written without that `file` attribute	`keyIncludesFile: false` (`PROBARA_KEY_INCLUDES_FILE`)

Both paths read the same two parts of a test: the spec path relative to the project root, and one title segment, its describe titles and its it title joined by spaces, without the case ids. Run the import from the project root (or pass --root-dir): the import makes the spec path relative to the directory it runs in, and the reporter makes it relative to Cypress’s projectRoot.

Check the keys before you switch

Run the suite once with the JUnit reporter and once with the reporter’s results file, reporting off, and compare the dry runs of both: each prints one line per result, with its status, its case and its key.

npx cypress run   # with the JUnit reporter
npx --yes @probara/cli@0.3 import junit 'reports/cypress-*.xml' --dry-run > junit.txt
PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=probara-results.json npx cypress run   # with this reporter
npx --yes @probara/cli@0.3 import results 'probara-results*.json' --dry-run > reporter.txt

$ npx --yes @probara/cli@0.3 import results 'probara-results*.json' --dry-run
passed  SHOP-12  cypress/e2e/cart.cy.js > Cart adds an item

A line whose key changed is a test that would get a new case; retried tests and the tests a failing hook kept from running have more lines on the reporter’s side (what changes). Delete the XML and the results file afterwards.

The CLI reads Cypress’s JUnit XML as its cypress-junit dialect from version 0.3 on. It recognizes a report by its marks: a <testsuites> root named Mocha Tests, exactly one testsuite with a file that is a Cypress spec (cypress/ in its path, or a .cy.* name), and testcases whose classname ends their name. A report that renamed the root (testsuitesTitle), or specs outside cypress/ without a .cy.* name, is read as generic instead: pass --dialect cypress-junit on the import.

In CI

Before, two steps: Cypress writes the XML, and the CLI sends it.

- name: Run Cypress tests
  run: npx cypress run
- name: Send the results
  if: ${{ !cancelled() }}
  run: npx --yes @probara/cli@0.3 import junit 'reports/cypress-*.xml'
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: SHOP

After, one step: the reporter, registered in the Cypress config, sends every result when the run ends.

- name: Run Cypress tests
  run: npx cypress run
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: SHOP

What changes from the JUnit import

Cypress	`probara import junit`	The reporter
A test that fails, then passes on a retry	One result: the pass	Two results of one case: the failed attempt, then the pass
A test that fails on every attempt	One result: the last failure	One result per attempt
A hook that fails on every attempt	The hook failure only	The retried attempt of the test, and the hook failure Cypress names
The tests that hook kept from running	Nothing: they left no testcase	One skipped result each
Suite of a created case	The spec alone	The spec and the `describe` blocks of the test

The status, the error in the notes, the duration and the start time are the same. The reporter adds two parameters the JUnit has nowhere to put: browser and, from the second attempt on, attempt. The suite only changes where a case the report creates goes: an existing case never moves.

What you gain: every attempt of a retried test; the screenshot of each failed attempt, and the video with attachVideos; nested steps; files and console output; links and issues; the fields of new cases; run selection; and failed results assigned to the people you name.

From Qase

Moving from the Qase Cypress reporter is mostly renaming: both register as a reporter, a setupNodeEvents hook and a support file, and their helpers map one to one. The one change that touches your specs is the case id syntax.

Linking by case id

// with Qase
it('(Qase ID: 12) adds an item', () => {
  cy.get('.add').click();
});

// with Probara
it('SHOP-12 adds an item', () => {
  cy.get('.add').click();
});

(Qase ID: 12) is not a Probara id and links nothing: it stays in the title and in the key, and the test gets a new case. There is no option that reads Qase’s syntax. Rename the ids first, or call probara.id('SHOP-12') without touching the title (linking).

Names, side by side

In a Qase suite	In a Probara suite	What changes
`(Qase ID: 12)` in an `it` or a `describe` title	`SHOP-12` in the title, or `probara.id('SHOP-12')`	The id syntax
`qase.title()` / `qase.suite()`	`probara.title()` / `probara.suite()`	The same, for the case the report creates
`qase.fields`	`probara.fields({ ... })`	Takes effect only when the report creates a case
`qase.comment`	`probara.comment(text)`	Written first in the notes of the result, before the error
`qase.ignore`	`probara.ignore()`	The same
`qase.parameters` / `groupParameters`	`probara.parameters({ ... })`	One call for both; the browser is added on its own
`qase.step(title, params)`	`probara.step(title, body?, { expected, data })`	A body, or none; nesting by call order
`qase.attach({ paths \| content })`	`probara.attach({ name, path \| body })`	A name is required; a missing file is left out with a warning
A project per case id prefix	`projects` and the case-id prefix	Several projects
`QASE_REPORT`	`PROBARA_API_TOKEN` and `PROBARA_PROJECT`	Reporting is on when both are set; off with `enabled: false`
`QASE_RUN_ID`	`PROBARA_RUN_ULID` (`run.ulid`)	Report into a run that already exists
Report mode, its results dir	`resultsFile` and `probara import results`	A JSON file of what could not be sent, sent later
Screenshots searched in `screenshotsFolder`	Nothing to configure	Attached to the exact attempt (`attachScreenshots`, on by default)
The video searched in `videosFolder`	`video: true` and `attachVideos`	Off by default: one upload per failed result

Both reporters send one result per attempt, report it.skip as skipped, and report the tests a failing beforeEach kept from running as skipped. Probara adds the attempt parameter from the second attempt on.

Not ported from Qase

Qase feature	Why not
The network profiler (the `cy.intercept` recorder)	A task per request: more noise and cost than value
`qase.cucumberSteps`	Cucumber steps belong to a BDD preprocessor
A retry profile (which attempts to keep)	Every attempt is a result; `statusMapping` and `statusFilter` are the only rules that change or drop one
Custom run hooks (`beforeRunHook`, `afterRunHook`)	The plugin creates the run, sends it and closes it when the run ends; `closeRun` decides whether it is closed

From Allure Cypress

Allure’s plugin registers in setupNodeEvents and publishes its API in the support file, like this reporter: replace both registrations, and rename the calls.

Allure Cypress	Probara	What changes
`allure.step(name, body)`	`probara.step(title, body?, { expected, data })`	`expected` and `data` are the case step’s expected result and data
`stepsFromCommands` (every command a step)	Nothing: commands are not steps	Name the steps with `probara.step()`; Probara keeps 200 steps per result
`allure.attachment(name, content, type)`	`probara.attach({ name, body, contentType })`	A `path` is read by the plugin from the project root
`allure.link(url, name)` / `allure.issue(id)`	`probara.link(url, name?)` / `probara.issue(id)`	An issue becomes a link through `issueUrlTemplate`
`allure.tms`	`probara.link(url, name?)`	A plain link
`displayName`, `description`, `severity`	`probara.title()`, `probara.fields({ ... })`	Fields of the case the report creates
Labels and tags	`probara.tags('smoke', 'cart')`	Tags of the created case
`allure.parameter(name, value)`	`probara.parameters({ name: value })`	The browser is added on its own
`historyId`	Nothing: the key is the spec path and the full title	A key of your own would rename every case the import knows
The video of a spec	`video: true` and `attachVideos`	Off by default: one upload per failed result

Not ported: a case owner (Probara has no owner field), masked parameters (every parameter is sent), and Allure’s categories, known issues and defect links.

From TestRail

trcli is a command, not a reporter: it reads the JUnit XML of the run and uploads it after the tests. The reporter replaces both the XML and the upload.

trcli	Probara
`--user` / `--password` / `--project`	`PROBARA_API_TOKEN` (the Cypress card) and `projectId`
`--add-run`, `--close-run`	The run is created and closed for you; `run.ulid` reuses one
`--milestone`, `--plan`, `--configuration`	`run.milestone`, `run.plan`, `run.configurations`
`--refs`	Nothing: a Probara run has no custom reference fields; `run.tags` is what it takes
`parse_junit`	Nothing: the results are sent when the run ends
`-a`, `--assign`	`assignFailedTo`
`--case-fields`	`probara.fields({ ... })` for the case the report creates
`--case-matcher`	Nothing: the automation key is the spec path and the full title
`automation_id`, a `C123` in the title	A case id in the title (`SHOP-1`), or `probara.id('SHOP-1')`
`--run-id`	`PROBARA_RUN_ULID`
`--result-fields`	Not ported: Probara has no result fields
`--update-existing-cases`, `--update-strategy`	Not ported: a report never changes a case; change it in Probara, where it is reviewable

Upgrading the reporter

@probara/cypress-reporter follows semantic versioning. Read the changelog before you upgrade: every change that needs an action from you is listed there first.

0.x (now): the reporter is young. A minor version (0.2.0) may change options, log lines or defaults; a patch version (0.1.1) only fixes bugs. Pin the version in package.json (or keep the lockfile) and upgrade on purpose.
1.0 and later: breaking changes only in a major version.
@probara/cypress-reporter depends on @probara/core of the same release line; install the reporter alone and let it bring its core.

What is a breaking change

Any of these needs a major version from 1.0 on, and a minor one before:

An option or a PROBARA_* variable removed or renamed, or its meaning changed.
A default changed (closing runs, creating missing cases, keyIncludesFile, what gets attached).
A probara.* helper, the plugin (@probara/cypress-reporter/setup) or the support file (@probara/cypress-reporter/support) removed, or its behavior changed.
A status an attempt is sent with changed for the same attempt.
Any change to the automation key a test gets, with keyIncludesFile on or off. A different key unlinks the test from its case: the next run creates a new one.
Dropping a Cypress or Node.js version that is still supported upstream.

Adding an option, a helper, a log line or a file uploaded is not breaking. [probara] lines are for people: do not parse them.

The automation key contract

The key algorithm is automation key v1, frozen in @probara/core and pinned by golden vectors: no version of the reporter changes it. What the reporter adds is how a Cypress test becomes the key’s parts, the way probara import junit reads the report of Cypress’s junit reporter or of cypress-junit:

the spec file relative to Cypress’s projectRoot, when keyIncludesFile is on, as the file attribute of the report’s root suite;
the full title, the describe titles and the it title joined by single spaces, as one segment, the case ids removed.

That mapping is part of the contract too: a version that changes the key of a test is a breaking change, named as such in the changelog. To check an upgrade, compare what both versions would send, each run with its own results file (check what would be sent). Any changed key is a test that would get a new case.

Cypress and Node.js versions

Needs	Supported
Cypress	15.10.0 and later (the peer range); tested on 16.1.1
Node.js	22.12 and later (the `engines` field), for the Node.js that starts `cypress`

Cypress 15.10.0 is the first version with Cypress.expose(), which the plugin uses to hand the browser its settings: on an older Cypress the probara.* helpers would have nothing to read. The plugin runs on the Node.js that starts cypress; the reporter runs on the Node.js Cypress bundles. Cypress 15 still runs on Node.js 20, and this reporter does not: a project on Cypress 15 and Node.js 20 has to upgrade Node.js first. 0.1.0 is the first version: there is nothing to migrate from yet.