# CI integrations > Report test results to Probara from any CI, with the Probara CLI and JUnit XML or with the Playwright, Jest, Cypress, Vitest and Cucumber reporters. Probara records the tests your pipeline runs as an **automated test run**: one result per test, linked to its test case, with the files the test produced. You choose how the results get there. Each way below creates the run, links every test to its case (and creates the cases that do not exist yet), sends the results and closes the run, so the pipeline needs no extra step to finish it. ## Choose how to report | Way | Use it when | | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | [CLI and JUnit XML](https://docs.probara.net/en/developers/cli/) (`@probara/cli`) | Your framework writes JUnit XML: Jest, pytest, Playwright, Maven Surefire, Go with gotestsum, or anything else. | | [CLI and Cucumber JSON](https://docs.probara.net/en/developers/cli/#import-cucumber-results) (`@probara/cli`) | Your tests run with a tool that writes Cucumber JSON, such as Cucumber-JVM or playwright-bdd, or you import a report you already have. | | [Playwright reporter](https://docs.probara.net/en/developers/ci-integrations/playwright/) (`@probara/playwright-reporter`) | You run Playwright and want steps, attachments and every attempt of a retried test. | | [Jest reporter](https://docs.probara.net/en/developers/ci-integrations/jest/) (`@probara/jest-reporter`) | You run Jest and want steps, attachments, links, or to run only the tests of a Probara run. | | [Cypress reporter](https://docs.probara.net/en/developers/ci-integrations/cypress/) (`@probara/cypress-reporter`) | You run Cypress and want each attempt, the failure screenshots and to run only the tests of a Probara run. | | [Vitest reporter](https://docs.probara.net/en/developers/ci-integrations/vitest/) (`@probara/vitest-reporter`) | You run Vitest and want each attempt and repeat, steps, attachments, or to run only the tests of a Probara run. | | [Cucumber reporter](https://docs.probara.net/en/developers/ci-integrations/cucumber/) (`@probara/cucumber-reporter`) | You run cucumber-js and want every scenario and step, with its attachments, on the case its synced scenario names, with no JSON file. | All six are open source, in the [probara-sdk repository](https://github.com/cynch-me/probara-sdk), and published on npm. They need Node.js 22.12 or later on the machine that reports. ## What each one sends | What | CLI and JUnit XML | Playwright reporter | Jest reporter | Cypress reporter | Vitest reporter | Cucumber reporter | | -------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | | Framework | Any that writes JUnit XML | `@playwright/test` 1.42 or later | Jest 29.6 or later | Cypress 15.10.0 or later | Vitest 3.2 or later | `@cucumber/cucumber` 13.3 or later | | Install | `npx @probara/cli`, or `npm i -D @probara/cli` | `npm i -D @probara/playwright-reporter` | `npm install --save-dev @probara/jest-reporter` | `npm install --save-dev @probara/cypress-reporter` | `npm install --save-dev @probara/vitest-reporter` | `npm i -D @probara/cucumber-reporter` | | Results | One per test, with its final outcome | One per attempt | One per attempt | One per attempt | One per attempt | One per attempt, and one per row of a Scenario Outline | | Retries and attempts | Earlier attempts summarized in the notes, when the report keeps them | Every attempt is its own result | Every `jest.retryTimes` attempt is its own result | Every Cypress retry is its own result | Every `retry` attempt and every repeat is its own result | Every `--retry` attempt is its own result | | Steps | No: JUnit XML has none | `test.step` and hooks, nested | `probara.step()`, nested | `probara.step()`, nested | `probara.step()`, nested | Every step with its keyword, status and data; Background steps nested | | Attachments | Files the report points to, and test output with `--attach-output` | Screenshots, videos, traces and every attached file | `probara.attach()` files, and console output with `captureOutput` | Failure screenshots, the spec video, `probara.attach()` files, console output with `captureOutput` | `probara.attach()` and `annotate` files, failure screenshots in browser mode, console output with `captureOutput` | Files of `this.attach`, `this.log` and `this.link` | | Parameters | Only as part of the test name | The parameters of each attempt | `probara.parameters()` | `probara.parameters()`, and the browser each spec ran in | `probara.parameters()`, and the Vitest project each test ran in | The values of each Scenario Outline row | | Fields of new cases | No | Title, suite, tags, system and custom fields | Title, suite, tags, system and custom fields | Title, suite, tags, system and custom fields | Title, suite, tags, system and custom fields | Title, and the suite of `suiteUlid` | | Run selection | No | No | [Only the tests of a Probara run](https://docs.probara.net/en/developers/ci-integrations/jest/run-selection/) | [Only the tests of a Probara run](https://docs.probara.net/en/developers/ci-integrations/cypress/run-selection/) | [Only the tests of a Probara run](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/) | No | | Sharded CI | [One run for every shard](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/ci/sharding.md) | [Merged blob reports, or one shared run](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/) | [One shared run for every shard](https://docs.probara.net/en/developers/ci-integrations/jest/sharding/) | [One shared run for every machine](https://docs.probara.net/en/developers/ci-integrations/cypress/sharding/) | [Merged blob reports, or one shared run](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/) | One shared run: automatic on GitHub Actions, else `PROBARA_RUN_ULID` | A reporter never changes the exit code of your tests: a failed test fails the command, a reporting problem does not. The CLI runs as its own step, so only a reporting problem fails it. ## What they share ### An app token Every way signs in with an [app token](https://docs.probara.net/en/developers/authentication/#app-tokens): a token that belongs to your organization, not to a person. It uses no seat and keeps working when the person who created it leaves. An owner or admin creates it in **Workspace › Integrations**, from the card of the tool you use (**JUnit XML**, **Playwright**, **Jest**, **Cypress**, **Vitest** or **Cucumber**); it starts with `probara_app_` and Probara shows it once. Store it as a CI secret named `PROBARA_API_TOKEN`. The CLI and the reporters read the token only from that environment variable, so it never goes in a configuration file or on a command line. The token can only do what the reporters and the CLI need: create automated runs, send reports, add attachments to their results, close runs, read the case keys of a run, and sync feature files. Every other call answers `403 forbidden`. ### A paid plan Reporting from CI is part of the **Pro** and **Enterprise** plans, and of a Pro trial. On the Free plan, Probara keeps your app tokens but refuses them with `403 forbidden`; they work again once the organization is back on a paid plan. ### How a test finds its test case Each test becomes a result on one Probara test case: - **A case id** such as `SHOP-12` links the test to that case: in the test title (the CLI and the Playwright, Jest, Cypress and Vitest reporters), in a `probara_case` property of the JUnit report or a `probara_case` annotation in Playwright, or with `probara.id()` in any reporter. A Cucumber scenario links with a `@probara:SHOP-12` tag instead. - **Otherwise, the automation key** does: a key built from the test's file and title, which each reporter's documentation shows in full. By default the first report creates the case, and every later report finds it again by the same key. The Cucumber reporter builds the keys of `probara sync gherkin`, so a result lands on the case its scenario synced to. The Jest, Cypress and Vitest reporters build the same keys as a JUnit import of `jest-junit`, `cypress-junit` and Vitest's `junit` reporter output, so moving from the CLI to a reporter keeps every case and its history. ### An automated run Each command creates one automated run, named after the CI build, and closes it when every result is in. To report several jobs into one run, the way depends on the CI provider. In GitHub Actions, the jobs of one workflow run share a run automatically: set `PROBARA_CLOSE_RUN: false` on the parallel jobs, and close the run in a final job with `probara run close`, which needs no ULID there. On other CI providers, create the run first with `probara run create`, pass its ULID to every job in `PROBARA_RUN_ULID`, and close it at the end with `probara run close`. Runs, results and attachments show the app's name and the token name, not a person. With the [GitHub integration](https://docs.probara.net/en/guides/integrations/github/), Probara can also create the run and start your GitHub Actions workflow with its ULID: see [Running tests with GitHub Actions](https://docs.probara.net/en/guides/integrations/github-actions/). ## BDD with Gherkin Keep your `.feature` files in Git, and let the pipeline sync them and report their results: - **Sync** the scenarios on each push to the default branch, so Git stays the source of truth for their cases. See [Syncing feature files from Git](https://docs.probara.net/en/guides/syncing-feature-files-from-git/). - **Report** the scenarios with the [Cucumber reporter](https://docs.probara.net/en/developers/ci-integrations/cucumber/), which sends each scenario's result to its case at the end of the run. A cucumber-js run needs no JSON file. ```yaml name: BDD on: push: branches: [main] jobs: bdd: runs-on: ubuntu-latest env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: steps: - uses: actions/checkout@v5 - run: npm ci - name: Sync the feature files run: npx @probara/cli sync gherkin "features/**/*.feature" - name: Run the scenarios run: npx cucumber-js ``` The sync runs first, so the cases exist before the results arrive. The reporter is set in `cucumber.js` (see the [quick start](https://docs.probara.net/en/developers/ci-integrations/cucumber/)), so the failing scenarios are reported too. Other tools that write Cucumber JSON, such as Cucumber-JVM, send their report with [`probara import cucumber`](https://docs.probara.net/en/developers/cli/#import-cucumber-results) after the run instead. The [Cucumber CI providers](https://docs.probara.net/en/developers/ci-integrations/cucumber/ci-providers/) page has the same pipeline for GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines and Buildkite. ## Quick start - **Any framework with JUnit XML:** add `npx @probara/cli import junit ` after your tests, with `PROBARA_API_TOKEN` and `PROBARA_PROJECT` in the environment. The [CLI page](https://docs.probara.net/en/developers/cli/) has the step. - **Playwright:** install the reporter and add `['@probara/playwright-reporter', { projectId: 'SHOP' }]` to the reporters of `playwright.config`. Start with [Get started](https://docs.probara.net/en/developers/ci-integrations/playwright/). - **Jest:** install the reporter and add `['@probara/jest-reporter', { projectId: 'SHOP' }]` to the reporters of `jest.config.js`, after `default`. Start with [Get started](https://docs.probara.net/en/developers/ci-integrations/jest/). - **Cypress:** install the reporter and set it as the `e2e` reporter of `cypress.config.js`, with its `setupNodeEvents` helper and its support file. Start with [Get started](https://docs.probara.net/en/developers/ci-integrations/cypress/). - **Vitest:** install the reporter and add `['@probara/vitest-reporter', { projectId: 'SHOP' }]` to the reporters of `vitest.config.ts`, after `default`. Start with [Get started](https://docs.probara.net/en/developers/ci-integrations/vitest/). - **Cucumber:** install `@cucumber/cucumber` and `@probara/cucumber-reporter`, and add `@probara/cucumber-reporter` to the `format` of `cucumber.js`, with your Project ID under `formatOptions.probara`. Start with [Get started](https://docs.probara.net/en/developers/ci-integrations/cucumber/). Without a token and a project, the reporters send nothing, so local runs stay local. ## Setup guides The guides cover the Probara side: the card, its app tokens, and what Probara shows. - [Reporting from CI with JUnit XML](https://docs.probara.net/en/guides/integrations/junit-xml/) - [Reporting from CI with Playwright](https://docs.probara.net/en/guides/integrations/playwright/) - [Reporting from CI with Jest](https://docs.probara.net/en/guides/integrations/jest/) - [Reporting from CI with Cypress](https://docs.probara.net/en/guides/integrations/cypress/) - [Reporting from CI with Vitest](https://docs.probara.net/en/guides/integrations/vitest/) - [Reporting from CI with Cucumber](https://docs.probara.net/en/guides/integrations/cucumber/) - [Syncing feature files from Git](https://docs.probara.net/en/guides/syncing-feature-files-from-git/)