# Vitest reporter

> Send every Vitest test result to Probara from any CI with @probara/vitest-reporter — install, configure and see your runs.

`@probara/vitest-reporter` is a [Vitest](https://vitest.dev) reporter that sends every test result
of a run to Probara: each attempt with its status, duration and errors, linked to its test case,
with the steps, attachments and links your tests add. Cases that are missing are created, and the
run is closed once everything is in.

It is open source, published on [npm](https://www.npmjs.com/package/@probara/vitest-reporter) at
`0.1.0`, and lives in the
[probara-sdk repository](https://github.com/cynch-me/probara-sdk/tree/main/packages/vitest-reporter).
Before 1.0, a minor version may bring breaking changes: pin the version
([upgrading](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/#upgrading-the-reporter)).

## What you get

- **One result per attempt**, `retry` and `repeats` included, with Vitest's own verdict:
  `test.fails` is honored, `skip` and `todo` are sent as skipped (a todo with the note `Todo`), and
  `context.skip(note)` is skipped with its note.
- **The same automation keys as `probara import junit`** on Vitest's JUnit output, so switching
  from the JUnit import keeps every case and its history.
- **Linking** by a case id in a test or `describe` title (`SHOP-12 adds an item`) or with
  `probara.id()`.
- **Test helpers** with nothing to set up, in Node (`test.concurrent` included) and in browser
  mode: `probara.step()` (nested, with expected results and data), `probara.attach()`,
  `probara.link()`, `probara.issue()`, `probara.parameters()`, and the title, suite, tags and fields
  of a new case.
- **Vitest's own files**: the attachments of `context.annotate()`, the failure screenshots of
  browser mode, and each attempt's console output with `captureOutput`.
- **Runs** with a name, environment, milestone, test plan, configurations and tags; one run for a
  sharded CI job, merged from the blobs; one run per `--watch` session; and **run selection**: only
  the tests of a run created in Probara.
- **It never breaks your test run.** A reporting problem is logged, never thrown, and Vitest's exit
  code stays the tests' own. What could not be sent can be kept in a file and sent later.

## Before you start

| You need                                              | Why                                                                                                                                                          |
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Node.js 22.12 or later                                | The package's `engines` range.                                                                                                                               |
| Vitest 3.2 or a later 3.x, 4.x or 5.x                 | The peer range `^3.2.0 \|\| ^4.0.0 \|\| ^5.0.0`: 3.2 is the first Vitest with test annotations, which the reporter reads. Tested on 3.2.7, 4.1.11 and 5.0.2. |
| For browser mode, the provider of your Vitest version | At the same version as `vitest` ([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)).                                                       |
| A paid plan (**Pro** or **Enterprise**, or any trial) | Reporting from CI is not part of the Free plan. On Free, Probara answers `403 forbidden`.                                                                    |
| A Probara **owner** or **admin**                      | Only owners and admins create and revoke app tokens.                                                                                                         |
| The **Project ID** to report into                     | Such as `SHOP`: a capital letter, then capital letters or digits.                                                                                            |

## 1. Create an app token

The reporter signs in with an [app token](https://docs.probara.net/en/developers/authentication/#app-tokens): a token that
belongs to your organization, not to a person. It uses no seat and keeps working when the person
who created it leaves.

1. In Probara, open **Workspace › Integrations** and select the **Vitest** card.
2. Open the **Access tokens** tab and select **Create an app token**.
3. Enter a **Token name** after what uses it (the repository or the pipeline) and select
   **Create token**.
4. Select **Copy token** in **Copy your new token**. It is shown once: store it right away as a CI
   secret named `PROBARA_API_TOKEN`.

The [Vitest integration guide](https://docs.probara.net/en/guides/integrations/vitest/) covers the card, managing and
revoking tokens, and what Probara shows for runs an app token creates.

## 2. Install the reporter

```bash
npm install --save-dev @probara/vitest-reporter
```

## 3. Add it to the Vitest config

Add the reporter to the `reporters` of the `test` section, after `default` (the one you read in the
terminal), with your Project ID:

```ts
// vitest.config.mts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: ['default', ['@probara/vitest-reporter', { projectId: 'SHOP' }]],
  },
});
```

Keep `'default'` in the list: a `reporters` list replaces Vitest's own. The reporter entry is the
same in any file Vitest reads its config from (`vitest.config.ts`, the `test` section of
`vite.config.ts`...).

You can leave `projectId` out and set `PROBARA_PROJECT` in the environment instead. The token never
goes in the config: the config is committed, and the reporter reads the token from
`PROBARA_API_TOKEN`.

### Nothing else to set up

The reporter adds its own setup file and runner to every Vitest project of the config, so there is
nothing to add to `setupFiles` or `runner`. The `probara.*` helpers,
[`captureOutput`](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#console-output) and
[run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/) work without any other entry,
and a browser project gets the browser setup file of your Vitest version
([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)).

## 4. Run your tests with the token

Give the step that runs `vitest` the secret. In GitHub Actions:

```yaml
- name: Run Vitest tests
  run: npx vitest run
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
```

There is no separate upload step: the reporter sends the results at the end of the same command.
[CI providers](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/) has complete pipelines for
GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines and Buildkite.

## What you see

Without a token, nothing is sent and the tests run as usual: with no project either, the reporter
stays quiet, so local runs send nothing; with a project, it says in one line that the token is not
set. In CI it logs on stderr, in `[probara]` lines, and ends with the link to the run:

```text
$ npx vitest run
[probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked)
[probara] Recorded 3 results (2 new cases, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1
```

In Probara, the run appears in the project's run list as an automated run, named after the CI build
(such as `Vitest #42`), with its branch, commit and build URL. It shows the **Vitest** app and the
token's name (such as **Vitest · unit-ci**) instead of a person.

A sharded CI job writes one blob per shard, with the reporter beside `blob`, and one job merges
them into one run ([sharding](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/)):

```bash
npx vitest run --shard=1/2 --reporter=blob --reporter=@probara/vitest-reporter
npx vitest run --shard=2/2 --reporter=blob --reporter=@probara/vitest-reporter
npx vitest --merge-reports
```

A `vitest --watch` session reports every re-run into one run, which stays open
([watch mode](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode)).

## How Vitest maps to Probara

| Vitest                                                                               | In Probara                                                                |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| A test, or a row of `test.each` and `test.for`                                       | A test case, matched by its automation key or linked by a case id         |
| Each attempt (`retry`) and each execution (`repeats`)                                | A result in the run, with its status, duration, start time and errors     |
| `skip`, `skipIf`, `runIf`, a test `.only` or `-t` left out, `todo`, `context.skip()` | A skipped result (`Todo`, or `Skipped: <note>`, in the notes)             |
| `probara.step()` calls                                                               | The steps of the result, nested, with their status, duration and error    |
| `probara.attach()`, `context.annotate()` files, failure screenshots, `captureOutput` | Files of the result, or of the step they were attached in                 |
| `probara.link()`, `probara.issue()`                                                  | Links of the result                                                       |
| Other `probara.*` calls, Vitest tags                                                 | The title, suite, tags, fields and steps of a new case; result parameters |
| A `beforeAll` or `afterAll` hook that throws, a test file that fails to load         | One failed result, as Vitest's JUnit output writes it                     |
| The `vitest run` command                                                             | One automated run, named after the CI build, closed at the end            |

An unhandled error (a rejection no test awaited) is no result: one warning counts them
([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)).

## Same cases as the JUnit import

The reporter gives each test the automation key `probara import junit` gives it on Vitest's own
JUnit output (`vitest run --reporter=junit`): the test file, then the `describe` blocks and the
title, each after `>` between spaces, without the case ids. For a `tests/cart.test.ts` whose
`describe('cart')` holds `it('SHOP-12 adds an item')` and `it('removes an item')`, and a
`tests/login.test.ts` whose `describe('login')` holds `it('logs in with a valid password')`, the
reporter sends:

```json
[
  {
    "automationKey": "tests/cart.test.ts > cart > adds an item",
    "status": "passed",
    "caseDisplayId": "SHOP-12"
  },
  {
    "automationKey": "tests/cart.test.ts > cart > removes an item",
    "status": "passed"
  },
  {
    "automationKey": "tests/login.test.ts > login > logs in with a valid password",
    "status": "passed"
  }
]
```

Vitest's JUnit output always names the file, so the default, `keyIncludesFile: true`, gives the
keys of the import; `keyIncludesFile: false` leaves the file out, for cases whose keys were made
without it ([keys without the file](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#without-the-file)).
A test keeps its case and its history when you switch from the JUnit import, on Vitest 3, 4 and 5
([migrating from the JUnit import](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/#from-the-junit-import));
an import by CLI 0.3.0 or earlier keyed a file that failed to load differently
([cases made by an earlier import](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/#cases-made-by-an-earlier-import)).

## Reporting never breaks your test run

The reporter never throws into Vitest and never changes its exit code: a failed test fails the
command, a reporting problem does not. Every problem (a wrong setting, Probara unreachable, a
refused token) is logged on stderr, never with the token, and the tests keep their outcome.
`probara.*` calls never throw into a test either: a call Probara cannot use is left out with a
warning. There is no option to fail the command on a reporting error; to keep what could not be
sent, set a [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file).

## Reference

| Page                                                                                  | What it covers                                                                                    |
| ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| [Configuration](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/)                 | Every option, its variable, type and default; precedence; network, proxies and watch mode         |
| [Linking tests to cases](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)              | Case ids in titles and describes, `probara.id()`, automation keys, `test.each` and `test.for`     |
| [Metadata and links](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)                 | Title, suite, comment, tags, fields, parameters, ignoring an attempt; links; the Jest differences |
| [Steps and attachments](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/) | `probara.step()`, `probara.attach()`, Vitest annotations, screenshots, console output, limits     |
| [Results and retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/)     | Run options, statuses, retries and repeats, the results file, assigning failed results            |
| [Run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/)                 | `runCasesOnly`: run only the tests of a Probara run                                               |
| [Browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)                   | The providers, the helpers in a browser, `probara.attach()` and Vite's file rules                 |
| [Sharding and projects](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/)              | One run for every shard; Vitest `projects`; several Probara projects                              |
| [CI providers](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/)                   | GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines, Buildkite     |
| [Troubleshooting](https://docs.probara.net/en/developers/ci-integrations/vitest/troubleshooting/)             | Problems and their solutions, every line the reporter logs, debugging                             |
| [Migrating](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/)                         | From the JUnit import, Qase, Allure, ReportPortal and TestRail; upgrading                         |

Related: [CI integrations overview](https://docs.probara.net/en/developers/ci-integrations/) ·
[Playwright reporter](https://docs.probara.net/en/developers/ci-integrations/playwright/) ·
[Jest reporter](https://docs.probara.net/en/developers/ci-integrations/jest/) ·
[Cypress reporter](https://docs.probara.net/en/developers/ci-integrations/cypress/) · [CLI](https://docs.probara.net/en/developers/cli/) ·
[Vitest integration guide](https://docs.probara.net/en/guides/integrations/vitest/) ·
[Source on GitHub](https://github.com/cynch-me/probara-sdk/tree/main/packages/vitest-reporter)
