# Reporting from CI with Vitest

> Create the app token the Probara Vitest reporter signs in with, add the reporter to your Vitest configuration, and see what it sends.

The **Vitest** app sends your Vitest test results to Probara from any CI pipeline, with the Probara
reporter for Vitest. Each Vitest run becomes an automated test run with its results, the test cases
it names and their attachments.

The reporter signs in with an **app token**: a token that belongs to the organization, not to a
person. It uses no seat, and it keeps working when the person who created it leaves. This guide
covers the card in Probara and the setup; the
[reporter's documentation](https://docs.probara.net/en/developers/ci-integrations/vitest/) covers every
[reporter option](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/) and test helper.

## Before you start

| You need                                                          | Why                                                                                                                         |
| ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| A paid plan (**Pro** or **Enterprise**, or any trial)             | Reporting from CI is not part of the Free plan. On Free, its dialog shows **See plans** instead of **Create an app token**. |
| A Probara **owner** or **admin**                                  | Only owners and admins can create and manage app tokens. Other members see the card without its tokens.                     |
| A Vitest project (Vitest 3.2 or later, on Node.js 22.12 or later) | The reporter runs with your tests, in the same `npx vitest run` command.                                                    |

## Set it up

1. In **Workspace › Integrations**, select the **Vitest** card. The dialog lists what the app can
   do and links to this guide.
2. Open the **Access tokens** tab and select **Create an app token**.
3. Enter a **Token name** and select **Create token**. Name it after what uses it — the repository
   or the pipeline — so you know which one to revoke later.
4. Select **Copy token** in **Copy your new token**. This is the only time Probara shows it: store it
   in your CI as a secret named `PROBARA_API_TOKEN` before you select **Done**.
5. Install the reporter in your Vitest project:

   ```bash
   npm install --save-dev @probara/vitest-reporter
   ```

6. Add it to the reporters of `vitest.config.ts`, after `default` (the one you read in the
   terminal), and replace `<PROJECT_CODE>` with your project's Project ID, such as `SHOP`:

   ```ts
   // vitest.config.ts
   import { defineConfig } from 'vitest/config';

   export default defineConfig({
     test: {
       reporters: ['default', ['@probara/vitest-reporter', { projectId: '<PROJECT_CODE>' }]],
     },
   });
   ```

   Listing `reporters` replaces Vitest's default reporter, so keep `'default'` in the list to keep
   your terminal output.

7. Pass the secret to your tests as the `PROBARA_API_TOKEN` environment variable and run them. For
   example, in GitHub Actions:

   ```yaml
   - name: Run Vitest tests
     run: npx vitest run
     env:
       PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
   ```

The token never goes in `vitest.config.ts`: the reporter reads it from the environment. The card on
the Integrations page shows how many active tokens the app has.

## What the reporter sends

| What                        | How Probara records it                                                                                                                                                                                                                                                          |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Results                     | One result per attempt: a test retried with Vitest's `retry` option sends each attempt, in order, and so does each repetition of a test run with `repeats`. The run case keeps the last outcome, and every attempt stays in the case's history.                                 |
| Skipped tests               | Tests skipped with `test.skip`, `skipIf`, `runIf` or `ctx.skip()`, `test.todo` tests, and tests left out by `.only` or `-t` are recorded as skipped. When you [run only the tests of a run](#run-only-the-tests-of-a-run), those that match none of its cases are not reported. |
| Errors                      | Every error of a failed attempt, `expect.soft` failures included, in the notes of its result.                                                                                                                                                                                   |
| Steps                       | The steps an attempt ran with `probara.step()`, as a tree with their status, duration and error, shown under the result.                                                                                                                                                        |
| Attachments                 | The files an attempt attached with `probara.attach()`, on its result — or on the step they were attached in — and the files it attached with Vitest's own `annotate`. In browser mode, the screenshot Vitest takes of a failed test too.                                        |
| Links                       | The links an attempt added with `probara.link()`, shown with the result. The issues it named with `probara.issue()` become links only when the reporter's `issueUrlTemplate` option, a URL with `%s` where the issue id goes, turns their ids into links.                       |
| Parameters                  | The parameters an attempt set with `probara.parameters()`, such as the user or the locale. A name the case declares selects its run case. The Vitest project it ran in, when your configuration has `projects`, is context: it is shown with the result only.                   |
| New test cases              | A test with no case yet creates one, in a suite named after its test file — with `keyIncludesFile: false`, after its outermost `describe` block, when it has one — unless `probara.suite()` names another.                                                                      |
| Existing test cases         | An archived case the test reports to is restored. A case found by its automation key moves to the suite the test is in now; a case linked by its ID keeps its suite. Its title, steps and fields are never changed.                                                             |
| Fields of a new case        | Its description, tags (Vitest's own test tags included), steps and fields — system fields such as priority or severity, and custom fields by name. Only when the report creates the case: an existing case is never changed.                                                    |
| Assignees of failed results | With the reporter's `assignFailedTo` option, a failed result is assigned to the people it names, in turn — only when its run case has no assignee yet, and only to members who can work in the project. An existing assignee is never replaced.                                 |

A field or option Probara does not know, or one deleted while the report runs, is skipped and
answered as a warning, never a failure: the report still records every result. A field with an
empty value is skipped without a warning. An email in `assignFailedTo` that matches no member who
can work in the project is skipped too, with a warning that says only how many were skipped. The
[reporter's documentation](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/) shows how a test sets
its parameters, tags, fields, steps and links. The helpers need no setup file: the reporter adds its
own to every Vitest project.

## Same cases as a JUnit import

If you already send Vitest results with `probara import junit` on the report of Vitest's `junit`
reporter, the Probara reporter finds the same test cases: its automation keys equal those of the
import — the test file's path relative to the project root plus the test's full title. Switching to
the reporter therefore keeps each test linked to its case and its history. One difference is worth
knowing: the reporter sends **every attempt**, while the import keeps only the last one.

## Run only the tests of a run

To run only the tests of a run created in Probara, set `PROBARA_RUN_CASES_ONLY` to `true` and
`PROBARA_RUN_ULID` to the run's ULID, only in the job that runs that run's tests. Nothing changes in
`vitest.config.ts`. For example, in a GitHub Actions workflow you start by hand with a `run_ulid`
input:

```yaml
- name: Run the tests of a Probara run
  run: npx vitest run
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_RUN_CASES_ONLY: 'true'
    PROBARA_RUN_ULID: ${{ inputs.run_ulid }}
```

The reporter sends the results into that run and leaves it open.

To get the ULID, open the run, select **Run actions** (the menu in the run's header), and choose
**Copy run ULID**; the same item is in each run's menu in the runs list. The reporter option
`runCasesOnly: true` in `vitest.config.ts` turns on the same selection, but for every job: while it
reports, each job without a ULID then warns and runs every test.

Before the tests start, the reporter reads the run's cases. A test belongs to the run when its
automation key is the key of one of the run's cases, or when its title or one of its `describe`
blocks names one of the run's cases, such as `SHOP-12`. A `probara.id()` call does not count: it
runs with the test, after the selection. Reading the case keys is the one read an app token can
make.

Vitest skips every other test, and the reporter does not report it: Vitest's summary counts it as
skipped. A test of the run that the file itself skips, with `test.skip` or `test.todo`, is still
reported as skipped. When the run's cases cannot be read, or when no ULID is set, every test runs
and is reported, with one warning. The
[run selection reference](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/) covers the matching
rules and every fallback.

## Sharded runs

With `vitest --shard` and the `blob` reporter, list `blob` before the Probara reporter in the
shards' configuration. The shards then send nothing, and `vitest --merge-reports`, run once with the
Probara reporter in its configuration, reports every shard's results into one run. Without blob
reports, create the run first with `probara run create`, pass its ULID to every shard in
`PROBARA_RUN_ULID`, and close it at the end with `probara run close`. In GitHub Actions the shards of
one workflow run already share a run: the parallel jobs set `PROBARA_CLOSE_RUN: false`, and a final
job runs `probara run close`
([parallel jobs](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job)).
The [sharding reference](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/) has both workflows.

## Manage tokens

Under **App tokens**, the **Access tokens** tab lists each active token with who created it (or
**Created by a removed user**), when, and when it was last used (**Never** until its first report).

**Revoke** a token, and confirm, when it may have leaked or when someone who could read it leaves
your team: pipelines that use it stop reporting at once, and it cannot be undone. Create a new token
for them first. Because an app token outlives the person who created it, rotating it is your call:
Probara never revokes it on its own.

## What an app token can do

Only what the reporter does: create automated runs, send reports (which can create the test cases
and suites they name), add attachments to their results, close runs, and read the case keys of a
run. It cannot read anything else, cannot remove attachments, and cannot be used with
[Probara MCP](https://docs.probara.net/en/developers/mcp/) or for any other API call. For scripts and other API clients, use
a personal [API token](https://docs.probara.net/en/developers/authentication/).

Runs, activity entries and attachments that an app token created show the app's logo and name beside
the token name — for example **Vitest · unit-ci** — in the runs list, the run and milestone
summaries, [Workspace activity](https://docs.probara.net/en/guides/workspace/activity/) and the attachments list.

## When the plan changes

If your organization moves to the Free plan, its app tokens are kept but Probara refuses them, so
pipelines stop reporting; its dialog says so and offers **See plans**. You can still see and revoke the
tokens. They work again as soon as the organization is back on a paid plan, with nothing to set up
again. Manual test runs are not affected.

## Next step

- [Reporter documentation](https://docs.probara.net/en/developers/ci-integrations/vitest/) —
  [options](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/),
  [test helpers](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/), browser mode and
  [run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/).
- [CI integrations](https://docs.probara.net/en/developers/ci-integrations/) — how the Vitest reporter compares with the
  [CLI](https://docs.probara.net/en/developers/cli/), the [Playwright](https://docs.probara.net/en/guides/integrations/playwright/), the
  [Jest](https://docs.probara.net/en/guides/integrations/jest/) and the [Cypress](https://docs.probara.net/en/guides/integrations/cypress/)
  reporters.
- [Run cases API](https://docs.probara.net/en/developers/api/run-cases/#list-a-runs-case-keys) — the case keys the reporter
  reads to run only a run's tests.
- [Reports API](https://docs.probara.net/en/developers/api-reference/#reports) — the endpoint the reporter calls, for
  your own tools.