Reporting from CI with Vitest

The Vitest app sends your Vitest test results to Probara from any CI pipeline, with the Probara reporter for Vitest. Each Vitest run becomes an automated test run with its results, the test cases it names and their attachments.

The reporter signs in with an app token: a token that belongs to the organization, not to a person. It uses no seat, and it keeps working when the person who created it leaves. This guide covers the card in Probara and the setup; the reporter’s documentation covers every reporter option and test helper.

Before you start

Set it up

1. In Workspace › Integrations, select the Vitest card. The dialog lists what the app can do and links to this guide.

2. Open the Access tokens tab and select Create an app token.

3. Enter a Token name and select Create token. Name it after what uses it — the repository or the pipeline — so you know which one to revoke later.

4. Select Copy token in Copy your new token. This is the only time Probara shows it: store it in your CI as a secret named PROBARA_API_TOKEN before you select Done.

5. Install the reporter in your Vitest project:

   npm install --save-dev @probara/vitest-reporter

6. Add it to the reporters of vitest.config.ts, after default (the one you read in the terminal), and replace <PROJECT_CODE> with your project’s Project ID, such as SHOP:

   import { defineConfig } from 'vitest/config';
   
   export default defineConfig({
     test: {
       reporters: ['default', ['@probara/vitest-reporter', { projectId: '<PROJECT_CODE>' }]],
     },
   });

   Listing reporters replaces Vitest’s default reporter, so keep 'default' in the list to keep your terminal output.

7. Pass the secret to your tests as the PROBARA_API_TOKEN environment variable and run them. For example, in GitHub Actions:

   - name: Run Vitest tests
     run: npx vitest run
     env:
       PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}

The token never goes in vitest.config.ts: the reporter reads it from the environment. The card on the Integrations page shows how many active tokens the app has.

What the reporter sends

A field or option Probara does not know, or one deleted while the report runs, is skipped and answered as a warning, never a failure: the report still records every result. A field with an empty value is skipped without a warning. An email in assignFailedTo that matches no member who can work in the project is skipped too, with a warning that says only how many were skipped. The reporter’s documentation shows how a test sets its parameters, tags, fields, steps and links. The helpers need no setup file: the reporter adds its own to every Vitest project.

Same cases as a JUnit import

If you already send Vitest results with probara import junit on the report of Vitest’s junit reporter, the Probara reporter finds the same test cases: its automation keys equal those of the import — the test file’s path relative to the project root plus the test’s full title. Switching to the reporter therefore keeps each test linked to its case and its history. One difference is worth knowing: the reporter sends every attempt, while the import keeps only the last one.

Run only the tests of a run

To run only the tests of a run created in Probara, set PROBARA_RUN_CASES_ONLY to true and PROBARA_RUN_ULID to the run’s ULID, only in the job that runs that run’s tests. Nothing changes in vitest.config.ts. For example, in a GitHub Actions workflow you start by hand with a run_ulid input:

- name: Run the tests of a Probara run
  run: npx vitest run
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_RUN_CASES_ONLY: 'true'
    PROBARA_RUN_ULID: ${{ inputs.run_ulid }}

The reporter sends the results into that run and leaves it open.

To get the ULID, open the run, select Run actions (the menu in the run’s header), and choose Copy run ULID; the same item is in each run’s menu in the runs list. The reporter option runCasesOnly: true in vitest.config.ts turns on the same selection, but for every job: while it reports, each job without a ULID then warns and runs every test.

Before the tests start, the reporter reads the run’s cases. A test belongs to the run when its automation key is the key of one of the run’s cases, or when its title or one of its describe blocks names one of the run’s cases, such as SHOP-12. A probara.id() call does not count: it runs with the test, after the selection. Reading the case keys is the one read an app token can make.

Vitest skips every other test, and the reporter does not report it: Vitest’s summary counts it as skipped. A test of the run that the file itself skips, with test.skip or test.todo, is still reported as skipped. When the run’s cases cannot be read, or when no ULID is set, every test runs and is reported, with one warning. The run selection reference covers the matching rules and every fallback.

Sharded runs

With vitest --shard and the blob reporter, list blob before the Probara reporter in the shards’ configuration. The shards then send nothing, and vitest --merge-reports, run once with the Probara reporter in its configuration, reports every shard’s results into one run. Without blob reports, create the run first with probara run create, pass its ULID to every shard in PROBARA_RUN_ULID, and close it at the end with probara run close. In GitHub Actions the shards of one workflow run already share a run: the parallel jobs set PROBARA_CLOSE_RUN: false, and a final job runs probara run close (parallel jobs). The sharding reference has both workflows.

Manage tokens

Under App tokens, the Access tokens tab lists each active token with who created it (or Created by a removed user), when, and when it was last used (Never until its first report).

Revoke a token, and confirm, when it may have leaked or when someone who could read it leaves your team: pipelines that use it stop reporting at once, and it cannot be undone. Create a new token for them first. Because an app token outlives the person who created it, rotating it is your call: Probara never revokes it on its own.

What an app token can do

Only what the reporter does: create automated runs, send reports (which can create the test cases and suites they name), add attachments to their results, close runs, and read the case keys of a run. It cannot read anything else, cannot remove attachments, and cannot be used with Probara MCP or for any other API call. For scripts and other API clients, use a personal API token.

Runs, activity entries and attachments that an app token created show the app’s logo and name beside the token name — for example Vitest · unit-ci — in the runs list, the run and milestone summaries, Workspace activity and the attachments list.

When the plan changes

If your organization moves to the Free plan, its app tokens are kept but Probara refuses them, so pipelines stop reporting; its dialog says so and offers See plans. You can still see and revoke the tokens. They work again as soon as the organization is back on a paid plan, with nothing to set up again. Manual test runs are not affected.

Next step

• Reporter documentation — options, test helpers, browser mode and run selection.
• CI integrations — how the Vitest reporter compares with the CLI, the Playwright, the Jest and the Cypress reporters.
• Run cases API — the case keys the reporter reads to run only a run’s tests.
• Reports API — the endpoint the reporter calls, for your own tools.