# Sharding Vitest and Vitest projects > One Probara run for every shard of a sharded Vitest pipeline, from the merged blob reports or a shared run, Vitest projects, and reporting one test suite into several Probara projects. Outside GitHub Actions, each `vitest run --shard` makes its own run unless the shards are tied together. In GitHub Actions the shards of one workflow run share one run automatically ([parallel jobs](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job)). Elsewhere, this page shows the two ways to get **one** run for a whole pipeline, how Vitest's own `projects` behave, and how one test suite reports into several Probara projects. The workers of one `vitest run` need nothing: they all report through the one reporter of Vitest's main process, into one run. ## One run for every shard | Pattern | Choose it when | | ---------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | [1. Merge the blob reports](#pattern-1-merge-the-blob-reports) | The usual way, and Vitest's own: one job reports every shard, and the shards need no token | | [2. Create the run, report from each shard, close it](#pattern-2-create-the-run-report-from-each-shard-close-it) | Each shard's results should reach Probara as soon as it ends, `runCasesOnly` should apply, or no job can merge at the shards' path | The jobs that report need `PROBARA_API_TOKEN`, an app token from the **Vitest** card in **Workspace › Integrations**, set up as in your [CI provider's section](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/). ### Pattern 1: merge the blob reports Each shard writes a Vitest blob report, with the reporter beside the `blob` reporter, and one job merges the blobs: `vitest --merge-reports` reports every attempt of every shard, with its steps and files, into one run, and closes it. 1. Each shard runs with both reporters on its command line. `--reporter` replaces the reporters of the config, so name this one beside `blob`: `npx vitest run --shard=1/4 --reporter=blob --reporter=@probara/vitest-reporter`. The shard writes its blob, keeps the files its tests attached, sends nothing, and says so. It needs no token, and none of the reporter's options: the merge reports with those of the config. 2. Copy the blobs and the attachments of every shard into the job that merges, at the same path as on the shards ([below](#carry-the-blobs-and-the-attachments)). 3. That job runs `npx vitest --merge-reports` with the token: the reporter of the config sends every shard's results into one run. In one shell, the whole flow is: ```text $ npx vitest run --shard=1/2 --reporter=blob --reporter=@probara/vitest-reporter [probara] Sharded run with the blob reporter: the results are reported by vitest --merge-reports $ npx vitest run --shard=2/2 --reporter=blob --reporter=@probara/vitest-reporter [probara] Sharded run with the blob reporter: the results are reported by vitest --merge-reports $ npx vitest --merge-reports [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked) [probara] Recorded 3 results (2 new cases, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1 ``` The complete workflows are in the [GitHub Actions](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/#sharded-jobs-one-run-from-the-blobs) and [GitLab CI](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/#parallel-shards-one-run-from-the-blobs) sections of CI providers. #### The reporter goes beside `blob` | The shards run with | What the merge sends | | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--reporter=blob --reporter=@probara/vitest-reporter` | Every attempt, the `probara.*` calls and their files | | `--reporter=blob` alone | Only the final attempt of a retried test (one warning says how many), and no `probara.*` call: the reporter never ran in the shards ([when retries run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#when-retries-run)) | Keep `blob` out of the config `vitest --merge-reports` reads: Vitest refuses to merge with it (``Cannot merge reports when `--reporter=blob` is used``). #### Carry the blobs and the attachments | What | Vitest 3 and 4 | Vitest 5 | | ----------------------------------------------- | ---------------------- | ---------------------- | | The blobs, which `vitest --merge-reports` reads | `.vitest-reports/` | `.vitest/blob/` | | `attachmentsDir`: the files the tests attached | `.vitest-attachments/` | `.vitest/attachments/` | Keep both folders of every shard as artifacts, and download them all into the same folders of the merging job: each blob has a name of its own (`blob-1-4.json`), and so has each copy of an attached file. A file the merge cannot find is skipped, with the warning `Skipped an attachment: its file is missing or unreadable`, and its result is sent all the same ([where the copies are kept](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#where-the-copies-are-kept)). #### Merge at the same path A blob names each test file, and each attached file, by its full path on the shard, and an automation key starts with the test file's path from the root of the merging run. Merged at another path, every key changes, and every test gets a new case. Run the merge in a checkout at the same path as the shards. The logs show it: the root the merge's `RUN` line names must be the folder that holds the `.vitest` folder (`.vitest-reports` on Vitest 3 and 4) of each shard's `blob report written to` line. #### The same Vitest everywhere `vitest --merge-reports` reads only the blobs of its own Vitest version, and fails on any other. Install from the lockfile in every job (`npm ci`), and upgrade the shards and the merging job together ([any upgrade of a sharded pipeline](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/#any-upgrade-of-a-sharded-pipeline)). #### Run selection `runCasesOnly` does not apply to shards that write a blob: each shard warns (`runCasesOnly does not apply to a sharded run with the blob reporter: every test runs, and vitest --merge-reports reports them all`), runs every test, and the merge reports them all. To run only the tests of a Probara run on shards, use pattern 2 ([run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/#setups)). ### Pattern 2: create the run, report from each shard, close it 1. A first job runs `probara run create`, which creates an empty automated run and prints its ULID on stdout. 2. Every shard runs `npx vitest run --shard=/` with that ULID in `PROBARA_RUN_ULID`, and sends its results when it ends. A shard never closes a run it did not create, so they all report into the open run. Probara reconciles shards that report at the same moment: no case is added to the run twice. 3. A last job runs `probara run close`, after every shard, **even when shards failed**. In a shell, the whole flow is: ```bash PROBARA_RUN_ULID=$(npx @probara/cli run create) export PROBARA_RUN_ULID npx vitest run --shard=1/2 npx vitest run --shard=2/2 npx @probara/cli run close ``` Assign the ULID on its own line: `export PROBARA_RUN_ULID=$(...)` would hide a failed creation. `probara` is the [Probara CLI](https://docs.probara.net/en/developers/cli/); it takes the same token and project. Options of a new run go to `run create` (`--run-name`, `--environment`, `--milestone`, `--plan`, `--tag`...): the shards reuse the run, and ignore their own run options with one warning ([an existing run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#an-existing-run)). With `runCasesOnly`, each shard reads the cases of the run, and runs only those of its tests that belong to the run ([run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/#setups)). #### GitHub Actions ```yaml jobs: probara-run: runs-on: ubuntu-latest outputs: ulid: ${{ steps.create.outputs.ulid }} env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP # Fork pull requests get no secrets: report nothing there. PROBARA_ENABLED: ${{ secrets.PROBARA_API_TOKEN != '' }} steps: - uses: actions/setup-node@v5 with: node-version: 22 - id: create run: | ulid=$(npx @probara/cli run create) echo "ulid=$ulid" >> "$GITHUB_OUTPUT" test: needs: probara-run runs-on: ubuntu-latest strategy: fail-fast: false matrix: shardIndex: [1, 2, 3, 4] shardTotal: [4] steps: - uses: actions/checkout@v5 - uses: actions/setup-node@v5 with: node-version: 22 - run: npm ci - run: npx vitest run --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP PROBARA_ENABLED: ${{ secrets.PROBARA_API_TOKEN != '' }} PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }} close-probara-run: needs: [probara-run, test] if: always() && needs.probara-run.outputs.ulid != '' runs-on: ubuntu-latest steps: - uses: actions/setup-node@v5 with: node-version: 22 - run: npx @probara/cli run close env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }} ``` Without the token (a fork), `PROBARA_ENABLED` is `false`: `run create` prints nothing and exits 0, the shards report nothing, and the close job is skipped. #### GitLab CI A dotenv artifact hands the ULID to every later job: ```yaml stages: - test create-probara-run: stage: .pre image: node:22 variables: PROBARA_PROJECT: SHOP rules: - if: $PROBARA_API_TOKEN script: - PROBARA_RUN_ULID=$(npx @probara/cli run create) - echo "PROBARA_RUN_ULID=$PROBARA_RUN_ULID" > probara.env artifacts: reports: dotenv: probara.env vitest: stage: test image: node:22 parallel: 4 variables: PROBARA_PROJECT: SHOP script: - npm ci - npx vitest run --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL close-probara-run: stage: .post image: node:22 variables: PROBARA_PROJECT: SHOP rules: - if: $PROBARA_API_TOKEN when: always script: - npx @probara/cli run close ``` `.pre` and `.post` are GitLab's built-in first and last stages; `when: always` closes the run even after failed shards. A pipeline without the variable (a fork's) has neither job, and its shards report nothing. #### Other CI providers [CI providers](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/) shows the same three steps for CircleCI, Jenkins, Azure Pipelines, Bitbucket Pipelines and Buildkite. Pattern 1 works on them too when the merging job gets every shard's folders at the shards' path, and runs after failed shards: Bitbucket skips the steps after a failed one. ### If the close never ran A run stays open until it is closed. Close it by hand, with the ULID from the log of the create job: ```bash npx @probara/cli run close --run-ulid 01J9Z3K4M5N6P7Q8R9S0T1V2W3 ``` ### Which pattern loses less | Something goes wrong | Pattern 1 (merge) | Pattern 2 (shared run) | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | A shard is killed before Vitest ends | Its blob is missing: the other shards are reported | It sends nothing: the other shards are reported | | Probara is unreachable | The merging job's [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#in-ci) keeps what it could not send (if set) | Each shard's [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#in-ci) keeps what it could not send (if set) | | The close job does not run | Nothing to close: the merge closes its own run | The run stays open: [close it by hand](#if-the-close-never-ran) | In both, the exit code of every job stays the tests' own: a reporting problem never fails a shard or the merge. ## Vitest projects Two different things are called projects here. Vitest's [`projects`](https://vitest.dev/guide/projects) split one `vitest` command into several configurations (a `unit` and a `smoke` one, say); the reporter's `projects` option lets one test suite report to [several Probara projects](#several-probara-projects). Each works on its own. Register the reporter once, in the root config, next to `projects`: Vitest runs one set of reporters for all its projects, and the reporter sends every project's tests into one Probara run. It adds its setup file and runner to each project itself. ```ts // vitest.config.mts import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { reporters: ['default', ['@probara/vitest-reporter', { projectId: 'SHOP' }]], projects: [ { test: { name: 'unit', include: ['tests/**/*.test.ts'] } }, { test: { name: 'smoke', include: ['tests/cart.test.ts'] } }, ], }, }); ``` ```ts // tests/cart.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('adds an item', () => { probara.parameters({ currency: 'EUR' }); }); ``` ```ts // tests/orders.test.ts import { test } from 'vitest'; test('lists the orders', () => {}); ``` ```json [ { "automationKey": "tests/cart.test.ts > adds an item", "parameters": { "project": "unit", "currency": "EUR" } }, { "automationKey": "tests/orders.test.ts > lists the orders", "parameters": { "project": "unit" } }, { "automationKey": "tests/cart.test.ts > adds an item", "parameters": { "project": "smoke", "currency": "EUR" } } ] ``` - **Keys have no project.** As in Vitest's JUnit output, a Vitest project adds nothing to the automation key: the file (relative to the Vitest root), then the names. Moving tests between Vitest projects keeps their cases. - **A file that two projects run** (their `include` overlap, as `tests/cart.test.ts` above) is reported once per project, under the same key: two results of one case, like two retries. - **The `project` parameter** names the Vitest project of each result when the config has more than one. A config with one project, or none, sends no `project` parameter. A `project` you set with [`probara.parameters()`](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#parameters) wins over Vitest's; `attempt` and `repeat` win over both. - **Each project's run of a file is a test of its own**: its attempts, `probara.*` calls and files stay with that project's result, as `currency` above shows. - **Nothing to add per project.** The reporter puts its setup file first in each project's `setupFiles` (your own setup files keep their order) and wraps each project's `runner`, so retries, [run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/) and the helpers work in every project. A browser project gets the browser setup file instead ([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)). ## Several Probara projects One Vitest suite can hold the tests of several Probara projects: a shared suite whose cases live in `SHOP`, `WEB` and `API`. List the other projects in `projects`, and each result goes to the project of its case, into a run of that project. ```ts reporters: ['default', ['@probara/vitest-reporter', { projectId: 'SHOP', projects: ['WEB'] }]], ``` ```ts // tests/shared.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('SHOP-12 pays', () => {}); test('WEB-3 shows the landing page', () => {}); test('API-7 answers the health check', () => {}); test('opens the help', () => { probara.id('API-9'); }); ``` ```text $ npx vitest run [probara] Did not send the results linked to cases of API: API is neither the project (SHOP) nor one of projects (PROBARA_PROJECTS) (first seen in "opens the help"; repeats are logged at debug) [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked); 1 linked only to cases of unlisted projects [probara] Recorded 2 results (1 new case, 0 unmatched) in R-1 of SHOP (closed): https://app.probara.net/projects/SHOP/runs/R-1 [probara] Recorded 1 result (0 new cases, 0 unmatched) in R-2 of WEB (closed): https://app.probara.net/projects/WEB/runs/R-2 ``` `PROBARA_PROJECTS=WEB,API` lists them from the environment. ### What goes where - **By the case's project.** A result linked to `WEB-3` goes into a run of `WEB`. The ids of every listed project link from titles too, and leave the key, like the configured project's. - **A test linked to cases of several projects** is sent once per case, each to its project, with its files uploaded to each result. - **A test without a case** goes to the configured project (`projectId`), where it is matched by its key or created: automation keys belong to one project, so only the configured project creates cases, under `suiteUlid`. - **An id of a project that is not listed** in a title (`API-7` above) is no id: it stays in the title and the key, like any text, and the test goes to the configured project. Given to `probara.id()` (`API-9`), it is a case the reporter cannot send anywhere: the result would land in the wrong project, where Probara refuses its id. It is left out, with one warning per project, and the count of the `Sending` line says so. List the project to report it. ### One run per project Each listed project gets its own run, created with its first result (a project without results gets no run), with the same name, description, environment (by name), tags and CI source as the configured project's, and closed at the end like any run the reporter creates. The milestone, plan and configurations are planning entities of one project: a name that exists in `SHOP` may not exist in `WEB`, where Probara would refuse the whole report. So `run.milestone`, `run.plan` and `run.configurations`, like their ULID forms (`run.milestoneId`, `run.configurationUlids`, `run.environmentId`), only go with the configured project's run, and one warning names them. To give another project's run its own, create it first and reuse it: ```bash WEB_RUN=$(npx @probara/cli run create --project WEB --milestone "Web sprint 4") PROBARA_RUN_ULIDS="WEB=$WEB_RUN" PROBARA_PROJECTS=WEB npx vitest run ``` `run.ulids` (`PROBARA_RUN_ULIDS=SHOP=,WEB=`) names the run to reuse in each project; the configured project's entry counts as `run.ulid`. A reused run stays open unless `closeRun` is `true`: close it with `probara run close --project WEB --run-ulid `. ### Sharded jobs with several projects Shards that each send their own results (no `blob` reporter) share runs the same way: create one run per project before the shards, pass them all in `PROBARA_RUN_ULIDS`, and close them after the last shard ([pattern 2](#pattern-2-create-the-run-report-from-each-shard-close-it)): ```bash SHOP_RUN=$(npx @probara/cli run create) WEB_RUN=$(npx @probara/cli run create --project WEB) export PROBARA_PROJECTS=WEB PROBARA_RUN_ULIDS="SHOP=$SHOP_RUN,WEB=$WEB_RUN" npx vitest run --shard=1/2 npx vitest run --shard=2/2 npx @probara/cli run close --run-ulid "$SHOP_RUN" npx @probara/cli run close --project WEB --run-ulid "$WEB_RUN" ``` When the configured project's run is reused but a listed project has no run in `run.ulids`, each shard creates its own run in that project, and a warning says so. Shards that write a blob send nothing: `vitest --merge-reports` reports every project's results once, as one run does. ### Failures A failed report stops only its project: the others go on. The log names the project of every error (`WEB: ...`), and a [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) keeps what each project could not send, with the run it goes back into: one file for the whole run, which `probara import results 'probara-results*.json'` sends to each project. Related: [CI providers](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/) · [Results and retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/) · [CLI](https://docs.probara.net/en/developers/cli/) · Source: [ci/sharding.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/ci/sharding.md), [projects.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/projects.md)