Sharding Cypress and several projects

Outside GitHub Actions, each cypress run creates its own run by default, so four machines that each run part of the specs make four runs. In GitHub Actions the jobs of one workflow run share one run automatically (parallel jobs). Elsewhere, this page shows how to get one run for a whole pipeline, and how one Cypress suite reports into several Probara projects.

One run for every machine

Pattern	Choose it when
Create the run, report from each machine, close it	The usual way: each machine’s results appear as soon as it finishes
Results files, imported by one job	The machines cannot hold the token, or one job should do all the sending

Every job that reports needs PROBARA_API_TOKEN, an app token from the Cypress card in Workspace › Integrations, set up as in your CI provider’s section.

How the specs get split

Cypress has no --shard flag: each machine runs its own cypress run, and something decides which specs it runs. The reporter does not depend on what:

Splitter	What each machine runs
`--spec`	The specs you list, such as `npx cypress run --spec 'cypress/e2e/cart/**'`
Cypress Cloud `--record --parallel`	The specs Cypress Cloud hands it
A plugin such as cypress-split	Its share of the specs; register it with cypress-on-fix, since it registers `after:spec` and `after:run` too

Cypress Cloud’s own run (--ci-build-id, --group) is a Cypress Cloud concept: the reporter neither reads nor needs it. What makes the machines report into one Probara run is PROBARA_RUN_ULID, the same on every machine.

Create the run, report from each machine, close it

A first job runs probara run create, which creates an empty automated run and prints its ULID on stdout.
Every machine runs npx cypress run (with its share of the specs) with that ULID in PROBARA_RUN_ULID. A cypress run never closes a run it did not create, so they all report into the open run. Probara reconciles machines that report at the same moment: no case is added to the run twice.
A last job runs probara run close, after every machine, even when tests failed.

In a shell, the whole flow is:

PROBARA_RUN_ULID=$(npx @probara/cli run create)
export PROBARA_RUN_ULID
npx cypress run --spec 'cypress/e2e/cart/**'
npx cypress run --spec 'cypress/e2e/login/**'
npx @probara/cli run close

Assign the ULID on its own line: export PROBARA_RUN_ULID=$(...) would hide a failed creation. In CI, the jobs below pin the CLI (npx --yes @probara/cli@0.3), which is not a dependency of the reporter; with @probara/cli in your devDependencies, npx @probara/cli after npm ci runs the locked version instead. probara is the Probara CLI; it takes the same token and project. Options of a new run go to run create (--run-name, --environment, --milestone, --plan, --tag…), since the machines reuse the run and ignore their own run options.

Each machine sends its results when its own cypress run ends, so the run fills in machine by machine, not test by test.

GitHub Actions

jobs:
  probara-run:
    runs-on: ubuntu-latest
    outputs:
      ulid: ${{ steps.create.outputs.ulid }}
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - id: create
        run: |
          ulid=$(npx --yes @probara/cli@0.3 run create)
          echo "ulid=$ulid" >> "$GITHUB_OUTPUT"

  test:
    needs: probara-run
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        spec: ['cypress/e2e/cart/**', 'cypress/e2e/login/**', 'cypress/e2e/search/**']
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx cypress run --spec '${{ matrix.spec }}'
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }}

  close-probara-run:
    needs: [probara-run, test]
    if: always() && needs.probara-run.outputs.ulid != ''
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx --yes @probara/cli@0.3 run close
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }}

With the token missing (a fork), run create prints nothing and exits 0, the machines report nothing, and the close job is skipped. With Cypress Cloud, replace the matrix of specs with containers: [1, 2, 3] and run npx cypress run --record --parallel in each: the PROBARA_RUN_ULID part does not change.

GitLab CI

A dotenv artifact hands the ULID to every later job:

stages:
  - test

create-probara-run:
  stage: .pre
  image: node:22
  variables:
    PROBARA_PROJECT: SHOP
  rules:
    - if: $PROBARA_API_TOKEN
  script:
    - PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.3 run create)
    - echo "PROBARA_RUN_ULID=$PROBARA_RUN_ULID" > probara.env
  artifacts:
    reports:
      dotenv: probara.env

cypress:
  stage: test
  image: cypress/browsers:latest
  parallel:
    matrix:
      - SPEC: ['cypress/e2e/cart/**', 'cypress/e2e/login/**', 'cypress/e2e/search/**']
  variables:
    PROBARA_PROJECT: SHOP
  script:
    - npm ci
    - npx cypress run --spec "$SPEC"

close-probara-run:
  stage: .post
  image: node:22
  variables:
    PROBARA_PROJECT: SHOP
  rules:
    - if: $PROBARA_API_TOKEN
      when: always
  script:
    - npx --yes @probara/cli@0.3 run close

.pre and .post are GitLab’s built-in first and last stages; when: always closes the run even after failed tests.

Other CI providers

Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines and Buildkite follow the same three steps; their complete pipelines are in CI providers.

Alternative: results files, imported by one job

Each machine runs with reporting off and writes every result to a results file, kept as an artifact with its attachments folder. One last job, which holds the token, downloads them all and sends them.

A results file written with reporting off names no run: imported on its own, each file creates a run of its own. To get one run, create it first, import every file into it, and close it:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [cart, login, search]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx cypress run --spec 'cypress/e2e/${{ matrix.shard }}/**'
        env:
          PROBARA_ENABLED: 'false'
          PROBARA_PROJECT: SHOP
          PROBARA_RESULTS_FILE: probara-results-${{ matrix.shard }}.json
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: probara-results-${{ matrix.shard }}
          path: probara-results-*
          retention-days: 1

  report:
    needs: test
    # Forks get no token: there is nothing to send with.
    if: ${{ !cancelled() && github.event.pull_request.head.repo.fork != true }}
    runs-on: ubuntu-latest
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - uses: actions/download-artifact@v4
        with:
          pattern: probara-results-*
          merge-multiple: true
      - run: |
          PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.3 run create)
          export PROBARA_RUN_ULID
          npx --yes @probara/cli@0.3 import results 'probara-results*.json'
          npx --yes @probara/cli@0.3 run close

Give each machine its own file name (probara-results-<shard>.json), so the downloads never overwrite each other.
probara import results sends every file into PROBARA_RUN_ULID, leaves that run open (it reused it), and deletes each file once sent. run close then closes it.
Screenshots and videos are named in the file by their path on the test machine, not copied: the report job, which never had them, cannot send them. Use the shared run above when they matter.
Nothing reaches Probara until the last job: a machine that crashed leaves no results file, and its results are missing.
Re-running the report job runs run create again: the files, downloaded again as they were, go into a second run with the same results. Close or delete the one you do not keep.

If the close never ran

A run stays open until it is closed. Close it by hand, with the ULID from the log of the create job:

npx @probara/cli run close --run-ulid 01J9Z3K4M5N6P7Q8R9S0T1V2W3

Which approach loses less

Something goes wrong	Shared run (create, machines, close)	Results files, one import
A machine crashes before its run ends	Its results are lost: they are sent when its `cypress run` ends	Its file is missing: none of its results are sent
Probara is unreachable	Each machine’s results file keeps what it could not send (if one is set)	The import exits 1 and keeps in each file what it could not send
The close job does not run	The run stays open: close it by hand	Same

Several Probara projects

One Cypress suite can hold the tests of several Probara projects. List the other projects in projects, and each result goes to the project of its case, into a run of that project.

reporterOptions: { projectId: 'SHOP', projects: ['WEB'] },

PROBARA_PROJECTS=WEB lists them from the environment.

What goes where

By the case’s project. A result linked to WEB-3 goes into a run of WEB. The ids of every listed project link from titles too, and leave the key, like the configured project’s.
A test linked to cases of several projects is sent once per case, each to its project, with its files uploaded to each result.
A test without a case goes to the configured project (projectId), where it is matched by its key or created: only the configured project creates cases, under suiteUlid.
An id of a project that is not listed, in a title, is no id: it stays in the title and the key, and the test goes to the configured project. Given to probara.id(), it is a case the reporter cannot send anywhere: it is left out, and the Sending line counts it.

[probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked); 1 linked only to cases of unlisted projects

A Cypress project (the e2e or component block of defineConfig) says nothing about where a result goes: two configs reporting into the same Probara project produce the same keys, so the same cases.

One run per project

Each listed project gets its own run, created with its first result (a project without results gets no run), with the same name, description, environment (by name), tags and CI source as the configured project’s, and closed at the end like any run the reporter creates. The Recorded line names the project when there is more than one: in R-1 of WEB.

The milestone, plan and configurations are planning entities of one project: a name that exists in SHOP may not exist in WEB, where Probara would refuse the whole report. So they, like the ULID forms of the environment, milestone and configurations, only go with the configured project’s run, and one warning names them. To give another project’s run its own, create it first and reuse it:

WEB_RUN=$(npx @probara/cli run create --project WEB --milestone "Web sprint 4")
PROBARA_RUN_ULIDS="WEB=$WEB_RUN" PROBARA_PROJECTS=WEB npx cypress run

run.ulids (PROBARA_RUN_ULIDS=SHOP=<ulid>,WEB=<ulid>) names the run to reuse in each project; the configured project’s entry counts as run.ulid. A reused run stays open unless closeRun is true: close it with probara run close --project WEB --run-ulid <ulid>.

Split pipelines with several projects

Create one run per project before the machines, pass them all in PROBARA_RUN_ULIDS, and close them after the last machine:

SHOP_RUN=$(npx @probara/cli run create)
WEB_RUN=$(npx @probara/cli run create --project WEB)
export PROBARA_PROJECTS=WEB PROBARA_RUN_ULIDS="SHOP=$SHOP_RUN,WEB=$WEB_RUN"
npx cypress run --spec 'cypress/e2e/cart/**'
npx @probara/cli run close --run-ulid "$SHOP_RUN"
npx @probara/cli run close --project WEB --run-ulid "$WEB_RUN"

When the configured project’s run is reused but a listed project has no run in run.ulids, each machine creates its own run in that project, and a warning says so. A failed report stops only its project: the others go on.

Related: CI providers · Results and retries · CLI · Source: ci/sharding.md, projects.md