# Sharding Jest and Jest projects

> One Probara run for every shard of a sharded Jest pipeline, Jest projects, and reporting one test suite into several Probara projects.

Outside GitHub Actions, each `jest` command creates its own run by default, so four shards
(`--shard=1/4` ... `--shard=4/4`) make four runs. In GitHub Actions the shards of one workflow run
share one run automatically
([parallel jobs](https://docs.probara.net/en/developers/ci-integrations/jest/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job)).
Elsewhere, this page shows how to get **one** run for a whole pipeline, how Jest's own `projects`
behave, and how one test suite reports into several Probara projects.

Parallel workers of one `jest` (`--maxWorkers`) need nothing: they all report through the one
reporter of Jest's main process, into one run.

## One run for every shard

| Pattern                                                                                             | Choose it when                                                         |
| --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| [Create the run, report from each shard, close it](#create-the-run-report-from-each-shard-close-it) | The usual way: results appear while the shards run                     |
| [Results files, imported by one job](#alternative-results-files-imported-by-one-job)                | The shards cannot hold the token, or one job should do all the sending |

Every job that reports needs `PROBARA_API_TOKEN`, an app token from the **Jest** card in
**Workspace › Integrations**, set up as in your
[CI provider's section](https://docs.probara.net/en/developers/ci-integrations/jest/ci-providers/).

### Create the run, report from each shard, close it

1. A first job runs `probara run create`, which creates an empty automated run and prints its ULID
   on stdout.
2. Every shard runs `npx jest --shard=<i>/<n>` with that ULID in `PROBARA_RUN_ULID`. A shard never
   closes a run it did not create, so they all report into the open run. Probara reconciles shards
   that report at the same moment: no case is added to the run twice.
3. A last job runs `probara run close`, after every shard, **even when shards failed**.

In a shell, the whole flow is:

```bash
PROBARA_RUN_ULID=$(npx @probara/cli run create)
export PROBARA_RUN_ULID
npx jest --shard=1/2
npx jest --shard=2/2
npx @probara/cli run close
```

Assign the ULID on its own line: `export PROBARA_RUN_ULID=$(...)` would hide a failed creation. In
CI, the jobs below pin the CLI (`npx --yes @probara/cli@0.2`), which is not a dependency of the
reporter; with `@probara/cli` in your `devDependencies`, `npx @probara/cli` after `npm ci` runs the
locked version instead.
`probara` is the [Probara CLI](https://docs.probara.net/en/developers/cli/); it takes the same token and project. Options of
a new run go to `run create` (`--run-name`, `--environment`, `--milestone`, `--plan`, `--tag`...),
since the shards reuse the run and ignore their own run options.

### GitHub Actions

```yaml
jobs:
  probara-run:
    runs-on: ubuntu-latest
    outputs:
      ulid: ${{ steps.create.outputs.ulid }}
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
      # A fork's build gets no secret: reporting off, so run create prints nothing and exits 0.
      PROBARA_ENABLED: ${{ secrets.PROBARA_API_TOKEN != '' }}
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - id: create
        run: |
          ulid=$(npx --yes @probara/cli@0.2 run create)
          echo "ulid=$ulid" >> "$GITHUB_OUTPUT"

  test:
    needs: probara-run
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx jest --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }}

  close-probara-run:
    needs: [probara-run, test]
    if: always() && needs.probara-run.outputs.ulid != ''
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx --yes @probara/cli@0.2 run close
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }}
```

Without a token the CLI exits 2, so the first job sets `PROBARA_ENABLED` from whether the secret is
there: in a fork's build, `run create` prints nothing and exits 0, the shards report nothing, and
the close job is skipped.

### GitLab CI

A dotenv artifact hands the ULID to every later job:

```yaml
stages:
  - test

create-probara-run:
  stage: .pre
  image: node:22
  variables:
    PROBARA_PROJECT: SHOP
  rules:
    - if: $PROBARA_API_TOKEN
  script:
    - PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.2 run create)
    - echo "PROBARA_RUN_ULID=$PROBARA_RUN_ULID" > probara.env
  artifacts:
    reports:
      dotenv: probara.env

jest:
  stage: test
  image: node:22
  parallel: 4
  variables:
    PROBARA_PROJECT: SHOP
  script:
    - npm ci
    - npx jest --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

close-probara-run:
  stage: .post
  image: node:22
  variables:
    PROBARA_PROJECT: SHOP
  rules:
    - if: $PROBARA_API_TOKEN
      when: always
  script:
    - npx --yes @probara/cli@0.2 run close
```

`.pre` and `.post` are GitLab's built-in first and last stages; `when: always` closes the run even
after failed shards.

### Other CI providers

Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines and Buildkite follow the same three steps;
their complete sharded pipelines are in
[CI providers](https://docs.probara.net/en/developers/ci-integrations/jest/ci-providers/).

### Alternative: results files, imported by one job

Each shard runs with reporting off and writes every result to a
[results file](https://docs.probara.net/en/developers/ci-integrations/jest/results-and-retries/#results-file), kept as an
artifact with its attachments folder. One last job, which holds the token, downloads them all and
sends them. The attachments are referenced relative to each results file, so the files and their
folders can be downloaded anywhere.

A results file written with reporting off names no run: imported on its own, each file creates a run
of its own. To get one run, create it first, import every file into it, and close it:

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx jest --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          PROBARA_ENABLED: 'false'
          PROBARA_PROJECT: SHOP
          PROBARA_RESULTS_FILE: probara-results-${{ matrix.shardIndex }}.json
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: probara-results-${{ matrix.shardIndex }}
          path: probara-results-*
          retention-days: 1

  report:
    needs: test
    # Forks get no token: there is nothing to send with.
    if: ${{ !cancelled() && github.event.pull_request.head.repo.fork != true }}
    runs-on: ubuntu-latest
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - uses: actions/download-artifact@v4
        with:
          pattern: probara-results-*
          merge-multiple: true
      - run: |
          PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.2 run create)
          export PROBARA_RUN_ULID
          npx --yes @probara/cli@0.2 import results 'probara-results*.json'
          npx --yes @probara/cli@0.2 run close
```

- Give each shard its own file name (`probara-results-<shard>.json`), so the downloads never
  overwrite each other.
- `probara import results` sends every file into `PROBARA_RUN_ULID`, leaves that run open (it
  reused it), and deletes each file once sent. `run close` then closes it.
- Nothing reaches Probara until the last job: a crashed shard leaves no results file, and its
  results are missing.
- Re-running the report job runs `run create` again: the shards' files, downloaded again as they
  were, go into a second run with the same results. Close or delete the one you do not keep.

### If the close never ran

A run stays open until it is closed. Close it by hand, with the ULID from the log of the create job:

```bash
npx @probara/cli run close --run-ulid 01J9Z3K4M5N6P7Q8R9S0T1V2W3
```

### Which approach loses less

| Something goes wrong               | Shared run (create, shards, close)                                     | Results files, one import                                        |
| ---------------------------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------------- |
| A shard crashes before it finishes | Its results so far are in the run                                      | Its file is missing: none of its results are sent                |
| Probara is unreachable             | Each shard's results file keeps what it could not send (if one is set) | The import exits 1 and keeps in each file what it could not send |
| The close job does not run         | The run stays open: [close it by hand](#if-the-close-never-ran)        | Same                                                             |

## Jest projects

Two different things are called projects here. Jest's
[`projects`](https://jestjs.io/docs/configuration#projects-arraystring--projectconfig) split one
`jest` command into several configs (a `node` and a `jsdom` one, say); the reporter's `projects`
option lets one test suite report to [several Probara projects](#several-probara-projects). Each
works on its own.

Register the reporter once, at the top level of the config, next to `projects`: Jest runs one set of
reporters for all its projects, and the reporter sends every project's tests into one Probara run.

```js
// jest.config.js
module.exports = {
  reporters: ['default', ['@probara/jest-reporter', { projectId: 'SHOP' }]],
  projects: [
    {
      displayName: 'api',
      testEnvironment: 'node',
      testMatch: ['<rootDir>/tests/api/**/*.test.js'],
    },
    { displayName: 'ui', testEnvironment: 'jsdom', testMatch: ['<rootDir>/tests/ui/**/*.test.js'] },
  ],
};
```

```json
[
  { "automationKey": "tests/ui/cart.test.js > shows the cart" },
  { "automationKey": "tests/api/orders.test.js > lists the orders" }
]
```

- **Keys have no project.** As in jest-junit's report, a Jest project adds nothing to the automation
  key: the file (relative to the directory `jest` runs in, not to the project's `rootDir`), then the
  names. Moving tests between Jest projects keeps their cases.
- **A file that two projects run** (their `testMatch` overlap, such as a shared folder) is reported
  once per project, under the same key: two results of one case, like two retries.
- **Helpers in a file two projects run at once.** Jest may run both at the same moment, in two
  workers, and a `probara.*` call cannot tell which project's run it belongs to. What the helpers
  said about that file's tests is then left out, with one warning that names the file; the results
  are sent all the same. Give each project its own files, or run with `--runInBand`, to keep them.
- **Setup files per project.** `setupFilesAfterEnv: ['@probara/jest-reporter/setup']` (for
  `captureOutput` and [run selection](https://docs.probara.net/en/developers/ci-integrations/jest/run-selection/)) is a
  project option: give it to every project whose tests need it.
- **`reporters` inside a project is ignored** by Jest: register the reporter at the top level.

### `{displayName}` in a title

jest-junit fills `{displayName}` in a test's name with the name of the Jest project that runs it, and
so does the reporter, so a test that runs in two projects can have a key, and a case, per project:

```js
// jest.config.js
module.exports = {
  reporters: ['default', ['@probara/jest-reporter', { projectId: 'SHOP' }]],
  projects: [
    { displayName: 'node', testEnvironment: 'node' },
    { displayName: 'browser', testEnvironment: 'jsdom' },
  ],
};
```

```js
// tests/format.test.js
test('formats prices in {displayName}', () => {});
```

```json
[
  { "automationKey": "tests/format.test.js > formats prices in browser" },
  { "automationKey": "tests/format.test.js > formats prices in node" }
]
```

A project without a `displayName` fills in `undefined`, as jest-junit does. When the two projects
run such a file at the same moment, each sends only the last attempt of such a test, with one
warning: Jest does not say which project a single attempt belongs to until the file ends.

## Several Probara projects

One Jest suite can hold the tests of several Probara projects: a shared suite whose cases live in
`SHOP`, `WEB` and `API`. List the other projects in `projects`, and each result goes to the project
of its case, into a run of that project.

```js
reporters: ['default', ['@probara/jest-reporter', { projectId: 'SHOP', projects: ['WEB'] }]],
```

```js
// tests/shared.test.js
const { probara } = require('@probara/jest-reporter');

test('SHOP-12 pays', () => {});

test('WEB-3 shows the landing page', () => {});

test('API-7 answers the health check', () => {});

test('opens the help', () => {
  probara.id('API-9');
});
```

```text
$ npx jest
[probara] Did not send the results linked to cases of API: API is neither the project (SHOP) nor one of projects (PROBARA_PROJECTS) (first seen in "opens the help"; repeats are logged at debug)
[probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked); 1 linked only to cases of unlisted projects
[probara] Recorded 2 results (1 new case, 0 unmatched) in R-1 of SHOP (closed): https://app.probara.net/projects/SHOP/runs/R-1
[probara] Recorded 1 result (0 new cases, 0 unmatched) in R-2 of WEB (closed): https://app.probara.net/projects/WEB/runs/R-2
```

`PROBARA_PROJECTS=WEB,API` lists them from the environment.

### What goes where

- **By the case's project.** A result linked to `WEB-3` goes into a run of `WEB`. The ids of every
  listed project link from titles too, and leave the key, like the configured project's.
- **A test linked to cases of several projects** is sent once per case, each to its project, with
  its files uploaded to each result.
- **A test without a case** goes to the configured project (`projectId`), where it is matched by its
  key or created: automation keys belong to one project, so only the configured project creates
  cases, under `suiteUlid`.
- **An id of a project that is not listed** in a title (`API-7` above) is no id: it stays in the
  title and the key, like any text, and the test goes to the configured project. Given to
  `probara.id()` (`API-9`), it is a case the reporter cannot send anywhere: it is left out, with one
  warning per project, and the count in the log says so. List the project to report it.

### One run per project

Each listed project gets its own run, created with its first result (a project without results gets
no run), with the same name, description, environment (by name), tags and CI source as the
configured project's, and closed at the end like any run the reporter creates.

The milestone, plan and configurations are planning entities of one project: a name that exists in
`SHOP` may not exist in `WEB`, where Probara would refuse the whole report. So `run.milestone`,
`run.plan` and `run.configurations`, like their ULID forms (`run.milestoneId`,
`run.configurationUlids`, `run.environmentId`), only go with the configured project's run, and one
warning names them. To give another project's run its own, create it first and reuse it:

```bash
WEB_RUN=$(npx @probara/cli run create --project WEB --milestone "Web sprint 4")
PROBARA_RUN_ULIDS="WEB=$WEB_RUN" PROBARA_PROJECTS=WEB npx jest
```

`run.ulids` (`PROBARA_RUN_ULIDS=SHOP=<ulid>,WEB=<ulid>`) names the run to reuse in each project; the
configured project's entry counts as `run.ulid`. A reused run stays open unless `closeRun` is
`true`: close it with `probara run close --project WEB --run-ulid <ulid>`.

### Sharded jobs with several projects

Create one run per project before the shards, pass them all in `PROBARA_RUN_ULIDS`, and close them
after the last shard:

```bash
SHOP_RUN=$(npx @probara/cli run create)
WEB_RUN=$(npx @probara/cli run create --project WEB)
export PROBARA_PROJECTS=WEB PROBARA_RUN_ULIDS="SHOP=$SHOP_RUN,WEB=$WEB_RUN"
npx jest --shard=1/2
npx jest --shard=2/2
npx @probara/cli run close --run-ulid "$SHOP_RUN"
npx @probara/cli run close --project WEB --run-ulid "$WEB_RUN"
```

When the configured project's run is reused but a listed project has no run in `run.ulids`, each
shard creates its own run in that project, and a warning says so.

### Failures

A failed report stops only its project: the others go on. The log names the project of every error
(`WEB: ...`), and a results file keeps what each project could not send, with the run it goes back
into: one file for the whole run, which `probara import results 'probara-results*.json'` sends to
each project.

Related: [CI providers](https://docs.probara.net/en/developers/ci-integrations/jest/ci-providers/) ·
[Results and retries](https://docs.probara.net/en/developers/ci-integrations/jest/results-and-retries/) ·
[CLI](https://docs.probara.net/en/developers/cli/) · Source:
[ci/sharding.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/jest-reporter/docs/ci/sharding.md),
[projects.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/jest-reporter/docs/projects.md)