# Sharding Playwright runs > Get one Probara run for a sharded Playwright pipeline with merge-reports or a shared run, and report one suite into several Probara projects. Outside GitHub Actions, each `playwright test` creates its own run by default, so four shards make four runs. In GitHub Actions the shards of one workflow run share one run automatically ([parallel jobs](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job)). Elsewhere, to get **one** run for the whole pipeline, pick one of two patterns. Every job that reports needs `PROBARA_API_TOKEN`, set up as in your [CI provider's section](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/). | Pattern | Choose it when | | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | [1. Merge the blob reports](#pattern-1-merge-the-blob-reports) | Simplest, and Playwright's own way. One job reports; no open run if a shard crashes. | | [2. Create the run, report from each shard, close it](#pattern-2-share-one-run) | Results should appear while the shards run. | Parallel workers of one `playwright test` need nothing: they all report through the one reporter of the main process, into one run. ## Pattern 1: merge the blob reports Each shard writes a Playwright [blob report](https://playwright.dev/docs/test-sharding#merging-reports-from-multiple-shards) and keeps it as an artifact; a last job merges them with the reporter. The merged run holds every attempt of every shard, with the same keys as one run of the whole suite, and the files the blob reports carry (screenshots, traces, `testInfo.attach()` files). 1. Register `blob` on CI and the reporter elsewhere: ```ts // playwright.config.ts import { defineConfig } from '@playwright/test'; export default defineConfig({ reporter: process.env.CI ? 'blob' : [['list'], ['@probara/playwright-reporter']], }); ``` 2. Run each shard: `npx playwright test --shard=1/4` writes `blob-report/`. The shards need no token. 3. Collect every `blob-report/` into one folder, and merge it with the token in the environment: ```bash npx playwright merge-reports --reporter @probara/playwright-reporter ./all-blob-reports ``` To give the reporter options while merging, put them in a config for the merge (`npx playwright merge-reports --config merge.config.ts ./all-blob-reports`): ```ts // merge.config.ts export default { testDir: './tests', reporter: [['@probara/playwright-reporter', { projectId: 'SHOP', run: { tags: ['sharded'] } }]], }; ``` Keep `testDir` the same as in `playwright.config`: the keys are relative to it. Complete workflows are in the [GitHub Actions](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/#sharded-github-actions-jobs), [GitLab CI](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/#sharded-gitlab-ci-jobs) and [Bitbucket Pipelines](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/#bitbucket-pipelines) sections. ## Pattern 2: share one run Create the run, report from each shard, close it: 1. A first job runs `probara run create`, which creates an empty automated run and prints its ULID on stdout. 2. Every shard runs with that ULID in `PROBARA_RUN_ULID`. A shard never closes a run it did not create, so they all report into the open run. Probara reconciles shards that report at the same moment: no case is added to the run twice. 3. A last job runs `probara run close`, after every shard, **even when shards failed**. In a shell, the whole flow is: ```bash PROBARA_RUN_ULID=$(npx @probara/cli run create) export PROBARA_RUN_ULID npx playwright test --shard=1/2 npx playwright test --shard=2/2 npx @probara/cli run close ``` Assign the ULID on its own line: `export PROBARA_RUN_ULID=$(...)` would hide a failed creation. In CI, the jobs below pin the CLI (`npx --yes @probara/cli@0.2`), which is not a dependency of the reporter; with `@probara/cli` in your `devDependencies`, `npx @probara/cli` after `npm ci` runs the locked version instead. `probara` is the [Probara CLI](https://docs.probara.net/en/developers/cli/); it takes the same token and project. Options of a new run go to `run create` (`--run-name`, `--environment`, `--milestone`, `--plan`, `--tag`...), since the shards reuse the run and ignore their own run options. ### GitHub Actions ```yaml jobs: probara-run: runs-on: ubuntu-latest outputs: ulid: ${{ steps.create.outputs.ulid }} env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP # A fork's build gets no secret: reporting off, so run create prints nothing and exits 0. PROBARA_ENABLED: ${{ secrets.PROBARA_API_TOKEN != '' }} steps: - uses: actions/setup-node@v5 with: node-version: 22 - id: create run: | ulid=$(npx --yes @probara/cli@0.2 run create) echo "ulid=$ulid" >> "$GITHUB_OUTPUT" test: needs: probara-run runs-on: ubuntu-latest strategy: fail-fast: false matrix: shardIndex: [1, 2, 3, 4] shardTotal: [4] steps: - uses: actions/checkout@v5 - uses: actions/setup-node@v5 with: node-version: 22 - run: npm ci - run: npx playwright install --with-deps - run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }} close-probara-run: needs: [probara-run, test] if: always() && needs.probara-run.outputs.ulid != '' runs-on: ubuntu-latest steps: - uses: actions/setup-node@v5 with: node-version: 22 - run: npx --yes @probara/cli@0.2 run close env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP PROBARA_RUN_ULID: ${{ needs.probara-run.outputs.ulid }} ``` Without a token the CLI exits 2, so the first job sets `PROBARA_ENABLED` from whether the secret is there: in a fork's build, `run create` prints nothing and exits 0, the shards report nothing, and the close job is skipped. ### GitLab CI A dotenv artifact hands the ULID to every later job: ```yaml stages: - test create-probara-run: stage: .pre image: node:22 variables: PROBARA_PROJECT: SHOP rules: - if: $PROBARA_API_TOKEN script: - PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.2 run create) - echo "PROBARA_RUN_ULID=$PROBARA_RUN_ULID" > probara.env artifacts: reports: dotenv: probara.env playwright: stage: test image: mcr.microsoft.com/playwright:v1.56.1-noble parallel: 4 variables: PROBARA_PROJECT: SHOP script: - npm ci - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL close-probara-run: stage: .post image: node:22 variables: PROBARA_PROJECT: SHOP rules: - if: $PROBARA_API_TOKEN when: always script: - npx --yes @probara/cli@0.2 run close ``` `.pre` and `.post` are GitLab's built-in first and last stages; `when: always` closes the run even after failed shards. CircleCI, Azure Pipelines and Buildkite use the same pattern: see their [CI provider sections](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/). ### If the close never ran A run stays open until it is closed. Close it by hand, with the ULID from the log of the create job: ```bash npx @probara/cli run close --run-ulid 01J9Z3K4M5N6P7Q8R9S0T1V2W3 ``` ## Which pattern loses less | Something goes wrong | Pattern 1 (merge) | Pattern 2 (shared run) | | ---------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | A shard crashes before it finishes | Its blob report is missing; the rest is reported | Its results so far are in the run | | Probara is unreachable | The merge job fails to report; re-run it | Each shard's [results file](https://docs.probara.net/en/developers/ci-integrations/playwright/results-and-retries/#results-file) keeps it (a sibling file on a shared disk) | | The close job does not run | Nothing to close | The run stays open: close it by hand | ## Several Probara projects One Playwright suite can hold the tests of several Probara projects: a shared end-to-end suite whose cases live in `SHOP`, `WEB` and `API`. List the other projects in `projects` (`PROBARA_PROJECTS=WEB,API`), and each result goes to the project of its case, into a run of that project. ```ts reporter: [['@probara/playwright-reporter', { projectId: 'SHOP', projects: ['WEB'] }]], ``` ```ts // tests/shared.spec.ts import { test } from '@playwright/test'; test('SHOP-12 pays', async () => {}); test('WEB-3 shows the landing page', async () => {}); test( 'answers the health check', { annotation: { type: 'probara_case', description: 'API-7' } }, async () => {}, ); test('opens the help', async () => {}); ``` ```text $ npx playwright test [probara] Did not send the results linked to cases of API: API is neither the project (SHOP) nor one of projects (PROBARA_PROJECTS) (first seen in "answers the health check"; repeats are logged at debug) [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked); 1 linked only to cases of unlisted projects [probara] Recorded 2 results (1 new case, 0 unmatched) in R-1 of SHOP (closed): https://app.probara.net/projects/SHOP/runs/R-1 [probara] Recorded 1 result (0 new cases, 0 unmatched) in R-2 of WEB (closed): https://app.probara.net/projects/WEB/runs/R-2 ``` ### What goes where - **By the case's project.** A result linked to `WEB-3` goes into a run of `WEB`. The ids of every listed project link from titles too, and leave the key, like the configured project's. - **A test linked to cases of several projects** is sent once per case, each to its project, with its files uploaded to each result. - **A test without a case** goes to the configured project (`projectId`), where it is matched by its key or created: automation keys belong to one project, so only the configured project creates cases, under `suiteUlid`. - **A case of a project that is not listed** is not sent: it would land in the wrong project, where Probara refuses its id. One warning per project says so, and the count in the log leaves those attempts out. List the project to report it. ### One run per project Each listed project gets its own run, created with its first result (a project without results gets no run), with the same name, description, environment (by name), tags and CI source as the configured project's, and closed at the end like any run the reporter creates. The milestone, plan and configurations are planning entities of one project: a name that exists in `SHOP` may not exist in `WEB`, where Probara would refuse the whole report. So `run.milestone`, `run.plan` and `run.configurations`, like the ULID forms (`run.milestoneId`, `run.configurationUlids`, `run.environmentId`), only go with the configured project's run, and one warning names them. To give another project's run its own, create it first and reuse it: ```bash WEB_RUN=$(npx @probara/cli run create --project WEB --milestone "Web sprint 4") PROBARA_RUN_ULIDS="WEB=$WEB_RUN" PROBARA_PROJECTS=WEB npx playwright test ``` `run.ulids` (`PROBARA_RUN_ULIDS=SHOP=,WEB=`) names the run to reuse in each project; the configured project's entry counts as `run.ulid`. A reused run stays open unless `closeRun` is `true`: close it with `probara run close --project WEB --run-ulid `. ### Sharded jobs with several projects Create one run per project before the shards, pass them all in `PROBARA_RUN_ULIDS`, and close them after the last shard: ```bash SHOP_RUN=$(npx @probara/cli run create) WEB_RUN=$(npx @probara/cli run create --project WEB) export PROBARA_PROJECTS=WEB PROBARA_RUN_ULIDS="SHOP=$SHOP_RUN,WEB=$WEB_RUN" npx playwright test --shard=1/2 npx @probara/cli run close --run-ulid "$SHOP_RUN" npx @probara/cli run close --project WEB --run-ulid "$WEB_RUN" ``` When the configured project's run is reused but a listed project has no run in `run.ulids`, each shard creates its own run in that project, and a warning says so. ### Failures A failed report stops only its project: the others go on. The log names the project of every error (`WEB: ...`), and a [results file](https://docs.probara.net/en/developers/ci-integrations/playwright/results-and-retries/#results-file) keeps what each project could not send, with the run it goes back into: one file for the whole run, which `probara import results 'probara-results*.json'` sends to each project. Next: [CI providers](https://docs.probara.net/en/developers/ci-integrations/playwright/ci-providers/), or the [`probara run create` and `run close` reference](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/commands.md#probara-run-create).