# Playwright reporter on CI providers

> Complete pipelines for @probara/playwright-reporter on GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines and Buildkite, with fork-safe secrets.

The reporter runs inside `npx playwright test`: give that step the token, and every test is
reported as it runs. No separate report step is needed. On every provider:

- Store an app token from the **Playwright** card in **Workspace › Integrations**
  ([get started](https://docs.probara.net/en/developers/ci-integrations/playwright/#1-create-an-app-token)) as a secret named
  `PROBARA_API_TOKEN`. The Project ID is not a secret: put it in `playwright.config` (`projectId`)
  or in the pipeline (`PROBARA_PROJECT`).
- Use Node.js 22.12 or later, like the rest of your Playwright run.
- Failed tests fail the step; reporting problems do not. Playwright's exit code stays the tests'
  own.
- Pin the reporter's version in `package.json` (or the lockfile), so a new release never changes a
  pipeline by surprise. The CLI is not a dependency of the reporter: the steps below that run it pin
  it in the command (`npx --yes @probara/cli@0.2`). With `@probara/cli` in your `devDependencies`,
  `npx @probara/cli` after `npm ci` runs the locked version instead.
- Keep the secret away from code you do not trust. A build of a pull request from a fork runs that
  fork's code: each section below says how the provider keeps the token out of it. Without the
  token, the reporter sends nothing (with the project set, it logs that the token is missing), and
  the tests run as usual. The CLI exits 2 without a token, so the recipes turn reporting off
  (`PROBARA_ENABLED=false`) where a fork's build would run it: it then exits 0.

For one run across shards, see [sharding](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/).

## GitHub Actions

### Add the secret

In the repository (or organization): **Settings → Secrets and variables → Actions → New
repository secret**, named `PROBARA_API_TOKEN`.

### The workflow

```yaml
name: Playwright

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
```

### Sharded GitHub Actions jobs

Shards write Playwright blob reports, and one job merges them with the reporter: the whole
pipeline becomes one run, attempts and attachments included. Register `blob` on CI and the
reporter elsewhere:

```ts
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: process.env.CI ? 'blob' : [['list'], ['@probara/playwright-reporter']],
});
```

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report
          retention-days: 1

  report:
    needs: test
    if: ${{ !cancelled() }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter @probara/playwright-reporter ./all-blob-reports
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
```

### Parallel jobs: one run, closed by the final job

In GitHub Actions the jobs of one workflow run report into one run: the run's source names the
workflow run by its repository, run id and attempt, and the first job to report creates the run. A
job closes its run at its end by default, so the parallel jobs set `PROBARA_CLOSE_RUN: false`. A
final job closes the run with `probara run close`, which needs no run ULID there: `needs` makes it
wait for every test job, and `if: always()` runs it when they fail.

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shard }}/2
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
          # A shard leaves the run open for the others.
          PROBARA_CLOSE_RUN: false

  probara:
    needs: test
    if: always()
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx --yes @probara/cli@0.6 run close
        env:
          PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
          PROBARA_PROJECT: SHOP
```

- **A job after the close** is refused with `409 ci_run_closed`: the reporter logs why, sends nothing
  more, and the tests keep their own exit code
  ([troubleshooting](https://docs.probara.net/en/developers/ci-integrations/playwright/troubleshooting/#a-job-reports-after-its-run-was-closed)).
  Keep every reporting job in `needs`.
- **A re-run** of the workflow is another attempt: it reports into a run of its own.
- **One job only** needs neither: its run is closed at the end, as anywhere else.

### What GitHub Actions fills in

| Field         | From                                                                                  |
| ------------- | ------------------------------------------------------------------------------------- |
| Run name      | `GITHUB_WORKFLOW #GITHUB_RUN_NUMBER`, such as `Playwright #42`                        |
| Branch        | `GITHUB_HEAD_REF` (the pull request's branch), else `GITHUB_REF_NAME`; none for a tag |
| Commit        | `GITHUB_SHA`                                                                          |
| Build URL     | `GITHUB_SERVER_URL/GITHUB_REPOSITORY/actions/runs/GITHUB_RUN_ID`                      |
| Repository    | `GITHUB_REPOSITORY`                                                                   |
| Commit author | `GITHUB_ACTOR`                                                                        |
| Pull request  | `<n>` of `GITHUB_REF` `refs/pull/<n>/merge`                                           |
| Workflow      | `GITHUB_WORKFLOW`                                                                     |
| CI run        | `GITHUB_RUN_ID`, attempt `GITHUB_RUN_ATTEMPT`                                         |
| CI job        | `GITHUB_JOB`                                                                          |

On `pull_request` events `GITHUB_SHA` is the merge commit GitHub made, not the branch's last
commit. To record the latter, add `PROBARA_COMMIT: ${{ github.event.pull_request.head.sha }}` to
the step's `env`. In GitHub Actions the commit must be a 7 to 40 character hex SHA: another value is
dropped with a warning.

### Forks on GitHub Actions

Workflows triggered by a pull request from a fork get empty secrets, so fork builds run their tests
and report nothing, with no extra setting. Do not use `pull_request_target` to get the secret into
a fork build: it runs the fork's code with your secrets.

## GitLab CI

### Add the variable

**Settings → CI/CD → Variables → Add variable**: key `PROBARA_API_TOKEN`, the token as the value,
with **Mask variable** on. **Protect variable** limits it to protected branches and tags: turn it on
only if you report from those alone.

### The pipeline

A `.gitlab-ci.yml` in Playwright's image:

```yaml
stages:
  - test

playwright:
  stage: test
  image: mcr.microsoft.com/playwright:v1.56.1-noble
  variables:
    PROBARA_PROJECT: SHOP
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    paths:
      - test-results/
```

- Match the image's Playwright version to the one in your `package.json`.
- A job without the variable (a fork's pipeline, an unprotected branch with a protected variable)
  runs its tests and reports nothing.

### Sharded GitLab CI jobs

`parallel:` shards write blob reports, and one job merges them with the reporter into one run.
Register `blob` on CI and the reporter elsewhere, as in
[sharding](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-1-merge-the-blob-reports):

```yaml
stages:
  - test
  - report

playwright:
  stage: test
  image: mcr.microsoft.com/playwright:v1.56.1-noble
  parallel: 4
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: always
    paths:
      - blob-report/

report-to-probara:
  stage: report
  image: mcr.microsoft.com/playwright:v1.56.1-noble
  needs:
    - job: playwright
      artifacts: true
  variables:
    PROBARA_PROJECT: SHOP
  rules:
    # After failed shards too, and only where the token is set.
    - if: $PROBARA_API_TOKEN
      when: always
  script:
    - npm ci
    - npx playwright merge-reports --reporter @probara/playwright-reporter ./blob-report
```

A `needs:` on a `parallel:` job downloads the artifacts of every instance into the same folder;
each shard's blob report has its own file name (`report-1.zip`...), so they sit side by side.

### What GitLab CI fills in

| Field     | From                                                                                                                  |
| --------- | --------------------------------------------------------------------------------------------------------------------- |
| Run name  | `CI_PROJECT_NAME #CI_PIPELINE_IID`, such as `shop #42`                                                                |
| Branch    | `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` in a merge request pipeline, else `CI_COMMIT_REF_NAME`; none for a tag pipeline |
| Commit    | `CI_COMMIT_SHA`                                                                                                       |
| Build URL | `CI_PIPELINE_URL`                                                                                                     |

### Forks on GitLab CI

A merge request from a fork runs its pipeline in the fork's project, which does not have your CI/CD
variables: the reporter stays off there. If maintainers run fork pipelines in the parent project,
those pipelines do get the variable, so review the change before you run it.

## Jenkins

### Add the credential

**Manage Jenkins → Credentials → (a domain) → Add Credentials**: kind **Secret text**, ID
`probara-api-token`, the token as the secret.

### The pipeline

A declarative `Jenkinsfile` with Node.js 22 on the agent binds the credential into the test stage
with `withCredentials`:

```groovy
pipeline {
  agent any
  environment {
    PROBARA_PROJECT = 'SHOP'
  }
  stages {
    stage('Test') {
      steps {
        sh 'npm ci'
        sh 'npx playwright install --with-deps'
        withCredentials([string(credentialsId: 'probara-api-token', variable: 'PROBARA_API_TOKEN')]) {
          sh 'npx playwright test'
        }
      }
    }
  }
}
```

Jenkins masks the bound secret in the build log. For parallel stages that share one run, create it
in a first stage and pass `PROBARA_RUN_ULID` to each
([shared run](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-2-share-one-run)).

With `JENKINS_URL` set, the run is named `JOB_NAME #BUILD_NUMBER`, and its branch, commit and build
URL come from `BRANCH_NAME` (or `GIT_BRANCH`), `GIT_COMMIT` and `BUILD_URL`.

### Forks on Jenkins

Jenkins does not hide credentials from builds of pull requests by itself: a multibranch job that
builds forks can hand the token to their code.

1. Set **Discover pull requests from forks → Trust** so that a pull request from anyone else builds
   with the `Jenkinsfile` of its target branch: in the GitHub branch source, **From users with Admin
   or Write permission** (or **Nobody**); in the Bitbucket branch source, **Nobody**, or **Forks in
   the same account** from plugin version 871 on (the fix for CVE-2024-28152). **Everyone** builds a
   fork with its own `Jenkinsfile`, which can bind the credential itself, and makes the guard below
   useless.
2. In the target branch's `Jenkinsfile`, bind the credential only when `env.CHANGE_FORK` is not set
   (Jenkins sets it for a pull request from a fork), and turn reporting off for a fork build:

```groovy
pipeline {
  agent any
  environment {
    PROBARA_PROJECT = 'SHOP'
  }
  stages {
    stage('Test') {
      steps {
        sh 'npm ci'
        sh 'npx playwright install --with-deps'
        script {
          if (env.CHANGE_FORK) {
            withEnv(['PROBARA_ENABLED=false']) {
              sh 'npx playwright test'
            }
          } else {
            withCredentials([string(credentialsId: 'probara-api-token', variable: 'PROBARA_API_TOKEN')]) {
              sh 'npx playwright test'
            }
          }
        }
      }
    }
  }
}
```

For parallel stages that share one run, wrap each `withCredentials` the same way: with reporting
off, `run create` writes no ULID and `run close` exits 0 without a token.

## CircleCI

### Add the token

**Organization Settings → Contexts → Create Context** (such as `probara-reporting`), then add the
environment variable `PROBARA_API_TOKEN`. A project environment variable (**Project Settings →
Environment Variables**) works too; then drop `context:` below.

### The pipeline

A `.circleci/config.yml` in Playwright's image, sharded with `parallelism` into one run created up
front ([shared run](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-2-share-one-run));
without `parallelism`, keep only the test steps:

```yaml
version: 2.1

jobs:
  create-run:
    docker:
      - image: mcr.microsoft.com/playwright:v1.56.1-noble
    environment:
      PROBARA_PROJECT: SHOP
    steps:
      - run: |
          # Without the token (a fork's build), turn reporting off: run create prints nothing.
          [ -n "$PROBARA_API_TOKEN" ] || export PROBARA_ENABLED=false
          PROBARA_RUN_ULID=$(npx --yes @probara/cli@0.2 run create)
          echo "PROBARA_RUN_ULID=$PROBARA_RUN_ULID" > probara.env
      - persist_to_workspace:
          root: .
          paths: [probara.env]
  test:
    docker:
      - image: mcr.microsoft.com/playwright:v1.56.1-noble
    parallelism: 4
    environment:
      PROBARA_PROJECT: SHOP
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run: npm ci
      - run: |
          export $(cat probara.env)
          npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
  close-run:
    docker:
      - image: mcr.microsoft.com/playwright:v1.56.1-noble
    environment:
      PROBARA_PROJECT: SHOP
    steps:
      - attach_workspace:
          at: .
      - run:
          when: always
          command: |
            export $(cat probara.env)
            [ -n "$PROBARA_API_TOKEN" ] || export PROBARA_ENABLED=false
            npx --yes @probara/cli@0.2 run close

workflows:
  playwright:
    jobs:
      - create-run:
          context: probara-reporting
      - test:
          context: probara-reporting
          requires: [create-run]
      - close-run:
          context: probara-reporting
          requires:
            - test: [success, failed]
```

- `CIRCLE_NODE_INDEX` counts from 0, Playwright's shards from 1.
- With `CIRCLECI=true`, the run is named `CIRCLE_PROJECT_REPONAME #CIRCLE_BUILD_NUM`, with the
  branch `CIRCLE_BRANCH`, the commit `CIRCLE_SHA1` and the build URL `CIRCLE_BUILD_URL`.

### Forks on CircleCI

CircleCI does not pass contexts or project variables to builds of forked pull requests unless
**Pass secrets to builds from forked pull requests** is on: leave it off. A fork's jobs that use a
context then do not run at all (CircleCI shows an authorization error). With project variables they
run without the token: the reporter sends nothing, and the `PROBARA_ENABLED=false` line of
`create-run` and `close-run` makes `run create` print nothing and both exit 0.

## Azure Pipelines

### Add the variable

**Pipelines → your pipeline → Edit → Variables → New variable**: name `PROBARA_API_TOKEN`, the
token as the value, **Keep this value secret** on. A variable group of the library works too. Secret
variables never reach scripts on their own: map them into each step's `env`, as below.

### The pipeline

An `azure-pipelines.yml`, sharded with `parallel` into one run created by a first job
([shared run](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-2-share-one-run)); without
sharding, keep the `variables` and the test job alone, without `strategy` and `PROBARA_RUN_ULID`:

```yaml
variables:
  # Off in a fork's build, which gets no secrets (see forks below).
  PROBARA_ENABLED: $[ ne(variables['System.PullRequest.IsFork'], 'True') ]

jobs:
  - job: create_run
    pool:
      vmImage: ubuntu-latest
    steps:
      - task: NodeTool@0
        inputs:
          versionSpec: '22.x'
      - script: |
          set -e
          ulid=$(npx --yes @probara/cli@0.2 run create)
          echo "##vso[task.setvariable variable=ulid;isOutput=true]$ulid"
        name: probara
        env:
          PROBARA_API_TOKEN: $(PROBARA_API_TOKEN)
          PROBARA_PROJECT: SHOP

  - job: test
    dependsOn: create_run
    strategy:
      parallel: 4
    variables:
      PROBARA_RUN_ULID: $[ dependencies.create_run.outputs['probara.ulid'] ]
    pool:
      vmImage: ubuntu-latest
    steps:
      - task: NodeTool@0
        inputs:
          versionSpec: '22.x'
      - script: npm ci
      - script: npx playwright install --with-deps
      - script: npx playwright test --shard=$(System.JobPositionInPhase)/$(System.TotalJobsInPhase)
        env:
          PROBARA_API_TOKEN: $(PROBARA_API_TOKEN)
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: $(PROBARA_RUN_ULID)

  - job: close_run
    dependsOn: [create_run, test]
    condition: always()
    variables:
      PROBARA_RUN_ULID: $[ dependencies.create_run.outputs['probara.ulid'] ]
    pool:
      vmImage: ubuntu-latest
    steps:
      - task: NodeTool@0
        inputs:
          versionSpec: '22.x'
      - script: npx --yes @probara/cli@0.2 run close
        env:
          PROBARA_API_TOKEN: $(PROBARA_API_TOKEN)
          PROBARA_PROJECT: SHOP
          PROBARA_RUN_ULID: $(PROBARA_RUN_ULID)
```

- `set -e` makes a failed `run create` fail the first job: Azure checks only the last command of a
  script, and the shards would otherwise each create a run of their own.
- With `TF_BUILD=True`, the run is named `BUILD_DEFINITIONNAME #BUILD_BUILDNUMBER`, and its branch,
  commit and build URL come from the pipeline's variables.

### Forks on Azure Pipelines

Builds of pull requests from forks get no secret variables unless **Make secrets available to
builds of forks** is on: leave it off. Azure leaves a macro it cannot resolve as it is, so in a
fork's build `$(PROBARA_API_TOKEN)` reaches the step as that literal text, a token Probara refuses
with 401: `run create` would fail the first job, and the reporter would log the 401 on every run
(the tests' exit code stays their own). The pipeline's `PROBARA_ENABLED` variable is `False` when
`System.PullRequest.IsFork` is `True`, so a fork's build runs its tests with reporting off, and
`run create` and `run close` exit 0.

## Bitbucket Pipelines

### Add the variable

**Repository settings → Pipelines → Repository variables**: name `PROBARA_API_TOKEN`, the token as
the value, **Secured** on. Bitbucket passes repository variables to every step, so the reporter
finds it in `npx playwright test`.

### The pipeline

A `bitbucket-pipelines.yml` in Playwright's image, with two parallel shards that merge their blob
reports into one run ([merge the blob reports](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-1-merge-the-blob-reports));
without sharding, keep one step running `npx playwright test`:

```yaml
image: mcr.microsoft.com/playwright:v1.56.1-noble

pipelines:
  default:
    - parallel:
        - step:
            name: Shard 1
            script:
              - npm ci
              - npx playwright test --shard=1/2 || touch shard-1.failed
            artifacts:
              - blob-report/**
              - shard-*.failed
        - step:
            name: Shard 2
            script:
              - npm ci
              - npx playwright test --shard=2/2 || touch shard-2.failed
            artifacts:
              - blob-report/**
              - shard-*.failed
    - step:
        name: Report to Probara
        script:
          - npm ci
          - export PROBARA_PROJECT=SHOP
          - npx playwright merge-reports --reporter @probara/playwright-reporter ./blob-report
          - if ls shard-*.failed >/dev/null 2>&1; then echo "Tests failed in a shard" >&2; exit 1; fi
```

Bitbucket runs no step after a failed one, so a shard that failed its step would keep the report
step from sending anything. Each shard records failed tests in a `shard-<n>.failed` artifact
instead, and the report step sends every result, then fails the pipeline when a shard left one: the
shards show as passed, and the report step carries the failure.

With `BITBUCKET_BUILD_NUMBER` set, the run is named `BITBUCKET_REPO_SLUG #BITBUCKET_BUILD_NUMBER`,
with the branch `BITBUCKET_BRANCH`, the commit `BITBUCKET_COMMIT` and a link to the pipeline.

### Forks on Bitbucket Pipelines

Pull requests from forks get no secured variables: the reporter sends nothing there.

## Buildkite

### Add the token

Expose `PROBARA_API_TOKEN` to the steps from a secret store: an agent `environment` hook that reads
it from your secrets manager, or [Buildkite secrets](https://buildkite.com/docs/pipelines/security/secrets/buildkite-secrets).

### The pipeline

A `pipeline.yml` whose parallel jobs report into one run created by a first step
([shared run](https://docs.probara.net/en/developers/ci-integrations/playwright/sharding/#pattern-2-share-one-run)):

```yaml
env:
  PROBARA_PROJECT: SHOP

steps:
  - label: Create the Probara run
    key: create-run
    command: |
      PROBARA_RUN_ULID="$(npx --yes @probara/cli@0.2 run create)"
      if [ -n "$$PROBARA_RUN_ULID" ]; then buildkite-agent meta-data set probara-run "$$PROBARA_RUN_ULID"; fi

  - label: Playwright
    depends_on: create-run
    parallelism: 4
    command: |
      npm ci
      export PROBARA_RUN_ULID="$(buildkite-agent meta-data get probara-run --default '')"
      npx playwright test --shard=$((BUILDKITE_PARALLEL_JOB + 1))/$$BUILDKITE_PARALLEL_JOB_COUNT

  - wait: ~
    continue_on_failure: true

  - label: Close the Probara run
    command: PROBARA_RUN_ULID="$(buildkite-agent meta-data get probara-run --default '')" npx --yes @probara/cli@0.2 run close
```

- `buildkite-agent pipeline upload` fills in `$NAME` and `${NAME}` when it uploads the pipeline, and
  a variable it does not know then, such as `BUILDKITE_PARALLEL_JOB_COUNT` that only the parallel
  jobs get, becomes blank. A `$$` is left as one `$` for the job's shell, while `$(...)` and
  `$((...))` are left as they are.
- The run is created on a line of its own, so a failed `run create` fails the first step rather
  than storing a blank ULID. With reporting off (a fork build on an agent that sets `PROBARA_ENABLED=false`, as below), `run create` prints no ULID and
  exits 0: nothing is stored, `meta-data get --default ''` gives the other steps an empty ULID
  instead of an error, and they run with reporting off too.
- `BUILDKITE_PARALLEL_JOB` counts from 0, Playwright's shards from 1.
- With `BUILDKITE=true`, the run is named `BUILDKITE_PIPELINE_SLUG #BUILDKITE_BUILD_NUMBER`, with
  the branch `BUILDKITE_BRANCH`, the commit `BUILDKITE_COMMIT` and the build URL
  `BUILDKITE_BUILD_URL`.

### Forks on Buildkite

Buildkite builds pull requests from forks only if the pipeline allows it (off by default). Those
builds run on your agents, and `pipeline upload` reads the fork's own `pipeline.yml`, which also
picks the agents' queue. The fork's code runs in the job's shell as the agent's user, so on an agent
that can read the token from your secret store, the fork's code can read it too: no hook or step
keeps it out.

Keep fork builds off, or keep them off those agents: their `environment` hook refuses a build of a
fork (`BUILDKITE_PULL_REQUEST_REPO` names it), and forks build only on a separate queue or cluster
whose agents cannot read the secret store:

```bash
# The environment hook of the agents that can read the token: no build of a fork runs there.
if [ -n "$BUILDKITE_PULL_REQUEST_REPO" ]; then
  case "${BUILDKITE_PULL_REQUEST_REPO#*://}" in
    github.com/acme/shop.git) ;;
    *)
      echo "Refusing a build of a fork: $BUILDKITE_PULL_REQUEST_REPO" >&2
      exit 1
      ;;
  esac
fi
export PROBARA_API_TOKEN="$(your-secrets-manager get probara-api-token)"
```

`BUILDKITE_PULL_REQUEST_REPO` is empty outside pull requests, and has been an `https://` URL since
2022 (a `git://` one before), so the hook compares it without the scheme: put your repository in
place of `github.com/acme/shop.git`. When it does not match, your own pull requests fail on these
agents too, which the first one shows. On the agents that build forks, set `PROBARA_ENABLED=false`
in their `environment` hook: the fork's tests run, `npx playwright test` reports nothing, and
`run create` and `run close` exit 0 without a token.

Next: [troubleshooting](https://docs.probara.net/en/developers/ci-integrations/playwright/troubleshooting/), or keep what a
failed report could not send with a
[results file](https://docs.probara.net/en/developers/ci-integrations/playwright/results-and-retries/#in-ci).