CI providers

Every pipeline below does the same two things. A sync job runs on the default branch and sends the scenarios of your .feature files, so Git holds the cases before any pull request runs its tests. A test job installs the dependencies and runs cucumber-js with the reporter.

Store the token as a secret named PROBARA_API_TOKEN, and set PROBARA_PROJECT to your project code. For a run that reports to another project, set projectId in the formatOptions of cucumber.js, or pass --format-options. Configuration lists every option.

The test job prints nothing on stdout, because the reporter’s lines go to stderr. To keep cucumber-js’s progress output, give the reporter a path as well, as Where the output goes shows.

GitHub Actions

name: BDD

on:
  push:
    branches: [main]
  pull_request:

jobs:
  sync-gherkin:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx @probara/cli sync gherkin "features/**/*.feature"

  test:
    runs-on: ubuntu-latest
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx cucumber-js --format @probara/cucumber-reporter

The sync-gherkin job runs on the default branch only, so Git holds the cases before the pull requests run their tests.

Parallel jobs: one run, closed by the final job

In GitHub Actions the jobs of one workflow run report into one run: the run’s source names the workflow run by its repository, run id and attempt, and the first job to report creates the run. A job closes its run at its end by default, so the parallel jobs set PROBARA_CLOSE_RUN: false. A final job closes the run with probara run close, which needs no run ULID there: needs makes it wait for every test job, and if: always() runs it when they fail.

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        area: [cart, checkout]
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npm ci
      - run: npx cucumber-js --format @probara/cucumber-reporter "features/${{ matrix.area }}/**/*.feature"
        env:
          # A job leaves the run open for the others.
          PROBARA_CLOSE_RUN: false

  probara:
    needs: test
    if: always()
    runs-on: ubuntu-latest
    env:
      PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
      PROBARA_PROJECT: SHOP
    steps:
      - uses: actions/setup-node@v5
        with:
          node-version: 22
      - run: npx --yes @probara/cli@0.6 run close

• A job after the close is refused with 409 ci_run_closed: the formatter logs why, sends nothing more, and the tests keep their own exit code (troubleshooting). Keep every reporting job in needs.
• A re-run of the workflow is another attempt: it reports into a run of its own.
• One job only needs neither: its run is closed at the end, as anywhere else.

GitLab CI

stages:
  - sync
  - test

sync-gherkin:
  stage: sync
  image: node:22
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  variables:
    PROBARA_PROJECT: SHOP
  script:
    - npx @probara/cli sync gherkin "features/**/*.feature"

test:
  stage: test
  image: node:22
  variables:
    PROBARA_PROJECT: SHOP
  script:
    - npm ci
    - npx cucumber-js --format @probara/cucumber-reporter

The sync-gherkin job runs on the default branch only. The reporter reads the same keys the sync wrote, so the results of every branch land on the cases of the default branch.

Jenkins

pipeline {
  agent { docker { image 'node:22' } }
  environment { PROBARA_PROJECT = 'SHOP' }
  stages {
    stage('Sync Gherkin') {
      when { branch 'main' }
      steps {
        withCredentials([string(credentialsId: 'probara-api-token', variable: 'PROBARA_API_TOKEN')]) {
          sh 'npx @probara/cli sync gherkin "features/**/*.feature"'
        }
      }
    }
    stage('Test') {
      steps {
        withCredentials([string(credentialsId: 'probara-api-token', variable: 'PROBARA_API_TOKEN')]) {
          sh 'npm ci'
          sh 'npx cucumber-js --format @probara/cucumber-reporter'
        }
      }
    }
  }
}

Store the token as a secret text credential named probara-api-token. The Sync Gherkin stage runs on the main branch only.

CircleCI

version: 2.1

jobs:
  sync-gherkin:
    docker:
      - image: cimg/node:22.12
    environment:
      PROBARA_PROJECT: SHOP
    steps:
      - checkout
      - run: npx @probara/cli sync gherkin "features/**/*.feature"

  test:
    docker:
      - image: cimg/node:22.12
    environment:
      PROBARA_PROJECT: SHOP
    steps:
      - checkout
      - run: npm ci
      - run: npx cucumber-js --format @probara/cucumber-reporter

workflows:
  build:
    jobs:
      - sync-gherkin:
          filters:
            branches:
              only: main
      - test

Store the token in a project environment variable named PROBARA_API_TOKEN. The sync-gherkin job runs on the default branch only.

Azure Pipelines

trigger:
  branches:
    include: [main]

pr:
  branches:
    include: [main]

variables:
  PROBARA_PROJECT: SHOP

stages:
  - stage: Sync
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    jobs:
      - job: SyncGherkin
        pool:
          vmImage: ubuntu-latest
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '22.x'
          - script: npx @probara/cli sync gherkin "features/**/*.feature"
            env:
              PROBARA_API_TOKEN: $(PROBARA_API_TOKEN)

  - stage: Test
    jobs:
      - job: Cucumber
        pool:
          vmImage: ubuntu-latest
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '22.x'
          - script: npm ci
          - script: npx cucumber-js --format @probara/cucumber-reporter
            env:
              PROBARA_API_TOKEN: $(PROBARA_API_TOKEN)

Store the token as a secret pipeline variable named PROBARA_API_TOKEN. The Sync stage runs on the main branch only.

Bitbucket Pipelines

image: node:22

definitions:
  steps:
    - step: &sync-gherkin
        name: Sync Gherkin
        script:
          - npx @probara/cli sync gherkin "features/**/*.feature"

    - step: &cucumber
        name: Cucumber
        script:
          - npm ci
          - npx cucumber-js --format @probara/cucumber-reporter

pipelines:
  default:
    - step: *cucumber
  branches:
    main:
      - step: *sync-gherkin
      - step: *cucumber

Store the token as a secured repository variable named PROBARA_API_TOKEN. Set PROBARA_PROJECT as a repository variable, or in the environment of each step. The main branch syncs before its tests; other branches run the tests only.

Buildkite

steps:
  - label: ':probara: Sync Gherkin'
    branches: 'main'
    command: npx @probara/cli sync gherkin "features/**/*.feature"
    env:
      PROBARA_PROJECT: SHOP

  - label: ':cucumber: Tests'
    command:
      - npm ci
      - npx cucumber-js --format @probara/cucumber-reporter
    env:
      PROBARA_PROJECT: SHOP

Store the token in the pipeline’s secrets as PROBARA_API_TOKEN, for example from your secrets plugin. The sync step runs on the main branch only. A Buildkite command runs each line in order.