# Configuration

> Every option of @probara/cucumber-reporter, its PROBARA_* variable and its default, and where the reporter writes its output.

Every setting is an option of the formatter under the `probara` key of `formatOptions`, and most have
a `PROBARA_*` variable too. Keep the token in the environment, the project in the configuration (or in
the environment), and set the rest where it fits your pipeline.

An option in `formatOptions` wins over its `PROBARA_*` variable, and the variable wins over the default.
The same options work on the command line:

```bash
npx cucumber-js --format-options '{"probara":{"projectId":"SHOP"}}'
```

## Quick path

1. Create an app token from the **Cucumber** card in **Integrations**
   ([get a token](#get-a-token)) and store it as a CI secret named `PROBARA_API_TOKEN`.
2. Set the project code: `projectId` in the formatter options, or `PROBARA_PROJECT`. A project code is
   a capital letter, then capital letters or digits (`SHOP`, `E2E`). Any other value turns reporting
   off, with an error.
3. Everything else has a default. The run is named after the CI build, and its branch, commit and
   build URL come from the CI on their own.

## Options

### Connection and project

| Option      | Variable            | Type     | Default                   | What it does                                                                                                                           |
| ----------- | ------------------- | -------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled`   | `PROBARA_ENABLED`   | boolean  | `true`                    | `false` turns reporting off: nothing is sent, and nothing is logged                                                                    |
| `apiToken`  | `PROBARA_API_TOKEN` | string   | none (required)           | The app token. Keep it in the environment                                                                                              |
| `projectId` | `PROBARA_PROJECT`   | string   | none (required)           | The project code, such as `SHOP`                                                                                                       |
| `baseUrl`   | `PROBARA_BASE_URL`  | string   | `https://app.probara.net` | The Probara to report to ([network](#network))                                                                                         |
| `projects`  | `PROBARA_PROJECTS`  | string[] | none                      | Other projects results may go to (`WEB,API`), by the case of each result ([linking](https://docs.probara.net/en/developers/ci-integrations/cucumber/linking/)) |

### The run

| Option                   | Variable                      | Type                      | Default                                        | What it does                                                                            |
| ------------------------ | ----------------------------- | ------------------------- | ---------------------------------------------- | --------------------------------------------------------------------------------------- |
| `run.ulid`               | `PROBARA_RUN_ULID`            | string                    | none: a new run                                | Report into an existing run. It is not closed by default                                |
| `run.name`               | `PROBARA_RUN_NAME`            | string                    | the CI build name, else `Automated run <date>` | Name of a new run                                                                       |
| `run.description`        | `PROBARA_RUN_DESCRIPTION`     | string                    | none                                           | Description of a new run                                                                |
| `run.environment`        | `PROBARA_ENVIRONMENT`         | string                    | none                                           | Environment of a new run by name, created when none matches                             |
| `run.environmentId`      | `PROBARA_ENVIRONMENT_ID`      | string                    | none                                           | Environment of a new run by ULID                                                        |
| `run.milestone`          | `PROBARA_MILESTONE`           | string                    | none                                           | Milestone of a new run by display id (`M-3`) or name                                    |
| `run.milestoneId`        | `PROBARA_MILESTONE_ID`        | string                    | none                                           | Milestone of a new run by ULID                                                          |
| `run.plan`               | `PROBARA_PLAN`                | string                    | none                                           | Test plan of a new run by display id (`PLAN-2`) or name: the run starts with its cases  |
| `run.configurations`     | `PROBARA_CONFIGURATIONS`      | `{ group, name }[]`       | none                                           | Configurations of a new run by name (`Browser=Chrome,OS=Linux`), each group once        |
| `run.configurationUlids` | `PROBARA_CONFIGURATION_ULIDS` | string[]                  | none                                           | Configurations of a new run by ULID                                                     |
| `run.tags`               | `PROBARA_RUN_TAGS`            | string[]                  | none                                           | Tags of a new run                                                                       |
| `run.ulids`              | `PROBARA_RUN_ULIDS`           | `Record<string, string>`  | none                                           | The run to reuse in each project (`WEB=<ulid>,API=<ulid>`)                              |
| `closeRun`               | `PROBARA_CLOSE_RUN`           | boolean                   | `true` for a new run, `false` for a reused one | Close the run at the end                                                                |
| `closeRuns`              | none                          | `Record<string, boolean>` | none                                           | Close the run of each project (`{ SHOP: true, WEB: false }`) when `closeRun` is not set |

### Source of the run

| Option            | Variable            | Type                                    | Default              | What it does                                                                |
| ----------------- | ------------------- | --------------------------------------- | -------------------- | --------------------------------------------------------------------------- |
| `source`          | none                | `false \| object (GitHub fields below)` | detected from the CI | `false` sends no source. An object overrides the detected fields one by one |
| `source.branch`   | `PROBARA_BRANCH`    | string                                  | detected from the CI | Branch of the run source                                                    |
| `source.commit`   | `PROBARA_COMMIT`    | string                                  | detected from the CI | Commit of the run source                                                    |
| `source.buildUrl` | `PROBARA_BUILD_URL` | string                                  | detected from the CI | CI build URL of the run source                                              |

In GitHub Actions the source also gets these fields, detected from `GITHUB_*` variables:

| Option                     | Variable | Type     | Default                      | What it does                                                  |
| -------------------------- | -------- | -------- | ---------------------------- | ------------------------------------------------------------- |
| `source.provider`          | none     | `string` | detected from GitHub Actions | `github_actions` (detected) or `other`                        |
| `source.repository`        | none     | `string` | detected from GitHub Actions | Repository of the run source, `owner/name`                    |
| `source.commitAuthor`      | none     | `string` | detected from GitHub Actions | Who triggered the build                                       |
| `source.pullRequestNumber` | none     | `number` | detected from GitHub Actions | Pull request of the build                                     |
| `source.workflow`          | none     | `string` | detected from GitHub Actions | CI workflow of the run source                                 |
| `source.ciRunId`           | none     | `string` | detected from GitHub Actions | CI run id: the jobs of one GitHub Actions run share one run   |
| `source.ciRunAttempt`      | none     | `number` | detected from GitHub Actions | Attempt of the CI run: a re-run reports into a run of its own |
| `source.ciJob`             | none     | `string` | detected from GitHub Actions | CI job of the run source                                      |

### Cases

| Option               | Variable                       | Type    | Default          | What it does                                                                                                                                 |
| -------------------- | ------------------------------ | ------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `createMissingCases` | `PROBARA_CREATE_MISSING_CASES` | boolean | `true`           | Create a case for a scenario that matches none ([linking](https://docs.probara.net/en/developers/ci-integrations/cucumber/linking/#scenarios-without-a-synced-case)) |
| `suiteUlid`          | `PROBARA_SUITE_ULID`           | string  | the project root | Suite that created cases go under                                                                                                            |

### Results, attachments and files

| Option              | Variable                     | Type                                   | Default                           | What it does                                                                                                                                     |
| ------------------- | ---------------------------- | -------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `statusMapping`     | `PROBARA_STATUS_MAPPING`     | `{ passed, failed, skipped, blocked }` | none                              | Send a status as another (`failed=blocked`) ([statuses](https://docs.probara.net/en/developers/ci-integrations/cucumber/results-and-retries/#status-mapping-and-filter)) |
| `statusFilter`      | `PROBARA_STATUS_FILTER`      | string[]                               | none                              | Send no result with these statuses, after the mapping (`skipped`)                                                                                |
| `assignFailedTo`    | `PROBARA_ASSIGN_FAILED_TO`   | string[]                               | none                              | Assign each failed result without an assignee to one of these members, in turn (`ana@example.com,bo@example.com`)                                |
| `uploadAttachments` | `PROBARA_UPLOAD_ATTACHMENTS` | boolean                                | `true`                            | `false` uploads no file ([steps and attachments](https://docs.probara.net/en/developers/ci-integrations/cucumber/steps-and-attachments/#attachments))                    |
| `resultsFile`       | `PROBARA_RESULTS_FILE`       | string                                 | none                              | JSON file for what could not be sent ([results file](https://docs.probara.net/en/developers/ci-integrations/cucumber/results-and-retries/#results-file))                 |
| `rootDir`           | none                         | string                                 | the directory cucumber-js runs in | Directory the feature paths of automation keys are relative to ([linking](https://docs.probara.net/en/developers/ci-integrations/cucumber/linking/#the-key))             |
| `debug`             | `PROBARA_DEBUG`              | boolean                                | `false`                           | Log every request, without the token ([troubleshooting](https://docs.probara.net/en/developers/ci-integrations/cucumber/troubleshooting/#debugging))                     |

### Network

| Option                  | Variable | Type   | Default | What it does                                              |
| ----------------------- | -------- | ------ | ------- | --------------------------------------------------------- |
| `chunkSize`             | none     | number | `500`   | Results per report request, 1 to 500                      |
| `timeoutMs`             | none     | number | `30000` | Timeout of one HTTP attempt, in milliseconds, 1 to 600000 |
| `maxRetries`            | none     | number | `4`     | Retries of a failed request, 0 to 10                      |
| `attachmentConcurrency` | none     | number | `2`     | Results whose attachments upload at the same time, 1 to 8 |

The results go out at the end of the run, in reports of up to `chunkSize` results: a run with more
results sends each full report during the run, and the rest when it ends.

A failed request is retried with a backoff. A request that still fails is logged and counted in the
summary, and the run goes on. Set `maxRetries` to `0` to fail fast when Probara is unreachable: the
results are then kept in the [results file](https://docs.probara.net/en/developers/ci-integrations/cucumber/results-and-retries/#results-file),
when one is set. A proxy or a self-hosted Probara works the same way: set `baseUrl` to its address.

The formatter also takes the seams of `@probara/core` for code and tests: `logger`, `env`, `fetch`,
`sleep`, `random` and `now`. They have no variable, and a `cucumber.js` rarely needs them.

## Where the output goes

cucumber-js gives stdout to one formatter only: the last `--format` without a path. With no `--format`
at all, that is its own `progress` formatter. So `--format @probara/cucumber-reporter` on its own prints
nothing on stdout, and the `[probara]` lines go to stderr.

To keep the progress output, give the formatter a path:

```bash
npx cucumber-js --format progress --format @probara/cucumber-reporter:reports/probara.txt
```

The reporter writes nothing to that file (its logs go to stderr), so the file stays empty. Two
formatters without a path do not both print: cucumber-js keeps the last one and drops the other, without
a message.

Only the `probara` key of `formatOptions` is read by this formatter. An option under `probara` that the
reporter does not know is ignored, with a warning in the log that names it.

## Get a token

Report with an app token. It belongs to your organization, not to a person:

1. As an owner or admin, open **Workspace › Integrations** in Probara and select the **Cucumber** card.
2. Enter a **Token name** after what uses it (the repository or the pipeline) and select **Create token**.
3. Select **Copy token**. It is shown once: store it as the CI secret `PROBARA_API_TOKEN` right away.

An app token uses no seat, keeps working when the person who created it leaves, and shows the app and
its name (such as **Cucumber · e2e-ci**) on its runs and results. It can only report and sync: create
automated runs, send reports (which can create cases and suites), sync feature files, add attachments to
results and close runs. Revoke it from the same card when it may have leaked.

## Runs

The formatter creates one run per cucumber-js run, named after the CI build (or `Automated run <date>`
outside CI), and closes it when the run ends. To report into a run that already exists, set `run.ulid`:
it is not closed unless you set `closeRun`.

In GitHub Actions the jobs of one workflow run share one run, and a final job closes it
([parallel jobs](https://docs.probara.net/en/developers/ci-integrations/cucumber/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job)).

A new run can start with its environment, milestone, test plan, configurations and tags. For example,
`PROBARA_PLAN=PLAN-2` starts the run with that plan's cases.
