# CLI

> Report JUnit XML and Cucumber JSON results, and sync Gherkin feature files, from any CI with the Probara CLI.

The Probara CLI, `@probara/cli`, sends the JUnit XML and Cucumber JSON reports your test framework already writes to
Probara from any CI, and it syncs the scenarios of your `.feature` files. Each import creates an automated test run, links every test to its test case
(and creates the cases that do not exist yet), records the results, uploads the files the report
points to, and closes the run.

The CLI is open source, in the [probara-sdk repository](https://github.com/cynch-me/probara-sdk),
which documents every command, option and CI system. This page covers what you set up in Probara.
To compare it with the Playwright, Jest, Cypress, Vitest and Cucumber reporters, see [CI integrations](https://docs.probara.net/en/developers/ci-integrations/).

## Before you start

| You need                                              | Why                                                                                                             |
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| A paid plan (**Pro** or **Enterprise**, or any trial) | Reporting from CI and the feature sync are not part of the Free plan. On Free, Probara answers `403 forbidden`. |
| A Probara **owner** or **admin**                      | Only owners and admins create and revoke app tokens.                                                            |
| The **Project ID** of the project to report into      | Such as `SHOP`. The CLI reads it from `PROBARA_PROJECT`.                                                        |

## 1. Create an app token

The CLI signs in with an [app token](https://docs.probara.net/en/developers/authentication/#app-tokens): a token that
belongs to your organization, not to a person, so it keeps working when people leave.

1. In **Workspace → Integrations**, select the **JUnit XML** card, or the **Cucumber** card for
   Gherkin sync and Cucumber results.
2. Open the **Access tokens** tab and select **Create an app token**.
3. Enter a **Token name** — name it after what uses it, such as the repository or the pipeline —
   and select **Create token**.
4. Copy the token from **Copy your new token**. It starts with `probara_app_`, and this is the only
   time Probara shows it.

## 2. Store it in your CI

Save the token as a CI secret named `PROBARA_API_TOKEN` (in GitHub Actions: **Settings → Secrets
and variables → Actions**). The CLI only reads the token from that variable; it has no `--token`
flag, so the token never shows up in a process list or a log. The Project ID is not a secret.

## 3. Add the step

Add this GitHub Actions step after your tests, replace `<PROJECT_CODE>` with
your Project ID, and point the path at the file your framework writes:

```yaml
- name: Report to Probara
  if: always()
  run: npx @probara/cli import junit reports/junit.xml
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: <PROJECT_CODE>
```

`if: always()` sends the report when tests fail too. For GitLab CI, Jenkins, CircleCI, Azure
Pipelines, Bitbucket Pipelines, Buildkite and sharded jobs, follow the
[CI guides](https://github.com/cynch-me/probara-sdk/tree/main/packages/cli/docs/ci).

## Sync feature files

Git can be the source of truth for your Gherkin cases. `probara sync gherkin` makes the synced cases
of a project match the scenarios of your `.feature` files: a new scenario creates a case, a changed
one updates it, and a removed one archives it. Run it on your default branch, after the checkout:

```yaml
- name: Sync the feature files to Probara
  run: npx @probara/cli sync gherkin "features/**/*.feature"
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: <PROJECT_CODE>
```

Quote the pattern, and run the command from the repository root. The
[sync guide](https://docs.probara.net/en/guides/syncing-feature-files-from-git/) explains how scenarios map to cases, how
moves and bindings work, and what a sync may archive. The
[command reference](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/commands.md#probara-sync-gherkin)
lists every option, such as `--dry-run`, `--file-url` and `--suite-ulid`.

## Import Cucumber results

`probara import cucumber` sends the results of a Cucumber JSON report. Each scenario becomes a result on
the case it synced to, each row of a Scenario Outline is a result of its own, and each step is a result
step with its keyword. Use it for the reports of Cucumber-JVM, playwright-bdd and other tools that write
Cucumber JSON.

For cucumber-js, the [Cucumber reporter](https://docs.probara.net/en/developers/ci-integrations/cucumber/) is the better path: it
sends each scenario's results at the end of the run, with the files its steps attach, and needs no JSON file. Both
read the same keys, so either lands on the same cases.

```bash
npx cucumber-js --format json:results/cucumber.json
npx @probara/cli import cucumber results/cucumber.json
```

The command reads the Cucumber JSON that cucumber-js writes with `--format json`. Probara is tested
against cucumber-js 13 only: Cucumber-JVM and playwright-bdd write the same layout, which Probara has not
tested. Reqnroll, which writes no Cucumber JSON, and Behave are not supported. Attachments in the
report are not sent. Keep the feature files in the checkout: a result finds its case through the
path of its file. The
[Gherkin reference](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/gherkin.md)
explains the keys and the status of each result.

## What the token can do

An app token can only do what the CLI does: create automated runs, send reports (which can create
test cases and suites), add attachments to their results, close runs, sync feature files (which
create, update and archive synced cases), and, for the framework reporters, read a run's case keys.
Every other call answers `403 forbidden` — see the [exact calls](https://docs.probara.net/en/developers/authentication/#allowed-calls).

- Runs, results and attachments it sends show its card, **JUnit XML** or **Cucumber**, and the token
  name, not a person.
- It uses no seat and survives the person who created it. Revoke it from the same card, and create
  a new one, when it may have leaked or when someone who could read the secret leaves your team.
- If your organization moves to the Free plan, Probara keeps the token but refuses it; it works
  again once the organization is back on a paid plan.
- A personal [API token](https://docs.probara.net/en/developers/authentication/) also works on a paid plan, but it acts
  with its user's permissions and stops working when that user leaves. In CI, use an app token.

## When Probara refuses the token

| Answer             | What to check                                                                                                            |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| `401 unauthorized` | The token was revoked, or `PROBARA_API_TOKEN` is missing or holds something else. Create a new token and store it again. |
| `403 forbidden`    | The organization is on the Free plan, or the token was used for something other than the CLI's calls.                    |

The CLI's [troubleshooting guide](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/troubleshooting.md)
covers the rest, including report and network errors.