Troubleshooting

Every message the reporter writes starts with [probara] and goes to stderr. Read them first: they say what was not sent, and why.

The formatter does not load

Failed to import formatter @probara/cucumber-reporter means cucumber-js cannot find the package from its own install. Install @probara/cucumber-reporter in the same project as @cucumber/cucumber, so both are in the same node_modules. The formatter needs @cucumber/cucumber 13.3 or later (13.3 is tested).

Reporting is off

Probara reporting is off names the cause: a missing token or project, or an option that is not valid. Check PROBARA_API_TOKEN and PROBARA_PROJECT in the job’s environment (the token is a secret: set it in the CI settings), or formatOptions.probara in cucumber.js. A project code is a capital letter, then capital letters or digits.

Results do not land on the synced cases

A result creates a case when its key matches no case. Check the three parts of the key:

• The path is relative to rootDir, which defaults to the directory cucumber-js runs in. Run the reporter and probara sync gherkin from the same directory, or give both the same rootDir.
• The Feature and scenario names are the names as written, and a Rule: is not in the key.
• The sync ran after the last rename. Sync on the default branch before the tests.

No output on stdout

--format @probara/cucumber-reporter prints nothing on stdout by design: the [probara] lines are on stderr. To keep cucumber’s progress output, give the formatter a path, as configuration shows.

A scenario has no steps, or a step is missing

A scenario whose name is empty, or whose feature file has no Feature, is not sent, and the warning names its file. A step written in another language has no keyword: its action is sent without one, and Probara reads English keywords only in the sync.

A warning says a tag is ignored

ignored the @probara tag "@probara:shop-3" means the tag is not a display ID such as SHOP-12. Use the case’s display ID, with capital letters. Other tools’ tags link nothing.

A warning says an option is ignored

Ignored the unknown option "projectID" of @probara/cucumber-reporter means formatOptions.probara has a name the reporter does not take. The option is left out, and the others still apply. Names are case-sensitive: the option is projectId, not projectID.

Attachments are missing

The summary counts uploads, skips and failures. A file is skipped when it is empty, too large, an image Probara refuses, or beyond the 20 files of a result. Set debug: true to log each request.

Probara cannot be reached

A request that still fails after its retries is logged and counted in the summary, and the run goes on. Check baseUrl, which must be the address of your Probara, and any proxy between the job and it. To keep the results that could not be sent, set a results file.

A dry run sends skipped results

cucumber-js --dry-run runs the formatter without running the steps. cucumber-js does not tell a formatter about the dry run, so the reporter sends it like any run: every scenario is sent as skipped, except one with a step that matches two definitions, which fails. A scenario that matches no case creates one, unless createMissingCases is off. To keep a dry run out of Probara, turn reporting off for it: PROBARA_ENABLED=false npx cucumber-js --dry-run. Nothing is sent and nothing is logged. With a results file set, the results are still written there.

A job reports after its run was closed

In GitHub Actions the jobs of one workflow run share a run, and a final job closes it. A job that reports after that close is refused with 409 ci_run_closed (the run was already closed): the formatter logs why, sends nothing more, and the tests keep their exit code. A parallel job that lacks PROBARA_CLOSE_RUN: false closes the run at its own end, so the jobs after it are refused the same way. Keep every reporting job in the needs of the final job, and set PROBARA_CLOSE_RUN: false on each parallel job (parallel jobs).

Debugging

Set debug: true (or PROBARA_DEBUG=true) to log every request, without the token. Reporting never changes the exit code of the run: a failing run exits with its own code, and a passing run exits 0 even when Probara cannot be reached.