# Results and retries

> How each attempt, Outline row and step status becomes a Probara result, how retries are sent, and how to map or filter statuses or keep what could not be sent.

## Results

Each attempt of a scenario is one result, in the order the attempts ran:

- A scenario that is not retried sends one result.
- A scenario that fails once and then passes, with `--retry`, arrives as two results of one key: the
  failed attempt, then the passed one.
- Each row of a Scenario Outline is a result of its own, recorded on the run case with its values.

```bash
npx cucumber-js --retry 2 --format @probara/cucumber-reporter
```

Every attempt stays in the case's history. The run case keeps the last outcome.

## Statuses

Every result has one of four statuses: `passed`, `failed`, `skipped` or `blocked`. The reporter sends
`passed`, `failed` and `skipped`. `blocked` comes from [status mapping](#status-mapping-and-filter).

### Step statuses

| Cucumber's status                            | Probara's status | Why                                                                    |
| -------------------------------------------- | ---------------- | ---------------------------------------------------------------------- |
| `passed`                                     | `passed`         | The step ran and passed.                                               |
| `failed`                                     | `failed`         | The step ran and failed. Its message is the step's error.              |
| `ambiguous`                                  | `failed`         | More than one definition matches the step. Its message says so.        |
| `skipped`, `pending`, `undefined`, `unknown` | `skipped`        | The step did not run: it is skipped, has no definition, or is pending. |

Cucumber's report spells these in lowercase (its JSON report) or in uppercase (its Messages). The
reporter reads both.

### Scenario statuses

A scenario, or one row of an outline, is:

1. `failed` when any of its steps failed, or any of its `Before` or `After` hooks failed;
2. otherwise `skipped` when any of its steps did not run;
3. otherwise `passed`.

## Status mapping and filter

`statusMapping` sends a status as another, and `statusFilter` leaves results with those statuses out of
the report, after the mapping. For example, `PROBARA_STATUS_MAPPING=failed=blocked` and
`PROBARA_STATUS_FILTER=skipped` send every failed scenario as blocked, and send no skipped one.

## Durations

A step's duration is its own. A scenario's duration is the sum of its steps and hooks, in milliseconds.
The start of each attempt is sent as its `executedAt`.

## Results file

When Probara cannot take a result, the reporter can keep it in a file to send later. Set `resultsFile`
(`PROBARA_RESULTS_FILE=reports/probara-results.json`): at the end of the run, every result that was not
sent is written there. With reporting off, every result is written there.

The file is never overwritten. When one is already there, the reporter writes to the first free name
beside it (`probara-results-2.json`, and so on), and the summary line says where it went.

To send the file later, run the CLI with the same token and project:

```bash
npx @probara/cli import results 'reports/probara-results*.json'
```

Its results keep their keys and their case links.

## Assigning failed results

`assignFailedTo` (`PROBARA_ASSIGN_FAILED_TO=ana@example.com,bo@example.com`) assigns each failed result
that has no assignee to one of the members it names, in turn.