# Vitest reporter troubleshooting > Problems with @probara/vitest-reporter and their solutions, every line the reporter logs, PROBARA_DEBUG, and how to check what would be sent without sending it. Each entry starts from what you see, then says why and what to do. The reporter logs on stderr in `[probara]` lines; [`PROBARA_DEBUG=true`](#probara_debug) adds every request. Whatever the problem, Vitest's exit code stays the tests' own. The [last section](#every-line-the-reporter-logs) lists every line the reporter logs. ## Nothing is reported, and nothing is logged **Why.** Without a token and a project, the reporter stays off and quiet, on purpose: local runs and fork builds send nothing. It says so only at debug: ```text $ PROBARA_DEBUG=true npx vitest run [probara] Probara reporting is not configured: set PROBARA_API_TOKEN and PROBARA_PROJECT to enable it ``` **Solution.** Give the step that runs `vitest` the `PROBARA_API_TOKEN` secret, and set the project (`projectId` or `PROBARA_PROJECT`). Check that the reporter is in the `reporters` of the root config Vitest uses: `--reporter` on the command line replaces the config's reporters, so `vitest run --reporter=dot` runs without the reporter, and with no `[probara]` line at all. ## Reporting is off: a configuration problem ```text $ PROBARA_RUN_ULID=R-12 npx vitest run [probara] Probara reporting is off: PROBARA_RUN_ULID is not a ULID ``` **Why.** A setting has a wrong value, or only one of the token and the project is set. Each problem is logged, naming the option or variable at fault (never its value), and nothing is sent. The most common: `The API token is not set: pass apiToken or set PROBARA_API_TOKEN`, and `The project is not set: pass projectId or set PROBARA_PROJECT`. **Solution.** Fix the setting the line names ([options](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#options)). ## Probara answers 401 or 403 ```text $ npx vitest run [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked) [probara] 3 results were not sent: Probara answered 401 unauthorized: . No run was created or updated ``` **Why.** `401`: the token is wrong, revoked, or from another Probara (`baseUrl`). `403`: the token cannot report to that project, or the organization is on the Free plan (reporting from CI needs a paid plan). **Solution.** Create a new app token from the **Vitest** card ([get started](https://docs.probara.net/en/developers/ci-integrations/vitest/#1-create-an-app-token)) and update the secret; check the Project ID; check the organization's plan. ## The run was left open The error line ends with `The run R-4 was left open: `, or, when not even the first report went through, `No run was created or updated`. After the retries, the reason reads like `Could not send the report after 5 attempts: the last one timed out after 30000 ms`. **Why.** A report failed after its retries (Probara unreachable, a refused report): the reports after it are not sent, and a run that already has results stays open, so nothing looks complete when it is not. A run of a [watch session](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode) and a reused run (`run.ulid`) stay open on purpose. **Solution.** Set a [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file): what was not sent goes there, with the run it belongs to, and `probara import results` sends it into that run and closes it. Otherwise close the run by hand with `npx @probara/cli run close --run-ulid `, and run the tests again. ## A job reports after its run was closed In GitHub Actions the jobs of one workflow run share a run, and a final job closes it. A job that reports after that close is refused with `409 ci_run_closed`, and the reporter logs that the run was already closed. It sends nothing more, and the tests keep their exit code. **Why.** A reporting job is missing from the final job's `needs`, so the final job closed the run before that job reported. Or a parallel job closed the run at its own end because it lacked `PROBARA_CLOSE_RUN: false`, and every job after it is refused the same way. **Solution.** Keep every reporting job in the `needs` of the final job, and set `PROBARA_CLOSE_RUN: false` on each parallel job. The [parallel jobs](https://docs.probara.net/en/developers/ci-integrations/vitest/ci-providers/#parallel-jobs-one-run-closed-by-the-final-job) section has the complete workflow. ## Results were not recorded (unmatched) ```text $ PROBARA_CREATE_MISSING_CASES=false npx vitest run --no-file-parallelism [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked) [probara] Recorded 1 result (0 new cases, 2 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1 [probara] 2 results were not recorded (case_not_found): tests/cart.test.ts > cart > removes an item; tests/login.test.ts > login > logs in with a valid password ``` **Why.** A result names a case that does not exist (a typo in an id, a case of another project), or matches no case while `createMissingCases` is off. Probara records the others and says which were left out, with the reason. **Solution.** Fix the id, create the case, or let the reporter create missing cases (the default). ## Every run creates new cases **Why.** The automation keys changed: a test, `describe` or file was renamed or moved, `vitest` runs with another root than before (keys are relative to it), `keyIncludesFile` changed ([what changes a key](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#automation-keys)), or a Vitest upgrade writes some `test.each` titles differently ([from Vitest 4 to 5](https://docs.probara.net/en/developers/ci-integrations/vitest/migrating/#from-vitest-4-to-5)). **Solution.** Name the cases in the titles ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)), so a rename keeps them. Set `rootDir` when `vitest` runs from different directories. Before a refactor or an upgrade, [compare the keys](#check-what-would-be-sent). ## A test file is not reported **Why.** Vitest never ran it: `--bail` stopped the run first, or the run was interrupted (`Ctrl+C`). It has no test to report, and one warning counts such files: `Vitest did not run 1 test file of this run: nothing of it is reported`. A file Vitest ran but could not load is reported: one failed result named after the file ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)). **Solution.** Run the tests again without `--bail`, or let the run finish. The tests Vitest left waiting in a file it had started are sent as skipped, with the note `Not run: the test run was interrupted` ([statuses](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#statuses)). ## Only the last attempt of a retried test was sent **Why.** The reporter's setup file did not run in those tests, so their earlier attempts left no record, and one warning says how many: `Vitest retried or repeated 1 test whose earlier attempts were not recorded: only its final attempt is reported`. Either a shard ran with `--reporter=blob` alone on the command line, which replaces the config's reporters, or the reporter could not add its setup file to a project (a warning at the start names the project). **Solution.** Name the reporter beside `blob` on the shards' command line, `--reporter=blob --reporter=@probara/vitest-reporter` ([the reporter goes beside `blob`](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#the-reporter-goes-beside-blob)). ## A helper warns that it only works while a test runs **Why.** A `probara.*` helper was called outside a test: at the top of a file, in a `describe` body, `beforeAll` or `afterAll`. Nothing is recorded, and the test goes on. The warning is in the test file's output (`stderr | tests/...`), once per helper and file. **Solution.** Call it in the test, `beforeEach`, `afterEach` or `onTestFinished` ([where to call the helpers](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#where-to-call-the-helpers)). ## A helper call was dropped | You see | Why, and what to do | | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `probara.() is dropped in a concurrent test: this worker cannot tell which test calls it without the reporter's runner` | A `test.concurrent` test where the reporter's runner does not run: in [browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/), or under a custom `runner` Node cannot import. Make the test sequential, or write the runner in JavaScript | | `the runner of project "" is , which Node does not import: the reporter keeps it as it is` | The project's custom `runner` is a file Node's own `import()` cannot load (`.jsx`, `.tsx`): the reporter leaves it as it is. Attempts are still recorded; the calls of its `test.concurrent` tests are dropped. Write the runner in `.js` or `.mjs` | | `the runner of project "" is TypeScript, which this Node cannot import: the reporter keeps it as it is` | The same, for a `.ts` runner on a Node that cannot run TypeScript itself. Use a newer Node, or write the runner in JavaScript | | `the probara.* helpers cannot see the running test: ` | Vitest's way to name the running test failed in that worker (a Vitest the reporter does not know yet): its helper calls are dropped. Report it, with the Vitest version | | `probara.() could not be recorded: ` | The helper failed inside, and the call is ignored. Report it | | No warning, and nothing recorded | The reporter is not in the run's reporters (`--reporter` on the command line): every helper then does nothing, and `probara.step()` still runs its body | ## Run selection ran every test **Why.** With `runCasesOnly`, the reporter could not tell which tests belong to the run, so every test ran and was reported, rather than none. One warning says why: no `PROBARA_RUN_ULID`, the run's cases could not be read (a wrong ULID, a token of another organization), a browser project, or a shard that writes a blob. **Solution.** Follow the warning ([when it cannot select](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/#when-it-cannot-select)). When every test was skipped instead, no test belongs to the run: check the ULID, and that the tests name the run's cases or have their keys. ## captureOutput attaches nothing **Why.** `captureOutput` is off (the default), or the output is of a kind it leaves out: a `test.concurrent` test (one warning counts them), output written outside a test (the file's own code, a `beforeAll` or `afterAll`), or a retried test whose attempts were not recorded ([console output](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#console-output)). **Solution.** Turn `captureOutput` on, and log with `console` inside the test, a `beforeEach` or an `afterEach`, in a test that is not `test.concurrent`. ## An attachment is missing **Why.** The log names every file it left out, and why: the file did not exist when `probara.attach()` ran (it is copied at once), it is a folder or over 32 MiB, an image over 10 MiB or 8192 px, a refused content type, more than 20 files in one result, or, at a merge, a copy the blob names that did not come along with `attachmentsDir` ([limits](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#limits-and-what-is-skipped)). A file attached inside a step goes to that step, not to the result. **Solution.** Attach a file that exists when you call `probara.attach()`, await the call, keep files under the limits, and carry `attachmentsDir` with the blobs ([where the copies are kept](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#where-the-copies-are-kept)). ## Browser mode | You see | Why, and what to do | | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `probara.attach() did not attach "": Vite does not serve "" (server.fs.allow or server.fs.deny)` | The file is outside the folders Vite serves, or Vite denies it. Attach a file inside the project, or a body ([`probara.attach()` in a browser](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/#probaraattach-in-a-browser)) | | `probara.attach() did not attach "": Vite's file rules are not reachable from this run, so "" is not served` | The reporter could not load the Vite the project's Vitest runs with, so it reads no path. Attach a body, and report it | | `probara.attach() "" of "" settled after the test ended, so it is not attached. Await probara.attach() before the test ends` | The test did not await the call: `await probara.attach(...)` | | `Vite has no isFileLoadingAllowed (it arrived in 5.4): the browser attach follows its isFileServingAllowed` | A notice, once per process: Vite 5.0 to 5.3 have an older file check, which decides. Upgrade Vite to 5.4 or later to get its current rules | | `Vite exports neither isFileLoadingAllowed nor isFileServingAllowed: the browser attach refuses every path` | A notice, once per process: that Vite has no file check, so no path is read. Attach bodies, or upgrade Vite | | `runCasesOnly does not apply in browser mode yet: the tests of project "" run and are reported` | Run selection is not supported in browser mode yet: every test of the project runs and is reported | | `could not add the browser setup file to project "" (): ...` | That project records no attempt and no helper call. Report it, with the Vitest version | | `could not add the browser commands to project "" (): ...`, `could not give the browser commands to project "": ` | `probara.attach()` copies no file in that project. Report it, with the Vitest version | ## A shard sends nothing `Sharded run with the blob reporter: the results are reported by vitest --merge-reports` is expected: a shard that writes a blob sends nothing, so that no result is sent twice. Run `vitest --merge-reports` with the reporter in its config: that run reports every shard ([pattern 1](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#pattern-1-merge-the-blob-reports)). A shard with `--reporter=blob` alone on the command line runs without the reporter, and logs nothing. ## Unhandled errors are not sent **Why.** Errors no test owns (a promise no test awaited, an error thrown from a timer) are not results. Vitest fails the run for them; Probara gets no result, since no test failed, and one warning counts them ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)): `Vitest reported 2 unhandled errors outside the tests: they are not results, and nothing is sent for them`. **Solution.** Fix the code: Vitest's output above the `[probara]` lines shows the error in full. ## Probara answers 429 (too many requests) **Why.** The organization's rate limit (60 requests per minute on Free, 1,200 on Pro, and 3,000 on Enterprise) is shared by every token and pipeline. Attachments cost two requests per result ([rate limit](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#rate-limit)). **Solution.** The reporter waits and retries on its own. For large suites, attach less, lower `attachmentConcurrency`, or spread sharded pipelines in time. ## Debugging ### The log The reporter writes to **stderr** only, through Vitest's own logger, every line prefixed with `[probara]`. Stdout stays Vitest's: a reporter that prints JSON there (`--reporter=json`) never gets a line of the reporter's. A run that reports logs two lines: ```text $ npx vitest run [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked) [probara] Recorded 3 results (2 new cases, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1 ``` | Line | What it says | | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `Sending N results of M tests (...)` | Before anything is sent: one result per attempt and case, by status after `statusMapping`, and what was left out | | `Recorded N results (...) in R-1` | Per run: what Probara recorded, the cases it created, the results it could not match, and the link | | `N results were not recorded (...)` | The unmatched results, by reason, with their cases or keys | | `Attached N files to results (...)` | Uploads: attached, skipped and failed | | `N results were not sent: ...` | A failed report: the reason, and the run left open | | `Wrote ... to ` | What went to the results file | | `Ran only the tests of run : ...` | Run selection: how many tests matched the run's cases, and how many were skipped and left out | | `Watch mode: every re-run reports into R-1 of SHOP, ...` | Watch mode: the one run of the session, which stays open, and the command that closes it | | `Sharded run with the blob reporter: ...` | A shard that writes a blob: it sends nothing, and `vitest --merge-reports` reports its results | | `Probara reporting is off: ...` | A configuration problem: nothing is sent ([reporting is off](#reporting-is-off-a-configuration-problem)) | ### Warnings A warning that would repeat is logged once, the first time; with `PROBARA_DEBUG`, the reporter's log also shows the repeats. Where it comes from decides where it shows and how it ends: | Warning about | Logged | | --------------------------------------------------------------- | ------------------------------------------------------------------------------- | | A call or a file of a test, or the tests of the run | Once per run, ending `(first seen in the run; repeats are logged at debug)` | | One result as it is sent (a skipped attachment, a dropped step) | Once per run, ending `(first seen in ""...; repeats are logged at debug)` | | A `probara.*` call where no test runs, or that no test can own | In the test file's output (`stderr \| tests/...`), once per test file | | The setup of the run (an unknown option, a project) | Once, when Vitest starts | Two tests that attach the same missing file, for example: ```ts // tests/checkout.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('pays by card', async () => { await probara.attach({ name: 'receipt', path: 'receipt.txt' }); }); test('pays by transfer', async () => { await probara.attach({ name: 'receipt', path: 'receipt.txt' }); }); ``` ```text $ npx vitest run [probara] probara.attach() did not attach "receipt": it does not exist (first seen in the run; repeats are logged at debug) [probara] Sending 2 results of 2 tests (2 passed, 0 failed, 0 skipped, 0 blocked) [probara] Recorded 2 results (2 new cases, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1 ``` The token never appears in any line, even when Probara's answer echoes it. ### `PROBARA_DEBUG` `debug: true` (`PROBARA_DEBUG=true`) adds a line for every request and every attempt of it, the repeats of warnings, and says why reporting is off when it is: ```text $ PROBARA_DEBUG=true npx vitest run [probara] Sending 3 results of 3 tests (3 passed, 0 failed, 0 skipped, 0 blocked) [probara] Sending report 4f8d2c1a-9b3e-4c5d-8e7f-0a1b2c3d4e5f (attempt 1 of 5) [probara] Recorded 3 results (2 new cases, 0 unmatched) in R-1 (closed): https://app.probara.net/projects/SHOP/runs/R-1 ``` The id after `report` is the request's `Idempotency-Key` ([idempotency](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#idempotency)). Keep debug off in normal runs: it is verbose, though it never logs the token or the bodies. ### Check what would be sent The reporter has no dry run of its own, but a results file is one: run the tests with reporting off and a results file, then let the CLI print what it would send. Nothing reaches Probara, and no token is needed: ```text $ PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=probara-results.json npx vitest run --no-file-parallelism [probara] Wrote 3 results to /work/shop/probara-results.json: send them with probara import results /work/shop/probara-results.json $ npx @probara/cli import results probara-results.json --dry-run passed SHOP-12 tests/cart.test.ts > cart > adds an item passed - tests/cart.test.ts > cart > removes an item passed - tests/login.test.ts > login > logs in with a valid password Total: 3 results from 1 file (3 passed, 0 failed, 0 skipped, 0 blocked) ``` The first command logs on stderr; the dry run prints on stdout, one line per result, in the order the results were written (`--no-file-parallelism` keeps it the same from run to run): the status, the case (`-` when none is named) and the automation key. `--dry-run` never touches the file; delete it afterwards, or import it for real (a file left there makes the next run write `probara-results-2.json`). `--json` prints the whole document instead, each entry as Probara would receive it ([`probara import results`](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/commands.md#probara-import-results)). Use it to compare keys before a refactor, an upgrade of Vitest or of the reporter, or a switch of `keyIncludesFile`: write one file per run, and diff the two dry runs. Give each run its own file name and start without them: with a file already at the path, a run writes to a sibling (`before-2.json`) and the dry run of `before.json` would show an older run, hiding the difference. ```bash rm -f before.json after.json PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=before.json npx vitest run --no-file-parallelism # Change the code, or upgrade Vitest or the reporter, then: PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=after.json npx vitest run --no-file-parallelism npx @probara/cli import results before.json --dry-run > before.txt npx @probara/cli import results after.json --dry-run > after.txt diff before.txt after.txt ``` Any line that changed is a test whose key or case link changed: it would get a new case. ### What the reporter never does to the run - **It never fails the run.** Whatever goes wrong while reporting is a `[probara]` line, and the exit code of `vitest` is the one the tests give: a failed report or a refused token leaves it as it is. - **It never changes a test's outcome.** The setup file the reporter adds records each attempt without throwing into the test, and a `probara.*` call that cannot be recorded is a warning, never an error of the test. The only tests it skips are those outside the run of [`runCasesOnly`](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/), when you turn it on. - **It never writes to stdout**, so a reporter that prints there keeps its output clean. ## Every line the reporter logs Every `[probara]` line, the reporter's own and those of the reporting library it shares with the other Probara reporters and the CLI. `<...>` stands for what changes. A warning that can repeat ends with `(first seen in the run; repeats are logged at debug)`, or names the result it was first seen in, and is logged at debug afterwards. **Where** is the reporter's log on stderr (log), or the output of the test file, which Vitest prints under `stderr | tests/...` (test output). The token never appears in any line, even when Probara's answer echoes it. ### Setup and configuration | Line | Level | What to do | | ------------------------------------------------------------------------------------------------------------------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `Probara reporting is not configured: set PROBARA_API_TOKEN and PROBARA_PROJECT to enable it` | debug | [Nothing is reported](#nothing-is-reported-and-nothing-is-logged) | | `Probara reporting is disabled by PROBARA_ENABLED` (or `by the enabled option`) | debug | Reporting was turned off on purpose | | `Probara reporting is off: ` | error | [A configuration problem](#reporting-is-off-a-configuration-problem) | | `Probara reporting is off: options must be an object` | error | Give the options as an object: `['@probara/vitest-reporter', { projectId: 'SHOP' }]` | | `Ignored the unknown option "" of @probara/vitest-reporter` | warn | A typo in an option name ([options](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#options)) | | `Ignored : a reused run (run.ulid) keeps its own` | warn | A reused run keeps its name and settings ([an existing run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#an-existing-run)) | | `Truncated the run name to 200 characters`, `Truncated the run description to 2000 characters` | warn | Shorten them | | `Dropped run tags beyond the limit of 50` | warn | Send fewer run tags | | `Ignored source.: it must be a string`, `Ignored a blank source branch`, `Ignored a source ...` | warn | Fix the field of `source` or its variable ([run source](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#run-source)) | | `Ignored the run of in PROBARA_RUN_ULIDS: is not the project nor one of projects` (or `run.ulids`) | warn | List the project in `projects`, or drop its run ([several projects](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#several-probara-projects)) | | `The run of is reused (...), but has no run in run.ulids: ...` | warn | Give every project a run ([sharded jobs](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#sharded-jobs-with-several-projects)) | | `Sent with the run of only: they belong to one project. ...` | warn | Create the other projects' runs with their own ([one run per project](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#one-run-per-project)) | | `could not register project "": ` | warn | That project may run without the reporter's setup file. Report it | | `could not add the setup file to project "" (): its retries and repeats are reported as their final attempt only` | warn | Report it, with the Vitest version | | `could not give project "" its attachments folder (): its files are not attached` | warn | Report it | | `the runner of project "" is , which Node does not import: the reporter keeps it as it is` | warn | [A helper call was dropped](#a-helper-call-was-dropped) | | `the runner of project "" is TypeScript, which this Node cannot import: the reporter keeps it as it is` | warn | [A helper call was dropped](#a-helper-call-was-dropped) | | `could not wrap the runner of project "" (): the reporter keeps it as it is` | warn | Report it | ### Sending and recording | Line | Level | What to do | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `Sending results of tests (...)`, then `; left out by statusFilter`, `; ignored with probara.ignore()` or `; linked only to cases of unlisted projects` | info | What the run hands to Probara ([the log](#the-log)) | | `Recorded results ( new cases, unmatched) in R-1 (closed): `, `in R-1 of ` with several projects | info | Nothing: the run is in Probara | | `Filtered out results by their status (statusFilter): not sent` | info | [Status filter](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#status-mapping-and-filter) | | ` results were not recorded (): ` | warn | [Unmatched results](#results-were-not-recorded-unmatched) | | ` results were not sent: . No run was created or updated`, `... The run R-1 was left open: ` | error | [401 or 403](#probara-answers-401-or-403), [left open](#the-run-was-left-open) | | `Probara warned: ` | warn | Read Probara's warning, such as [assignFailedTo](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#assign-failed-results)'s count | | `Did not send the results linked to cases of : is neither the project () nor one of projects (PROBARA_PROJECTS)` | warn | [What goes where](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#what-goes-where) | | ` attempt of got ; retrying in ms`, `... timed out after ms; ...`, `... failed: ; ...` | warn | Nothing: the request is retried ([retries and backoff](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#retries-and-backoff)) | | `Sending (attempt of )` | debug | [`PROBARA_DEBUG`](#probara_debug) | | `The run R-1 was already closed or aborted` | info | Nothing: someone closed the run before the reporter did | | `Could not close the run R-1: . It was left open: ` | error | Close it by hand: `npx @probara/cli run close --run-ulid ` | | `Wrote the results that were not sent to : send them with probara import results ` | warn | [Results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) | | `Wrote results to : send them with probara import results ` (reporting off) | info | [Reporting off: keep everything](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#reporting-off-keep-everything) | | `Could not write the results file : ` | error | Check that the folder of the file can be written | | `Could not keep a copy of the attachment () next to the results file (): ...` | warn | The file points at the original: import it before that file goes | ### Watch mode and shards | Line | Level | What to do | | ----------------------------------------------------------------------------------------------------------------------------------- | ----- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Watch mode: every re-run reports into R-1 of , which stays open: close it in Probara, or with probara run close ...` | info | Close the run when the session ends ([watch mode](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode)) | | `The run R-1 of was closed: sent the results of this re-run into a new run` (or `deleted`) | info | Nothing: the session goes on in the new run | | `The run R-1 of was closed: sent the results it did not record into a new run; the rest of this re-run is in R-1` | info | Nothing: the re-run is split over both runs | | `The run R-1 of was closed: the results file keeps this re-run, but names R-1, which refuses it: send it with ...` | info | Import the file into an open run ([when the run is closed](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#when-the-run-is-closed-during-the-session)) | | `Sharded run with the blob reporter: the results are reported by vitest --merge-reports` | info | [A shard sends nothing](#a-shard-sends-nothing) | ### Run selection | Line | Level | What to do | | ----------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------------- | | `Ran only the tests of run : of tests match its cases; skipped and not reported` | info | Nothing ([run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/)) | | `No test matches the cases of the run : every test was skipped, and none is reported` | warn | Check the ULID ([run selection ran every test](#run-selection-ran-every-test)) | | `runCasesOnly needs the run whose tests to run: set run.ulid or PROBARA_RUN_ULID. Every test runs and is reported` | warn | Set the ULID | | `runCasesOnly: could not read the cases of the run (). Every test runs and is reported` | warn | Check the ULID and the token | | `runCasesOnly does not apply to a sharded run with the blob reporter: every test runs, and vitest --merge-reports reports them all` | warn | [Setups](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/#setups) | | `runCasesOnly does not apply in browser mode yet: the tests of project "" run and are reported` | warn | [Browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/#not-supported-yet) | | `runCasesOnly needs the setup file in project "": its tests run and are reported` | warn | The setup file could not be added to that project: report it | | `could not give project "" the cases of the run (): its tests all run and are reported` | warn | Report it, with the Vitest version | ### Helpers These are logged once per run, unless **Where** says test output. | Line | Where | What to do | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `probara.() only works while a test runs (in a test, or a beforeEach or afterEach hook)` | test output | [A helper warns](#a-helper-warns-that-it-only-works-while-a-test-runs) | | `probara.() is dropped in a concurrent test: this worker cannot tell which test calls it without the reporter's runner` | test output | [A helper call was dropped](#a-helper-call-was-dropped) | | `the probara.* helpers cannot see the running test: ` | test output | [A helper call was dropped](#a-helper-call-was-dropped) | | `probara.id() takes a case id or a list of case ids, such as PRB-12`, `probara.title() takes a string`, `probara.suite() takes a suite title or a list of suite titles`, `probara.comment() takes a string` | log | A wrong argument: the call is ignored ([metadata](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)) | | `probara.parameters() takes an object of strings, numbers or booleans`, `probara.fields() takes an object of strings, numbers or booleans`, `probara.tags() takes strings` | log | A wrong argument: the call is ignored ([metadata](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)) | | `probara.link() takes an absolute http(s) URL of at most 2048 characters`, `probara.link() takes the name as a string`, `probara.issue() takes an issue id (a string), such as PRB-7` | log | A wrong argument: the call is ignored ([links and issues](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#links-and-issues)) | | `probara.() ignores blank ` | log | A blank tag, id or suite: it is left out | | `probara.() could not be recorded: ` | log | Report it | | `probara.step() takes an action (a string)`, `probara.step() takes the expected result as a string`, `probara.step() takes the data as a string`, `probara.step() takes its options as an object ({ expected, data })`, `probara.step() takes a function as its body` | log | Fix the call ([steps](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probarastep)) | | `Dropped the issues of probara.issue(): no issueUrlTemplate turns their ids into links` | log | Set `issueUrlTemplate` ([`probara.issue()`](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#probaraissue-and-issueurltemplate)) | | `Dropped a link without an absolute http(s) URL of at most 2048 characters` | log | Give `probara.link()` an absolute `http(s)` URL ([link rules](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#link-rules)) | | `Ignored malformed probara metadata (type "")` | log | Something wrote to `task.meta.probara`, which is the reporter's ([how metadata travels](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#how-metadata-travels)) | A warning of a call that belongs to no test is printed in the test output, whatever its text. ### Attachments and output | Line | Level | What to do | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `probara.attach() did not attach "": it does not exist`, `it cannot be read`, `it is not a file`, `it is larger than 32 MiB`, `it cannot be copied ()` | warn | [An attachment is missing](#an-attachment-is-missing) | | `probara.attach() takes { name, path } or { name, body }`, `probara.attach() could not attach "": its contentType is not a string`, `... it has neither a path nor a body` | warn | Fix the call ([`probara.attach()`](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probaraattach)) | | `probara.attach() could not attach "": this run has no attachments folder` | warn | The project got no attachments folder: a warning at the start says why | | `Attached files to results ( skipped, failed)` | info | Nothing; a warning when some failed | | `Skipped an attachment: (first seen in "", ; ...)` | warn | [Limits](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#limits-and-what-is-skipped) | | `Skipped the attachments beyond the first 20 uploadable ones of a result` | warn | Attach fewer files per test | | `Could not attach files to the result of "": ` | warn | The result is recorded without them; check the network ([network](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#network)) | | `Skipped an annotate file "": Vitest 3 loses some bytes of a binary body, so it cannot be sent intact; attach the file with a path instead` | warn | [Vitest annotations](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#vitest-annotations) | | ` annotated files are left out: the attempt each belongs to cannot be told apart` | warn | [Only the last attempt was sent](#only-the-last-attempt-of-a-retried-test-was-sent) | | `The console output of tests are left out: Vitest attributes the writes of concurrent tests to the wrong test, ...` | warn | [captureOutput attaches nothing](#captureoutput-attaches-nothing) | | `[probara] The output of this test was cut here: an attachment holds at most 32 MiB` | — | A line inside `stdout.log` or `stderr.log`, not the log ([limits of the output](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#limits-of-the-output)) | ### Vitest's run | Line | Level | What to do | | ----------------------------------------------------------------------------------------------------------------------- | ----- | ----------------------------------------------------------------------------------- | | `Vitest did not run test files of this run: nothing of them are reported` | warn | [A test file is not reported](#a-test-file-is-not-reported) | | `Vitest retried or repeated tests whose earlier attempts were not recorded: only their final attempts are reported` | warn | [Only the last attempt was sent](#only-the-last-attempt-of-a-retried-test-was-sent) | | `Vitest reported unhandled errors outside the tests: they are not results, and nothing is sent for them` | warn | [Unhandled errors](#unhandled-errors-are-not-sent) | ### In browser mode | Line | Level | What to do | | --------------------------------------------------------------------------------------------------------------------------------------- | ----- | ---------------------------------- | | `probara.attach() did not attach "": Vite does not serve "" (server.fs.allow or server.fs.deny)` | warn | [Browser mode](#browser-mode) | | `probara.attach() did not attach "": Vite's file rules are not reachable from this run, so "" is not served` | warn | [Browser mode](#browser-mode) | | `probara.attach() "" of "" settled after the test ended, so it is not attached. Await probara.attach() ...` | warn | Await the call (test output) | | `Vite has no isFileLoadingAllowed (it arrived in 5.4): the browser attach follows its isFileServingAllowed` | warn | [Browser mode](#browser-mode) | | `Vite exports neither isFileLoadingAllowed nor isFileServingAllowed: the browser attach refuses every path` | warn | [Browser mode](#browser-mode) | | `could not add the browser setup file to project "" (): its retries and repeats are reported as their final attempt only` | warn | Report it, with the Vitest version | | `could not add the browser commands to project "" (): its probara.attach() copies no file` | warn | Report it, with the Vitest version | | `could not give the browser commands to project "": ` | warn | Report it, with the Vitest version | ### Limits of a result Probara's limits per result, each a warning that names the first result it was seen in: | Line | Limit | | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | | `Dropped the steps beyond the first 200 of a result`, `Dropped the steps nested deeper than 10 levels` | Steps of a result | | `Dropped the case steps beyond the first 500`, `Dropped a case step without an action` | Steps of a created case | | `Dropped the parameters beyond the first 20 of a result`, `Truncated a parameter name longer than 100 characters` | Parameters | | `Ignored a parameter with a blank name`, `Ignored a parameter whose name repeats another once trimmed` | Parameters | | `Dropped the case tags beyond the first 50` | Tags of a created case | | `Dropped the case fields beyond the first 50`, `Ignored a case field whose name is longer than 200 characters` | Fields of a created case | | `Ignored a case field with a blank name`, `Ignored a case field whose name repeats another once trimmed and compared ignoring case` | Fields of a created case | | `Dropped the links beyond the first 20 of a result`, `Truncated a link name longer than 255 characters` | Links | | `Ignored a caseDisplayId longer than 64 characters` | Case ids | ### Errors to report These lines mean the reporter met something it did not expect. The run goes on, and what the line names is lost (a test, a file, the console output of a test) or left as it was (the copies of the run's files). Report them at [github.com/cynch-me/probara-sdk/issues](https://github.com/cynch-me/probara-sdk/issues), with the `[probara]` lines of a run with `PROBARA_DEBUG=true` and the Vitest version. | Line | Level | | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | | `Probara reporting is off: the reporter could not start: ` | error | | `Could not report : `, `Could not report "": `, `Could not report the failure of : ` | error | | `Could not finish reporting: `, `Could not read the cases of the run: `, `Could not keep the console output of a test: ` | error | | `Could not remove the attachment copies of this run: ` (remove `/probara/` by hand) | error | | `Skipped the invalid result of "": `, `Ignored results added after complete()` | warn | | `Ignored that is not `, such as `Ignored steps that are not a list`, and `Ignored a blank caseDisplayId` | warn | Related: [Configuration](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/) · [Network](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#network) · [Results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) · Source: [troubleshooting.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/troubleshooting.md), [debugging.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/debugging.md)