# Migrating to the Vitest reporter > Move to @probara/vitest-reporter from the JUnit import, vitest-qase-reporter, Allure, ReportPortal or TestRail's trcli, keeping your cases — and how to upgrade the reporter and Vitest safely. Whatever you come from, the reporter is one entry in `reporters`. Nothing goes in `setupFiles` or `runner`: the reporter adds its own setup file and runner to each Vitest project, in Node and in browser mode. ```ts // vitest.config.mts import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { reporters: ['default', ['@probara/vitest-reporter', { projectId: 'SHOP' }]], }, }); ``` The reporter sends what the Vitest integrations of Qase, Allure, ReportPortal and TestRail send, wherever Probara has a place for it: case links, the title, suite, fields, tags and steps of a new case, parameters, comments, nested steps, files, links, and the run's name, plan, environment and configurations. It also sends every attempt of a retried or repeated test, which none of them does, and keeps one run for a whole watch session. What it does not port is under [not supported](#not-supported), each with its reason. | You come from | Jump to | | ------------------------------------------------ | ------------------------------------------ | | Vitest's JUnit output and `probara import junit` | [The JUnit import](#from-the-junit-import) | | `vitest-qase-reporter` | [Qase](#from-qase) | | `allure-vitest` | [Allure](#from-allure) | | `@reportportal/agent-js-vitest` | [ReportPortal](#from-reportportal) | | Vitest's JUnit output and `trcli parse_junit` | [TestRail](#from-testrail) | ## From the JUnit import If your pipeline writes Vitest's JUnit output and sends it with `probara import junit`, the reporter can replace both steps **without unlinking a single case**: it gives each test the automation key and the case the import gives it, on Vitest 3, 4 and 5. Then each test sends what the XML cannot hold: every attempt, steps, files, parameters and the `probara.*` calls. The reporter is the recommended path; keep the import for a job that only has the XML. 1. Add the reporter to the Vitest config ([get started](https://docs.probara.net/en/developers/ci-integrations/vitest/#3-add-it-to-the-vitest-config)). There is nothing else to set: Vitest's JUnit output always names the test file, as the reporter's default keys do (`keyIncludesFile: true`). 2. [Check the keys](#check-the-keys-before-you-switch) of both on the same tests: they must be equal. 3. In CI, drop `--reporter=junit` and the import step, and give the test step `PROBARA_API_TOKEN` ([in CI](#in-ci)). | Vitest's JUnit output | Reporter setting | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------ | | `--reporter=junit`, with or without `addFileAttribute: true` | Nothing: `keyIncludesFile: true` is the default | | Written with another `ancestorSeparator` (Vitest 5) | Different keys: link the tests by case ids ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)) | Run `vitest` from the same root as before: the file of each key is relative to it, on both paths. `keyIncludesFile: false` has no JUnit counterpart: it gives the keys of cases made by hand or by another tool, without the file ([without the file](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#without-the-file)). ### Check the keys before you switch Run the suite once with Vitest's JUnit output and once with the reporter's [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file), reporting off, and compare the dry runs of both: each prints one line per result, with its status, its case and its key. No token is needed. For this test file: ```ts // tests/refund.test.ts import { describe, expect, test } from 'vitest'; describe('refund', () => { test('SHOP-12 refunds an order', () => { expect(25 - 25).toBe(0); }); test('keeps the receipt', () => { expect('receipt').toHaveLength(7); }); test.todo('refunds a gift card'); }); ``` ```text $ npx vitest run --reporter=junit --outputFile=reports/vitest.xml $ PROBARA_PROJECT=SHOP npx @probara/cli import junit reports/vitest.xml --dry-run passed SHOP-12 tests/refund.test.ts > refund > refunds an order passed - tests/refund.test.ts > refund > keeps the receipt skipped - tests/refund.test.ts > refund > refunds a gift card Total: 3 results from 1 file (2 passed, 0 failed, 1 skipped, 0 blocked) $ PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=probara-results.json npx vitest run [probara] Wrote 3 results to /work/shop/probara-results.json: send them with probara import results /work/shop/probara-results.json $ npx @probara/cli import results probara-results.json --dry-run passed SHOP-12 tests/refund.test.ts > refund > refunds an order passed - tests/refund.test.ts > refund > keeps the receipt skipped - tests/refund.test.ts > refund > refunds a gift card Total: 3 results from 1 file (2 passed, 0 failed, 1 skipped, 0 blocked) ``` The lines are the same: the same keys, cases and statuses. `--reporter=junit` replaces the config's reporters for that run, so the reporter sends nothing then. On a real suite, write each dry run to a file and `diff` them: a line whose key changed is a test that would get a new case ([check what would be sent](https://docs.probara.net/en/developers/ci-integrations/vitest/troubleshooting/#check-what-would-be-sent)). Delete `reports/` and the results file afterwards. ### In CI Before, two steps: Vitest writes the report, and the CLI sends it. ```yaml - name: Run the tests run: npx vitest run --reporter=junit --outputFile=reports/vitest.xml - name: Send the results if: ${{ !cancelled() }} run: npx @probara/cli import junit reports/vitest.xml env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP ``` After, one step: the reporter, registered in the Vitest config, sends every result when the run ends. ```yaml - name: Run the tests run: npx vitest run env: PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }} PROBARA_PROJECT: SHOP ``` Remove `--reporter=junit` from the command: it replaces the reporters of the config, the reporter's included. When another tool still reads the XML (a test summary of the CI), list `junit` beside the reporter instead: ```ts reporters: ['default', 'junit', '@probara/vitest-reporter'], outputFile: { junit: 'reports/vitest.xml' }, ``` The run is named, sourced and closed as the import did it ([the run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#the-run)). ### What changes from the JUnit import The keys stay; what each test sends grows, and a few results become more exact: | Topic | `probara import junit` on Vitest's JUnit output | The reporter | | --------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Retries (`retry`) | One result per test: a pass on a retry is a plain pass; a test that fails every attempt keeps its first failure | Every attempt, in order, each with its own errors; the run case keeps the last outcome ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | Repeats (`repeats`) | One result per test | Every execution ([repeats](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#repeats)) | | `test.todo` | `skipped` | `skipped`, with the note `Todo` ([statuses](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#statuses)) | | `context.skip('why')` | `skipped`: Vitest writes no reason | `skipped`, with the note `Skipped: why` | | `expect.soft` failures | The first failure's message | Every failure of the attempt, in the notes | | Console output | `--attach-output`: the test's `` and `` | `captureOutput`: `stdout.log` and `stderr.log` for each attempt ([console output](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#console-output)) | | Vitest projects | No project: Vitest's JUnit output does not name it | A `project` parameter on each result, when the config has more than one ([Vitest projects](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#vitest-projects)) | | Unhandled errors (Vitest 5) | Left out, with one warning per file; `--fail-on-failed-tests` exits 3 for them | Left out, with one warning; the exit code stays Vitest's ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)) | | When the results go | After the whole run, in a later step | At the end of the run, from the test step; a [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) keeps what could not be sent | The same on both paths, so no case moves: - The statuses of passed, failed and skipped tests, `test.fails` included, and the tests `.only` or `-t` leave out, which are skipped on both. - The failed results of a `beforeAll` or `afterAll` that throws, and of a file that fails to load, named as Vitest's JUnit writer names them ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)). - Case ids in test and `describe` titles, the names of `test.each` and `test.for` rows, and a `>` with a space on each side inside a title ([names as Vitest writes them](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#names-as-vitest-writes-them)). ### What the import cannot know The XML holds each test's name, outcome, time and output. What the tests tell the reporter, and what Vitest gives it beyond its JUnit output, never reaches the import: - **Every attempt** of a retried or repeated test, each with its status, duration and errors. - **Case ids from `probara.id()`**, which live in the test's own metadata, not in its title ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)). - **Steps**, nested, with their status and duration, and the steps of a new case ([steps](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probarastep)). - **Files**: `probara.attach()`, the files of Vitest's own `context.annotate()`, failure screenshots in browser mode ([attachments](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probaraattach)). - **Parameters, comments, links and the fields of a new case**: the `probara.*` helpers ([metadata and links](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)). - **Run selection**: run only the tests of a Probara run ([run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/)). Keep the import where the reporter cannot run: a job that only receives the XML of another job, or a Vitest older than 3.2, which the reporter does not support (the import is tested with Vitest 3.2, 4.1 and 5.0). The import's side is in the [CLI](https://docs.probara.net/en/developers/cli/) and in [Vitest JUnit](https://github.com/cynch-me/probara-sdk/blob/main/packages/cli/docs/junit.md#vitest). ### Cases made by an earlier import `probara import junit` reads Vitest's output with its `vitest` dialect, detected from the report's root (`vitest tests`), with no option. Versions of the CLI before that dialect (0.3.0 and earlier) read the same XML as generic JUnit, which gives every test the same key, with two differences: | Result | Generic JUnit (the CLI 0.3.0 and earlier) | The reporter, and the `vitest` dialect | | ----------------------------------------------------------------- | ----------------------------------------------- | ------------------------------------------------------------- | | A failure named after its file, such as a file that fails to load | Keyed by the file alone: `tests/broken.test.ts` | The file twice: `tests/broken.test.ts > tests/broken.test.ts` | | An unhandled error (Vitest 5) | A failed result of its own | Left out, with a warning | Only those results move: the next failure of such a file creates a case under its new key, and the old case gets no more results. Every other test keeps its case. ## From Qase Moving from `vitest-qase-reporter` is mostly renaming: the reporter entry, its settings, and the `ctx.qase` calls, which become `probara.*` calls without a `withQase()` wrapper. This section maps each one as Qase's code (version 1.5.1) has it, which differs from Qase's documentation in places ([documented, but not in the code](#documented-but-not-in-the-code)), then lists what works differently and what is not ported, with the reasons. 1. `npm uninstall vitest-qase-reporter && npm i -D @probara/vitest-reporter`. 2. In `vitest.config.mts`, replace `['vitest-qase-reporter', {...}]` with `['@probara/vitest-reporter', {...}]`, translating the options (below). Delete `qase.config.json` and `.qaserc`: the reporter reads only its options and `PROBARA_*` variables. Take `vitest-qase-reporter/setup` out of `setupFiles` if you added it: the reporter adds its own setup file. 3. Replace `QASE_*` variables with `PROBARA_*` ones in CI: at least `PROBARA_API_TOKEN` (an app token from the **Vitest** card, [get started](https://docs.probara.net/en/developers/ci-integrations/vitest/#1-create-an-app-token)) and `PROBARA_PROJECT`. 4. Import `probara` from `@probara/vitest-reporter`, turn each `withQase(async ({ qase }) => {...})` back into a plain test function, and translate the calls ([below](#the-ctxqase-calls)). Case ids become Probara display ids (`SHOP-12`), not numbers. Before: ```ts // vitest.config.mts import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { reporters: [ 'default', [ 'vitest-qase-reporter', { mode: 'testops', testops: { project: 'SHOP', run: { title: 'Nightly', complete: true, tags: ['nightly'] }, uploadAttachments: true, }, }, ], ], }, }); ``` After: ```ts // vitest.config.mts import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { reporters: [ 'default', [ '@probara/vitest-reporter', { projectId: 'SHOP', run: { name: 'Nightly', tags: ['nightly'] }, captureOutput: true }, ], ], }, }); ``` `captureOutput` is optional: it attaches each attempt's console output, which Qase's `captureLogs` never did for Vitest ([console output](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#console-output)). The run is closed at the end without `complete: true`, and `uploadAttachments` is on by default. ### Qase options and variables | Qase option (`QASE_*` variable) | Probara option (`PROBARA_*` variable) | | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `mode: 'testops'` (`QASE_MODE`) | On whenever a token and a project are set | | `mode: 'off'` | `enabled: false` (`PROBARA_ENABLED=false`), or no token | | `mode: 'report'`, `fallback` (`QASE_FALLBACK`) | `resultsFile` (`PROBARA_RESULTS_FILE`) ([results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file)) | | `report.connections.local.path`, `.format` (`QASE_REPORT_CONNECTION_PATH`, `_FORMAT`) | `resultsFile`, the path of a JSON file | | `mode: 'testops_multi'`, `testops_multi.*` | `projectId` plus `projects` and `run.ulids` ([several projects](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#several-probara-projects)) | | `debug` (`QASE_DEBUG`) | `debug` (`PROBARA_DEBUG`) | | `environment` (`QASE_ENVIRONMENT`), a slug, ignored when not found | `run.environment` (`PROBARA_ENVIRONMENT`), a name, created when missing | | `rootSuite` (`QASE_ROOT_SUITE`) | `suiteUlid` (`PROBARA_SUITE_ULID`): an existing suite, by ULID | | `captureLogs` (`QASE_CAPTURE_LOGS`): ignored by Qase's Vitest reporter | `captureOutput` (`PROBARA_CAPTURE_OUTPUT`) | | `statusMapping` (`QASE_STATUS_MAPPING`) | `statusMapping` (`PROBARA_STATUS_MAPPING`), without `invalid` and `disabled` | | `logging.console`, `logging.file` (`QASE_LOGGING_CONSOLE`, `QASE_LOGGING_FILE`) | Logs go to stderr; the `logger` option for code | | `testops.api.token` (`QASE_TESTOPS_API_TOKEN`) | `PROBARA_API_TOKEN` (`apiToken` for code) | | `testops.api.host` (`QASE_TESTOPS_API_HOST`) | `baseUrl` (`PROBARA_BASE_URL`), a full URL | | `testops.api.timeout` (`QASE_TESTOPS_API_TIMEOUT`), in seconds | `timeoutMs`, in milliseconds, for every request | | `testops.api.retries`, `testops.api.retryBackoff` | `maxRetries`, for every request; the backoff is fixed ([retries and backoff](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#retries-and-backoff)) | | `testops.project` (`QASE_TESTOPS_PROJECT`) | `projectId` (`PROBARA_PROJECT`) | | `testops.uploadAttachments` (`QASE_TESTOPS_UPLOAD_ATTACHMENTS`) | `uploadAttachments` (`PROBARA_UPLOAD_ATTACHMENTS`) | | `testops.attachments.concurrency`, `.timeout` | `attachmentConcurrency`; uploads get at least 120 s ([timeouts](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#timeouts)) | | `testops.run.id` (`QASE_TESTOPS_RUN_ID`) | `run.ulid` (`PROBARA_RUN_ULID`) | | `testops.run.title`, `.description`, `.tags` | `run.name`, `run.description`, `run.tags` (`PROBARA_RUN_*`) | | `testops.run.complete` (`QASE_TESTOPS_RUN_COMPLETE`) | `closeRun` (`PROBARA_CLOSE_RUN`); a created run is closed by default | | `testops.plan.id` (`QASE_TESTOPS_PLAN_ID`) | `run.plan` (`PROBARA_PLAN`): the plan's display id or name | | `testops.configurations.values` (`group=value`) | `run.configurations` (`PROBARA_CONFIGURATIONS=Group=Name`) | | `testops.statusFilter` (`QASE_TESTOPS_STATUS_FILTER`) | `statusFilter` (`PROBARA_STATUS_FILTER`) | | `testops.batch.size` (`QASE_TESTOPS_BATCH_SIZE`) | `chunkSize` (1 to 500) | | (none) | `run.milestone` (`PROBARA_MILESTONE`): Qase's reporter has no milestone | Qase settings without a counterpart are in [not ported](#not-ported-from-qase). Every Probara option is on the [configuration](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#options) page. ### The `ctx.qase` calls | Qase | Probara | | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `test(addQaseId('pays', [12]), ...)`, `(Qase ID: 12)` in a title | `test('SHOP-12 pays', ...)`: an id in the title or a `describe` ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)) | | `addQaseId('pays', [12, 13])` | `'SHOP-12 SHOP-13 pays'`, or `probara.id(['SHOP-12', 'SHOP-13'])` | | `addQaseProjects('pays', { WEB: [3] })`, `(Qase WEB: 3)` in a title | `WEB-3` in the title, with `projects: ['WEB']` ([several projects](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#several-probara-projects)) | | `withQase(async ({ qase }) => {...})` | The test function itself: `probara` finds the running test on its own | | `await qase.title('...')` | `probara.title('...')` | | `await qase.suite('Payments - Cards')` | `probara.suite(['Payments', 'Cards'])` | | `await qase.fields({ severity: 'critical' })` | `probara.fields({ severity: 'critical' })` | | `await qase.tags('smoke', 'checkout')` | `probara.tags('smoke', 'checkout')` | | `await qase.comment('...')` | `probara.comment('...')` | | `await qase.parameters({...})`, `await qase.groupParameters({...})` | `probara.parameters({...})` | | `await qase.step('Pay', fn, 'Paid', 'card=visa')` | `await probara.step('Pay', fn, { expected: 'Paid', data: 'card=visa' })` | | `await qase.attach({ paths: [a, b] })` | `await probara.attach({ name, path })`, once per file | | `await qase.attach({ name, content, type })` | `await probara.attach({ name, body, contentType })` | | `await qase.annotate(message, options)` | `await context.annotate(message, options)`: Vitest's own; its file is sent ([Vitest annotations](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#vitest-annotations)) | | (none: Qase's Vitest reporter has no `ignore`) | `probara.ignore()` | | (none) | `probara.link()` and `probara.issue()` ([links and issues](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#links-and-issues)) | Remove the `(Qase ID: 12)` markers from the titles: the reporter reads them as text, so they would stay in the automation key. Before: ```ts // tests/checkout.test.ts import { test } from 'vitest'; import { addQaseId, withQase } from 'vitest-qase-reporter/vitest'; test( addQaseId('pays with a card', [12]), withQase(async ({ qase }) => { await qase.fields({ severity: 'critical' }); await qase.tags('smoke'); await qase.step('Pay', async () => {}, 'The order is paid'); await qase.attach({ name: 'order.json', content: '{"id":1042}', type: 'application/json' }); }), ); ``` After: ```ts // tests/checkout.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('SHOP-12 pays with a card', async () => { probara.fields({ severity: 'critical' }).tags('smoke'); await probara.step('Pay', async () => {}, { expected: 'The order is paid' }); await probara.attach({ name: 'order.json', body: '{"id":1042}', contentType: 'application/json', }); }); ``` ```json [ { "caseDisplayId": "SHOP-12", "automationKey": "tests/checkout.test.ts > pays with a card", "status": "passed", "steps": [{ "action": "Pay", "expected": "The order is paid", "status": "passed" }], "case": { "tags": ["smoke"], "fields": { "severity": "critical" }, "steps": [{ "action": "Pay", "expected": "The order is paid" }] } } ] ``` The result also carries `order.json` (`application/json`). The id leaves the title and the key: adding or removing it never changes which case the test matches ([automation keys](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#automation-keys)). ### Documented, but not in the code Qase's README and guides show calls and settings that version 1.5.1 does not have, or has with another behavior. Code copied from them fails or does something else: | Qase's documentation | Qase's code (1.5.1) | With Probara | | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `test(qase(12, 'pays'), ...)`, with `qase` imported from `vitest-qase-reporter` or `vitest-qase-reporter/vitest` | No `qase` is exported: only `withQase`, `addQaseId` and `addQaseProjects` | `test('SHOP-12 pays', ...)` | | `qase.title()`, `qase.fields()`, `qase.suite()` on that import, in any test | Only `ctx.qase`, in a function `withQase()` wraps | `probara.title()`, `probara.fields()`, `probara.suite()`, in any test | | `qase.ignore()` | Not in the code | `probara.ignore()` | | `qase.mute()` | Not in the code | Not ported ([below](#not-ported-from-qase)) | | `setupFiles: ['vitest-qase-reporter/setup']`, for the network profiler | Not exported by the package: Vitest cannot load it | Nothing to add: the reporter adds its own setup file | | `qase.suite('Payments\tCards')`, a tab between suites | Split only on a dash with a space on each side (`'Payments - Cards'`) | `probara.suite(['Payments', 'Cards'])` | | Steps nested in steps | A step that holds steps is lost; only the inner ones are kept | Nested as they ran ([steps](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probarastep)) | | An attachment of a step | Always the test's | `probara.attach()` inside a step ([files of a step](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#files-of-a-step)) | | The reporter's options win over `QASE_*` variables, which win over `qase.config.json` | Variables win over the file, which wins over the options | Options, then variables ([precedence](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#sources-and-precedence)) | | `testops.run.complete` is `true` by default | Unset leaves the run open | A created run is closed by default ([closing](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#closing)) | | `report.connection.*` | `report.connections.*` | `resultsFile` | | Watch mode reports when you exit or run every test again | Each re-run creates a new run, unless `run.id` is set | One run for the whole session ([watch mode](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode)) | | `api.token` per project of `testops_multi` | One token for every project | One app token for every project of the organization | ### What is different from Qase, and why | Topic | Qase | Probara | | ----------------------------------- | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Precedence | Variables win over `qase.config.json`, which wins over the reporter options | Options in the Vitest config, then `PROBARA_*` variables, then defaults: what the config says is what runs | | Config file | `qase.config.json` or `.qaserc` | None: a config file holding a token could be committed. Options live in the Vitest config | | Case ids | Numbers, in the test's own title only | Display ids (`SHOP-12`) in the title or any `describe`: the Project ID routes each id to its project | | The helpers | `ctx.qase`, inside `withQase()`, until the test body ends: Vitest refuses its annotations in `afterEach` | `probara`, imported once, works in the body, `beforeEach`, `afterEach`, `onTestFinished` and the functions they call ([where to call them](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#where-to-call-the-helpers)) | | Steps | Flat; the expected result and data are encoded into the step name; no duration | Nested as they ran, each with its status, duration and error; the outermost become a new case's steps ([steps](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probarastep)) | | Statuses | `invalid` for a failure whose error does not look like an assertion (a guess from the message) | `passed`, `failed`, `skipped`; `blocked` only through `statusMapping`. No guessing from error text ([statuses](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#statuses)) | | Errors | The first error's message, which `comment()` replaces; every stack in a field of its own | Every error of the attempt in the notes, `expect.soft` failures included; a comment comes first and hides none | | `fields()` and `parameters()` calls | The last call replaces the others | Merged by name | | Case fields | Sent with every result | Only when the report creates the case: an existing case keeps what people wrote ([created cases only](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#created-cases-only)) | | Retries | The final attempt only | Every attempt is a result, with no flaky flag: the case's history shows the flakiness ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | Closing the run | Only with `run.complete: true`; unset leaves it open | A run the reporter creates is closed at the end, once everything was sent; a reused one stays open ([closing](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#closing)) | | Shards | Each shard makes its own run unless it gets a run id; closing a shared run is left to another step | One run: `vitest --merge-reports` over the shards' blobs, or `probara run create` with `PROBARA_RUN_ULID` ([sharding](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/)) | | When reporting fails | `fallback` to a local report | A results file, sent later by `probara import results` into the same runs ([results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file)) | | Strings in `attach()` | A string that looks like base64 is decoded (`'Hello'` uploads garbage) | A string is sent as it is; pass a `Buffer` or a `Uint8Array` for binary content ([`probara.attach()`](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#probaraattach)) | | `attach({ paths })` | A list of paths; the content type is `type`, not `contentType` | One file per call, `path` or `body`, and `contentType` | ### Not ported from Qase | Qase feature | Why | | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Public report link (`showPublicReportLink`) | A public link to test results is a sharing decision for the Probara app, not something a CI job should turn on | | Automatic defects (`testops.defect`) | Filing a defect for every failure floods the defect list; report defects from the result in Probara | | Network profiler (`profilers: ['network']`, `networkProfiler`) | Its steps are noise, up to the 200 steps of a result, and cannot be told apart when tests run in parallel; attach what matters with `probara.attach()` ([no network profiler](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#not-supported-a-network-profiler)) | | Jira link on runs (`run.externalLink`) | Probara links Jira issues to cases and defects, where traceability lives; `probara.issue()` links a result to an issue | | `qase.mute()` | Documented by Qase, but not in its code; triage lives in Probara, on the results | | Creating configurations (`createIfNotExists`) | Configurations are part of a project's planning; an unknown one is refused, so a typo never creates a new one | | The `invalid` status | It is a guess from the error text; Probara keeps four statuses and the whole error in the notes | | Logging to a file (`logging.file`) | Logs go to stderr, where CI keeps them; redirect stderr to keep a file | | Per-project plan and environment (`testops_multi.projects[].plan`, `.environment`) | Planning entities belong to one project, where an unknown name would refuse the report: create that project's run first with `probara run create` and pass it in `run.ulids` ([one run per project](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#one-run-per-project)) | | A config file (`qase.config.json`, `.qaserc`) | A file holding a token could be committed: the token stays in the environment, the options in the Vitest config | ## From Allure `allure-vitest` is a reporter that adds its setup file and its runner to each Vitest project, and writes result files into `allure-results/`; a separate step turns them into a report. The reporter sends the results to Probara instead, and adds its own setup file and runner the same way: swap the entries. ```ts // vitest.config.mts import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { reporters: [ 'default', [ 'allure-vitest/reporter', { resultsDir: 'allure-results', links: { issue: { urlTemplate: 'https://jira.example.com/browse/%s' } }, }, ], ], }, }); ``` ### Allure settings | Allure | Probara | | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `resultsDir` | Nothing: results go to Probara. A [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) keeps what could not be sent | | `links.issue.urlTemplate` | `issueUrlTemplate` (`PROBARA_ISSUE_URL_TEMPLATE`) ([`probara.issue()`](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#probaraissue-and-issueurltemplate)) | | `environmentInfo` | `run.environment` (a name) and the run source, detected from the CI ([the run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#the-run)) | | `globalLabels`, `ALLURE_LABEL_*` | `run.tags` (`PROBARA_RUN_TAGS`) for the run; the tags of a new case come from `probara.tags()` | | `ALLURE_TESTPLAN_PATH` | `runCasesOnly` with `PROBARA_RUN_ULID` ([run selection](https://docs.probara.net/en/developers/ci-integrations/vitest/run-selection/)) | | `reportMatchers`, `categories`, `listeners` | Not ported ([not supported](#not-supported)) | ### Allure calls Allure's calls are async (`await allure.tags('smoke')`); Probara's return at once, except `probara.attach()` and a `probara.step()` with an async body, which you await. | Allure (`allure-js-commons`) | Probara (`probara.*`) | | ------------------------------------------------ | ------------------------------------------------------------------------------ | | `@allure.id:12` in a title, `allureId('12')` | `SHOP-12` in the title or a `describe`, or `id('SHOP-12')` | | `displayName('...')` | `title('...')` | | `description('...')` | `fields({ description: '...' })` | | `severity('critical')`, `layer('api')` | `fields({ severity: 'critical', layer: 'api' })` | | `tag('smoke')`, `tags(...)` | `tags('smoke')` | | `epic`, `feature`, `story`, `label(name, value)` | `fields({ Feature: 'Checkout' })`: a custom field of that name, or `tags(...)` | | `parentSuite`, `suite`, `subSuite` | `suite(['Payments', 'Cards'])` | | `parameter('user', 'ana')` | `parameters({ user: 'ana' })` | | `step('Pay', async () => {...})` | `step('Pay', async () => {...})` | | `logStep('Paid')` | `step('Paid')`: a step without a body, passed | | `attachment(name, content, { contentType })` | `attach({ name, body: content, contentType })` | | `attachmentPath(name, path)` | `attach({ name, path })` | | `link(url, name)` | `link(url, name)` | | `issue('PAY-7')` | `issue('PAY-7')`, with `issueUrlTemplate` | | `tms('12')` | `id('SHOP-12')` | ```ts reporters: [ 'default', ['@probara/vitest-reporter', { issueUrlTemplate: 'https://jira.example.com/browse/%s' }], ], ``` ```ts // tests/payments.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('refunds an order', async () => { probara.fields({ severity: 'critical' }).issue('PAY-7'); await probara.step('Open the order', async () => {}); await probara.step('Refund it', async () => {}); }); ``` ```json [ { "automationKey": "tests/payments.test.ts > refunds an order", "steps": [ { "action": "Open the order", "status": "passed" }, { "action": "Refund it", "status": "passed" } ], "links": [{ "url": "https://jira.example.com/browse/PAY-7", "name": "PAY-7" }], "case": { "fields": { "severity": "critical" }, "steps": [{ "action": "Open the order" }, { "action": "Refund it" }] } } ] ``` ### What works differently from Allure | Topic | Allure | Probara | | --------------------------------- | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Statuses | `failed` for an assertion, `broken` for any other error | `failed` for both, with the error in the notes: no guess from the error's type ([statuses](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#statuses)) | | `test.skip`, a skipped `describe` | Not reported | Skipped results, as Vitest's JUnit output has them | | Assertions | Each `expect` is a step (`reportMatchers`, on by default) | The steps are the `probara.step()` calls | | A call where no test runs | A call in `beforeAll` or a `describe` body goes to every test of the suite | Left out, with one warning: no single test runs there ([where to call the helpers](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#where-to-call-the-helpers)) | | A hook that throws | Its error is a global error of the report, beside the tests | Vitest's verdict: a failing `beforeEach` or `afterEach` fails its test, a failing `beforeAll` or `afterAll` is a failed result for its `describe` or file ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)) | | Retries | One result per test, the final attempt | Every attempt is a result of the run ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | Watch mode | New result files on every re-run, which pile up in `resultsDir` | One run for the whole session ([watch mode](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode)) | | Case fields | Labels on every result | Fields and tags of the case a report creates; an existing case keeps its own ([created cases only](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#created-cases-only)) | | `test.concurrent` in browser mode | A call can go to the wrong test after an `await` (its README says so) | A call is dropped with a warning, never recorded in another test ([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/#not-supported-yet)) | ## From ReportPortal `@reportportal/agent-js-vitest` is a reporter object in `reporters`, with a setup file you list in `setupFiles` for its `ReportingApi`. Replace both with the reporter entry above: ```ts // vitest.config.mts import RPReporter from '@reportportal/agent-js-vitest'; import { defineConfig } from 'vitest/config'; export default defineConfig({ test: { setupFiles: ['@reportportal/agent-js-vitest/setup'], reporters: [ 'default', new RPReporter({ apiKey: process.env.RP_API_KEY, endpoint: 'https://rp.example.com/api/v2', project: 'shop', launch: 'Nightly', }), ], }, }); ``` ### ReportPortal settings | ReportPortal (variable) | Probara (variable) | | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `apiKey`, `oauth` | `PROBARA_API_TOKEN`: an app token from the **Vitest** card ([get started](https://docs.probara.net/en/developers/ci-integrations/vitest/#1-create-an-app-token)) | | `endpoint` | `baseUrl` (`PROBARA_BASE_URL`), the app's URL without a path | | `project` | `projectId` (`PROBARA_PROJECT`) | | `launch` | `run.name` (`PROBARA_RUN_NAME`) | | `description` | `run.description` (`PROBARA_RUN_DESCRIPTION`) | | `attributes` (`[{ key: 'build', value: '42' }]`) | `run.tags` (`PROBARA_RUN_TAGS=build:42,nightly`): tags are plain names, a colon included | | `launchId` (`RP_LAUNCH_ID`) | `run.ulid` (`PROBARA_RUN_ULID`) | | `launchUuidPrint` | `probara run create` prints the ULID every shard reports into ([an existing run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#an-existing-run)) | | `restClientConfig` (timeout, retries, proxy) | `timeoutMs`, `maxRetries`; a proxy through Node's variables ([network](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#network)) | | `headers` | A `fetch` of your own ([your own HTTP client](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#your-own-http-client)) | | `debug` | `debug` (`PROBARA_DEBUG`) | | `extendTestDescriptionWithLastError` | Nothing to set: every error of an attempt is in its notes | | `skippedIssue`, `rerun`, `rerunOf`, `mode: 'DEBUG'` | Not ported ([not supported](#not-supported)) | ### ReportPortal calls | ReportPortal (`ReportingApi`) | Probara (`probara.*`) | | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ReportingApi.attachment(task, { name, type, content })` | `attach({ name, body, contentType })`, with a `Buffer` or text, or `attach({ name, path })` | | `ReportingApi.description(task, text)` | `fields({ description: text })`, for a new case | | `ReportingApi.attributes(task, [{ key, value }])` | `parameters({ [key]: value })` on the result, or `tags(value)` on a new case | | `ReportingApi.testCaseId(task, id)` | Not ported: link a case by its id instead ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)) | | `ReportingApi.log(task, message, level)` | Not ported: `comment()` for a line in the notes, `captureOutput` for the console | | `console.log()`, `console.error()`, as logs | `captureOutput`: `stdout.log` and `stderr.log` of each attempt ([console output](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#console-output)) | | The global `ReportingApi` of the setup file | `probara`, imported: there is no `task` to pass | A base64 string as `content` would be sent as text: give `attach()` the file's `path`, or its bytes as a `Buffer`. ```ts // tests/checkout.test.ts import { probara } from '@probara/vitest-reporter'; import { test } from 'vitest'; test('pays with a card', async () => { probara.fields({ description: 'Pays with a saved card' }).parameters({ browser: 'chromium' }); await probara.attach({ name: 'response', body: '{"status":"paid"}', contentType: 'application/json', }); }); ``` ```json [ { "automationKey": "tests/checkout.test.ts > pays with a card", "parameters": { "browser": "chromium" }, "case": { "description": "Pays with a saved card" } } ] ``` The result also carries `response.json` (`application/json`). ### What works differently from ReportPortal | Topic | ReportPortal | Probara | | -------------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | In-test API | `ReportingApi`, global from its setup file, or with the test's `task` passed in | `probara`, imported; each call belongs to the test that made it ([metadata](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)) | | `test.concurrent` | The global `ReportingApi` follows the last test that started, so a call can go to another test | Each call goes to its own test | | Retries | The final attempt only | Every attempt is a result ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | `test.todo` | Skipped, with the attribute `todo` | Skipped, with the note `Todo` | | A skipped `describe` | One skipped suite item; its tests are not reported | Each test is a skipped result | | Errors | The first error, as a log and in the description | Every error of the attempt, in the notes of its result | | Structure | A suite item per `describe` | Each new case in a suite named after its file ([missing cases](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#missing-cases)) | | Watch mode | The launch ends with the first run, and the re-runs have none | One run for the whole session ([watch mode](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#watch-mode)) | ## From TestRail TestRail has no Vitest reporter: a pipeline writes Vitest's JUnit output, then TestRail's Python CLI `trcli parse_junit` sends it after the run. The reporter replaces both steps, and sends what the XML cannot hold: steps, files, comments and every attempt. ```bash npx vitest run --reporter=junit --outputFile=reports/vitest.xml trcli -h https://example.testrail.io --project "Shop" -u ci@example.com -k "$TESTRAIL_KEY" \ parse_junit --title "Nightly" --case-matcher name -f reports/vitest.xml --close-run ``` With the reporter, the test step is enough: ```bash npx vitest run ``` ### trcli settings | trcli | Probara | | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-h` (host), `-u`, `-p`, `-k` | `baseUrl` for a self-hosted Probara; `PROBARA_API_TOKEN` | | `--project` (a name) | `projectId` (`PROBARA_PROJECT`), the Project ID | | `--title`, `--run-description` | `run.name`, `run.description` | | `--run-id` | `run.ulid` (`PROBARA_RUN_ULID`) | | `--plan-id`, `--config-ids`, `--milestone-id` | `run.plan`, `run.configurations`, `run.milestone`, by display id or name ([the run](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#the-run)) | | `--close-run` | The default for a run the reporter creates; `closeRun` for a reused one | | `--case-matcher name` (`C123` in the name) | `SHOP-12` in the title or a `describe` ([linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/)) | | `--case-matcher auto` (`classname.name`) | The automation key: the file, the describes and the title | | `-y` (create sections and cases) | `createMissingCases`, on by default; suites from the test file | | `--suite-id`, `--section-id` | `suiteUlid` (`PROBARA_SUITE_ULID`): the suite created cases go under | | `--case-fields type_id:3` | `probara.fields({ type: 'Regression' })` in the test, by name ([metadata](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/)) | | `-a`, `--assign` (emails) | `assignFailedTo` (`PROBARA_ASSIGN_FAILED_TO`) ([assign failed results](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#assign-failed-results)) | | `-b`, `--batch-size` | `chunkSize` (1 to 500) | | `-t`, `--timeout` (seconds) | `timeoutMs` (milliseconds, for each attempt of a request) | | `--proxy`, `--noproxy` | `HTTPS_PROXY` and `NO_PROXY` with `NODE_USE_ENV_PROXY=1` ([proxies](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#proxies)) | | `-v` (verbose) | `debug` (`PROBARA_DEBUG`) | | `testrail_result_comment` property | `probara.comment()` | | `testrail_attachment` property | `probara.attach()` | | `testrail_result_step` property | `probara.step()` | ### What works differently from trcli | Topic | trcli on Vitest's JUnit output | Probara | | ------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Skipped tests | Retest | Skipped | | `test.todo` | Retest: Vitest writes it as skipped | Skipped, with the note `Todo` | | Retries | One result per test | Every attempt ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | Nested describes | One section per test file, flat: Vitest writes a suite per file | The suite of a created case is its file; `probara.suite()` names nested suites | | Durations | Rounded to seconds (`--allow-ms` for milliseconds) | Milliseconds, and the start of each attempt | | An unknown case id | Aborts the upload | That result is not recorded, and the log names it; the others are | | A failed upload | Rolls back the run, cases and sections it created | Nothing is rolled back; a [results file](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#results-file) keeps what was not sent | | Shards | One upload of every shard's file, with a glob | `vitest --merge-reports` over the blobs, or `probara run create`, `PROBARA_RUN_ULID` in every shard, `probara run close` ([sharding](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/)) | ## Not supported Each of these is left out, with the reason: | Feature | In | Why | | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Updating existing cases | trcli (`--update-existing-cases`, `--update-strategy`) | A report never changes an existing case: people own what is written there ([created cases only](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#created-cases-only)) | | A flaky flag | Vitest (`diagnostic().flaky`) | Every attempt is a result: a case whose history mixes failures and passes on one run is flaky, without a flag ([retries](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#retries-and-repeats)) | | Assertions as steps | Allure (`reportMatchers`) | Each `expect` would be a step, up to the 200 steps of a result, crowding out the steps that say what the test does: wrap what matters in `probara.step()` | | The `broken` and `invalid` statuses | Allure (`broken`), Qase (`invalid`) | Both are a guess from the error's type or text; Probara keeps four statuses (`passed`, `failed`, `skipped`, `blocked`) and the whole error in the notes | | Masked, hidden and excluded parameters | Allure (the modes of `parameter()`) | Parameters are shown with the result and never part of the key: leave a secret out rather than masking it | | Log levels and a log API | ReportPortal (`ReportingApi.log()`), Allure (`logStep()` with a status) | Probara has no log entity: `captureOutput` keeps each attempt's console output, `probara.comment()` writes a line in the notes, and a step's status is what its body did | | Defects, Jira links on runs, public report links | Qase (`testops.defect`, `run.externalLink`, `showPublicReportLink`) | Defects and their triage live in Probara, on the results; `probara.issue()` links a result to an issue; a public link to results is a sharing decision, not a CI setting | | A network profiler | Qase (`profilers: ['network']`) | Its steps are noise, up to the 200 steps of a result, and cannot be told apart when tests run in parallel ([no network profiler](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#not-supported-a-network-profiler)) | | Categories, known issues, muted tests | Allure (`categories`), ReportPortal (`skippedIssue`), Qase (`qase.mute()`, documented only) | Triage lives in Probara, on the results and defects, not in a CI setting | | Overriding the automation key | Allure (`historyId()`, `testCaseId()`), ReportPortal (`testCaseId()`), trcli (`custom_automation_id`) | The key comes from the file and the names, the same as `probara import junit`'s: link a case by its id instead ([automation keys](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#automation-keys)) | | An owner | Allure (`owner()`) | A Probara case has no owner field: assign failed results instead ([assign failed results](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#assign-failed-results)) | | Result fields | trcli (`--result-fields`, `testrail_result_field`) | Probara results have no custom fields: parameters (`probara.parameters()`) and links (`probara.link()`) carry the extra data | | Steps and calls from suite hooks | Allure (a call in `beforeAll` goes to every test of the suite) | A call where no single test runs belongs to no test: it is left out with a warning, not copied into each one. Make the steps that matter in the test or its `beforeEach`; a hook that throws still shows ([errors outside a test](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#errors-outside-a-test)) | | A config file | Qase (`qase.config.json`, `.qaserc`), trcli (`config.yml`) | A file holding a token could be committed: the token stays in the environment, the options in the Vitest config ([configuration](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/)) | | Benchmarks and coverage | Vitest (`vitest bench`, `--coverage`) | Probara has no benchmark or coverage results: the reporter sends test results | | Type tests | Vitest (`vitest typecheck`) | No test of the reporter covers them, so nothing is promised for them | | Rerun and debug launches | ReportPortal (`rerun`, `rerunOf`, `mode: 'DEBUG'`) | Report again into the same run with `run.ulid`: each run case keeps the last outcome and its history. To try a setup, [check what would be sent](https://docs.probara.net/en/developers/ci-integrations/vitest/troubleshooting/#check-what-would-be-sent) | | Signing in with OAuth | ReportPortal (`oauth`) | CI reports with an app token of the organization, which uses no seat and outlives the person who made it | | Global files and errors | Allure (`globalAttachment()`, `globalError()`) | Files and errors belong to a result: attach them from the test that produced them | | HTML descriptions | Allure (`descriptionHtml()`) | The description is text; Probara formats it | | Listeners | Allure (`listeners`) | No hook changes a result before it is sent; `statusMapping` and `statusFilter` cover statuses ([status mapping](https://docs.probara.net/en/developers/ci-integrations/vitest/results-and-retries/#status-mapping-and-filter)) | | Turning TLS verification off | trcli (`--insecure`) | It cannot be turned off: give Node your certificate authority instead ([custom certificate authorities](https://docs.probara.net/en/developers/ci-integrations/vitest/configuration/#custom-certificate-authorities)) | | Rolling back a failed upload | trcli | A result recorded is kept: rolling back would lose the ones that went through | | References on runs | trcli (`--test-run-ref`) | The run records its CI build URL, branch and commit; issues link to results with `probara.issue()` | ## Upgrading the reporter `@probara/vitest-reporter` follows [semantic versioning](https://semver.org). Read the [changelog](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/CHANGELOG.md) before you upgrade it. A new Vitest major can change the names it gives tests, and so their automation keys: before you upgrade Vitest, read [from Vitest 4 to 5](#from-vitest-4-to-5) and [compare the keys](#the-automation-key-contract). - **0.x (now)**: the reporter is young. A minor version (`0.2.0`) may change options, log lines or defaults; a patch version (`0.1.1`) only fixes bugs. Pin the version in `package.json` (or keep the lockfile) and upgrade on purpose. - **1.0 and later**: breaking changes only in a major version. - `@probara/vitest-reporter` depends on `@probara/core` of the same release line; install the reporter alone and let it bring its core. ### What is a breaking change Any of these needs a major version from 1.0 on, and a minor one before: - An option or a `PROBARA_*` variable removed or renamed, or its meaning changed. - A default changed (closing runs, creating missing cases, `keyIncludesFile`, what gets attached). - A `probara.*` helper removed, or its behavior changed. - A status an attempt is sent with changed for the same attempt. - **Any change to the automation key a test gets**, with `keyIncludesFile` on or off. A different key unlinks the test from its case: the next run creates a new one. - Dropping a Vitest or Node.js version that is still supported upstream. Adding an option, a helper, a log line or a file uploaded is not breaking. Log lines on stderr are for people: do not parse them. ### The automation key contract The key algorithm is [automation key v1](https://github.com/cynch-me/probara-sdk/blob/main/packages/core/README.md#the-automation-key-v1), frozen in `@probara/core` and pinned by golden vectors: no version of the reporter changes it. What the reporter adds is how a Vitest test becomes the key's parts, the way `probara import junit` reads Vitest's JUnit output ([keys](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/#automation-keys)): - the test file relative to `rootDir` (the Vitest root), when `keyIncludesFile` is on; - the `describe` titles and the test title, each a part, as Vitest's JUnit output names them, the case ids removed; - a `test.each` or `test.for` row by the title Vitest formats for it. That mapping is part of the contract too: a version of the reporter that changes the key of a test is a breaking change, named as such in the changelog. A version of Vitest that formats titles differently changes keys as well (below), for the reporter and for `probara import junit` alike. To check an upgrade of either, compare what both versions would send: run the suite with reporting off once per version, each with its own results file, and diff the dry runs ([check what would be sent](https://docs.probara.net/en/developers/ci-integrations/vitest/troubleshooting/#check-what-would-be-sent)). Any changed key is a test that would get a new case. ### Vitest and Node.js versions | Needs | Supported | | ------------ | ----------------------------------------------------------------------------------------------- | | `vitest` | 3.2 and later (the peer range `^3.2.0 \|\| ^4.0.0 \|\| ^5.0.0`): tested on 3.2.7, 4.1.11, 5.0.2 | | Browser mode | The provider of your `vitest` version: tested on 3.2.7, 4.0.18, 4.1.11 and 5.0.2 | | Node.js | 22.12 and later (the `engines` field) | Vitest 3.2 is the first version with test annotations, which the reporter reads, hence the peer range. Browser mode has its own setup per Vitest line ([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)). `0.1.0` is the first version of the reporter: there is nothing to migrate from yet, and coming from `probara import junit` on Vitest's JUnit output, the keys stay the same. ### From Vitest 4 to 5 **Some `test.each` and `test.for` titles change**, and with them their keys. Vitest 5 formats the values it puts in a title in another way: | The test | Vitest 3 and 4 | Vitest 5 | | --------------------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------ | | `test.each([{ name: 'Ana' }])('greets $name')` | `greets 'Ana'` | `greets Ana` | | `test.each([{ note }])('saves $note')`, a note of 48 characters | `saves 'Leave the parcel with the neighbour a…'` | `saves Leave the parcel with the neighbour at …` | | `test.each([[{ note }]])('stores %o')` | `stores { note: 'Leave the parcel with the neighbour at number 12' }` | `stores { …(1) }` | | `test.each([[new Map([['eur', 12]])]])('reads %o')` | `reads Map{ 'eur' => 12 }` | `reads Map { 'eur' => 12 }` | | Numbers, short objects and arrays, `%s` of a string | The same | The same | The keys of those rows, as Vitest 5 names them: ```ts // tests/rows.test.ts import { test } from 'vitest'; const note = 'Leave the parcel with the neighbour at number 12'; test.each([{ name: 'Ana' }])('greets $name', () => {}); test.each([[{ note }]])('stores %o', () => {}); ``` ```json [ { "automationKey": "tests/rows.test.ts > greets Ana" }, { "automationKey": "tests/rows.test.ts > stores { …(1) }" } ] ``` - **A changed key is a new case.** The old case keeps its history, and the new key's results go to a new case. Compare the keys before you upgrade, as above: each changed line is such a row. - **Rows can collide.** `{ …(1) }` stands for any object of one key that is too long to print, so two such rows get the same title and share one case. Put a short value of each row in its title (`'stores $id'`). - **`probara import junit` changes the same way**: it reads the names Vitest 5 writes in its JUnit output, so both paths keep giving a test the same key. - **The default folders move**: `attachmentsDir` from `.vitest-attachments` to `.vitest/attachments` ([where the copies are kept](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#where-the-copies-are-kept)), and the blobs of a sharded job from `.vitest-reports` to `.vitest/blob`, which `vitest --merge-reports` reads by default. A CI job that carries them, or cleans them, needs the new paths ([carry the blobs and the attachments](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#carry-the-blobs-and-the-attachments)). ### From Vitest 3 to 4 - **Browser mode** has a new provider package, `@vitest/browser-playwright`, with `provider: playwright()` ([browser mode](https://docs.probara.net/en/developers/ci-integrations/vitest/browser-mode/)). - **Vitest tags** (4.1 and later) join the tags of the cases a report creates; Vitest refuses a tag its config does not declare ([Vitest tags](https://docs.probara.net/en/developers/ci-integrations/vitest/metadata/#vitest-tags)). - **Text bodies of `context.annotate()`** need `bodyEncoding: 'utf-8'` from Vitest 4.1.3 on: Vitest labels a body base64 otherwise ([Vitest annotations](https://docs.probara.net/en/developers/ci-integrations/vitest/steps-and-attachments/#vitest-annotations)). ### Any upgrade of a sharded pipeline `vitest --merge-reports` reads only the blobs of its own Vitest version: upgrade the shards and the merge job together, in one change of the pipeline ([the same Vitest everywhere](https://docs.probara.net/en/developers/ci-integrations/vitest/sharding/#the-same-vitest-everywhere)). Related: [Get started](https://docs.probara.net/en/developers/ci-integrations/vitest/) · [Linking](https://docs.probara.net/en/developers/ci-integrations/vitest/linking/) · [CLI](https://docs.probara.net/en/developers/cli/) · Source: [migrating-from-junit.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/migrating-from-junit.md), [migrating-from-qase.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/migrating-from-qase.md), [coming-from-other-tools.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/coming-from-other-tools.md), [upgrade.md](https://github.com/cynch-me/probara-sdk/blob/main/packages/vitest-reporter/docs/upgrade.md)