Migrating to the Vitest reporter

Whatever you come from, the reporter is one entry in reporters. Nothing goes in setupFiles or runner: the reporter adds its own setup file and runner to each Vitest project, in Node and in browser mode.

// vitest.config.mts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: ['default', ['@probara/vitest-reporter', { projectId: 'SHOP' }]],
  },
});

The reporter sends what the Vitest integrations of Qase, Allure, ReportPortal and TestRail send, wherever Probara has a place for it: case links, the title, suite, fields, tags and steps of a new case, parameters, comments, nested steps, files, links, and the run’s name, plan, environment and configurations. It also sends every attempt of a retried or repeated test, which none of them does, and keeps one run for a whole watch session. What it does not port is under not supported, each with its reason.

You come from	Jump to
Vitest’s JUnit output and probara import junit	The JUnit import
vitest-qase-reporter	Qase
allure-vitest	Allure
@reportportal/agent-js-vitest	ReportPortal
Vitest’s JUnit output and trcli parse_junit	TestRail

From the JUnit import

If your pipeline writes Vitest’s JUnit output and sends it with probara import junit, the reporter can replace both steps without unlinking a single case: it gives each test the automation key and the case the import gives it, on Vitest 3, 4 and 5. Then each test sends what the XML cannot hold: every attempt, steps, files, parameters and the probara.* calls. The reporter is the recommended path; keep the import for a job that only has the XML.

1. Add the reporter to the Vitest config (get started). There is nothing else to set: Vitest’s JUnit output always names the test file, as the reporter’s default keys do (keyIncludesFile: true).
2. Check the keys of both on the same tests: they must be equal.
3. In CI, drop --reporter=junit and the import step, and give the test step PROBARA_API_TOKEN (in CI).

Vitest’s JUnit output	Reporter setting
--reporter=junit, with or without addFileAttribute: true	Nothing: keyIncludesFile: true is the default
Written with another ancestorSeparator (Vitest 5)	Different keys: link the tests by case ids (linking)

Run vitest from the same root as before: the file of each key is relative to it, on both paths. keyIncludesFile: false has no JUnit counterpart: it gives the keys of cases made by hand or by another tool, without the file (without the file).

Check the keys before you switch

Run the suite once with Vitest’s JUnit output and once with the reporter’s results file, reporting off, and compare the dry runs of both: each prints one line per result, with its status, its case and its key. No token is needed. For this test file:

tests/refund.test.ts

import { describe, expect, test } from 'vitest';

describe('refund', () => {
  test('SHOP-12 refunds an order', () => {
    expect(25 - 25).toBe(0);
  });

  test('keeps the receipt', () => {
    expect('receipt').toHaveLength(7);
  });

  test.todo('refunds a gift card');
});

$ npx vitest run --reporter=junit --outputFile=reports/vitest.xml
$ PROBARA_PROJECT=SHOP npx @probara/cli import junit reports/vitest.xml --dry-run
passed  SHOP-12  tests/refund.test.ts > refund > refunds an order
passed  -  tests/refund.test.ts > refund > keeps the receipt
skipped  -  tests/refund.test.ts > refund > refunds a gift card
Total: 3 results from 1 file (2 passed, 0 failed, 1 skipped, 0 blocked)
$ PROBARA_ENABLED=false PROBARA_PROJECT=SHOP PROBARA_RESULTS_FILE=probara-results.json npx vitest run
[probara] Wrote 3 results to /work/shop/probara-results.json: send them with probara import results /work/shop/probara-results.json
$ npx @probara/cli import results probara-results.json --dry-run
passed  SHOP-12  tests/refund.test.ts > refund > refunds an order
passed  -  tests/refund.test.ts > refund > keeps the receipt
skipped  -  tests/refund.test.ts > refund > refunds a gift card
Total: 3 results from 1 file (2 passed, 0 failed, 1 skipped, 0 blocked)

The lines are the same: the same keys, cases and statuses. --reporter=junit replaces the config’s reporters for that run, so the reporter sends nothing then. On a real suite, write each dry run to a file and diff them: a line whose key changed is a test that would get a new case (check what would be sent). Delete reports/ and the results file afterwards.

In CI

Before, two steps: Vitest writes the report, and the CLI sends it.

- name: Run the tests
  run: npx vitest run --reporter=junit --outputFile=reports/vitest.xml
- name: Send the results
  if: ${{ !cancelled() }}
  run: npx @probara/cli import junit reports/vitest.xml
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: SHOP

After, one step: the reporter, registered in the Vitest config, sends every result when the run ends.

- name: Run the tests
  run: npx vitest run
  env:
    PROBARA_API_TOKEN: ${{ secrets.PROBARA_API_TOKEN }}
    PROBARA_PROJECT: SHOP

Remove --reporter=junit from the command: it replaces the reporters of the config, the reporter’s included. When another tool still reads the XML (a test summary of the CI), list junit beside the reporter instead:

reporters: ['default', 'junit', '@probara/vitest-reporter'],
outputFile: { junit: 'reports/vitest.xml' },

The run is named, sourced and closed as the import did it (the run).

What changes from the JUnit import

The keys stay; what each test sends grows, and a few results become more exact:

Topic	probara import junit on Vitest’s JUnit output	The reporter
Retries (retry)	One result per test: a pass on a retry is a plain pass; a test that fails every attempt keeps its first failure	Every attempt, in order, each with its own errors; the run case keeps the last outcome (retries)
Repeats (repeats)	One result per test	Every execution (repeats)
test.todo	skipped	skipped, with the note Todo (statuses)
context.skip('why')	skipped: Vitest writes no reason	skipped, with the note Skipped: why
expect.soft failures	The first failure’s message	Every failure of the attempt, in the notes
Console output	--attach-output: the test’s <system-out> and <system-err>	captureOutput: stdout.log and stderr.log for each attempt (console output)
Vitest projects	No project: Vitest’s JUnit output does not name it	A project parameter on each result, when the config has more than one (Vitest projects)
Unhandled errors (Vitest 5)	Left out, with one warning per file; --fail-on-failed-tests exits 3 for them	Left out, with one warning; the exit code stays Vitest’s (errors outside a test)
When the results go	After the whole run, in a later step	At the end of the run, from the test step; a results file keeps what could not be sent

The same on both paths, so no case moves:

• The statuses of passed, failed and skipped tests, test.fails included, and the tests .only or -t leave out, which are skipped on both.
• The failed results of a beforeAll or afterAll that throws, and of a file that fails to load, named as Vitest’s JUnit writer names them (errors outside a test).
• Case ids in test and describe titles, the names of test.each and test.for rows, and a > with a space on each side inside a title (names as Vitest writes them).

What the import cannot know

The XML holds each test’s name, outcome, time and output. What the tests tell the reporter, and what Vitest gives it beyond its JUnit output, never reaches the import:

• Every attempt of a retried or repeated test, each with its status, duration and errors.
• Case ids from probara.id(), which live in the test’s own metadata, not in its title (linking).
• Steps, nested, with their status and duration, and the steps of a new case (steps).
• Files: probara.attach(), the files of Vitest’s own context.annotate(), failure screenshots in browser mode (attachments).
• Parameters, comments, links and the fields of a new case: the probara.* helpers (metadata and links).
• Run selection: run only the tests of a Probara run (run selection).

Keep the import where the reporter cannot run: a job that only receives the XML of another job, or a Vitest older than 3.2, which the reporter does not support (the import is tested with Vitest 3.2, 4.1 and 5.0). The import’s side is in the CLI and in Vitest JUnit.

Cases made by an earlier import

probara import junit reads Vitest’s output with its vitest dialect, detected from the report’s root (vitest tests), with no option. Versions of the CLI before that dialect (0.3.0 and earlier) read the same XML as generic JUnit, which gives every test the same key, with two differences:

Result	Generic JUnit (the CLI 0.3.0 and earlier)	The reporter, and the vitest dialect
A failure named after its file, such as a file that fails to load	Keyed by the file alone: tests/broken.test.ts	The file twice: tests/broken.test.ts > tests/broken.test.ts
An unhandled error (Vitest 5)	A failed result of its own	Left out, with a warning

Only those results move: the next failure of such a file creates a case under its new key, and the old case gets no more results. Every other test keeps its case.

From Qase

Moving from vitest-qase-reporter is mostly renaming: the reporter entry, its settings, and the ctx.qase calls, which become probara.* calls without a withQase() wrapper. This section maps each one as Qase’s code (version 1.5.1) has it, which differs from Qase’s documentation in places (documented, but not in the code), then lists what works differently and what is not ported, with the reasons.

1. npm uninstall vitest-qase-reporter && npm i -D @probara/vitest-reporter.
2. In vitest.config.mts, replace ['vitest-qase-reporter', {...}] with ['@probara/vitest-reporter', {...}], translating the options (below). Delete qase.config.json and .qaserc: the reporter reads only its options and PROBARA_* variables. Take vitest-qase-reporter/setup out of setupFiles if you added it: the reporter adds its own setup file.
3. Replace QASE_* variables with PROBARA_* ones in CI: at least PROBARA_API_TOKEN (an app token from the Vitest card, get started) and PROBARA_PROJECT.
4. Import probara from @probara/vitest-reporter, turn each withQase(async ({ qase }) => {...}) back into a plain test function, and translate the calls (below). Case ids become Probara display ids (SHOP-12), not numbers.

Before:

// vitest.config.mts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: [
      'default',
      [
        'vitest-qase-reporter',
        {
          mode: 'testops',
          testops: {
            project: 'SHOP',
            run: { title: 'Nightly', complete: true, tags: ['nightly'] },
            uploadAttachments: true,
          },
        },
      ],
    ],
  },
});

After:

// vitest.config.mts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: [
      'default',
      [
        '@probara/vitest-reporter',
        { projectId: 'SHOP', run: { name: 'Nightly', tags: ['nightly'] }, captureOutput: true },
      ],
    ],
  },
});

captureOutput is optional: it attaches each attempt’s console output, which Qase’s captureLogs never did for Vitest (console output). The run is closed at the end without complete: true, and uploadAttachments is on by default.

Qase options and variables

Qase option (QASE_* variable)	Probara option (PROBARA_* variable)
mode: 'testops' (QASE_MODE)	On whenever a token and a project are set
mode: 'off'	enabled: false (PROBARA_ENABLED=false), or no token
mode: 'report', fallback (QASE_FALLBACK)	resultsFile (PROBARA_RESULTS_FILE) (results file)
report.connections.local.path, .format (QASE_REPORT_CONNECTION_PATH, _FORMAT)	resultsFile, the path of a JSON file
mode: 'testops_multi', testops_multi.*	projectId plus projects and run.ulids (several projects)
debug (QASE_DEBUG)	debug (PROBARA_DEBUG)
environment (QASE_ENVIRONMENT), a slug, ignored when not found	run.environment (PROBARA_ENVIRONMENT), a name, created when missing
rootSuite (QASE_ROOT_SUITE)	suiteUlid (PROBARA_SUITE_ULID): an existing suite, by ULID
captureLogs (QASE_CAPTURE_LOGS): ignored by Qase’s Vitest reporter	captureOutput (PROBARA_CAPTURE_OUTPUT)
statusMapping (QASE_STATUS_MAPPING)	statusMapping (PROBARA_STATUS_MAPPING), without invalid and disabled
logging.console, logging.file (QASE_LOGGING_CONSOLE, QASE_LOGGING_FILE)	Logs go to stderr; the logger option for code
testops.api.token (QASE_TESTOPS_API_TOKEN)	PROBARA_API_TOKEN (apiToken for code)
testops.api.host (QASE_TESTOPS_API_HOST)	baseUrl (PROBARA_BASE_URL), a full URL
testops.api.timeout (QASE_TESTOPS_API_TIMEOUT), in seconds	timeoutMs, in milliseconds, for every request
testops.api.retries, testops.api.retryBackoff	maxRetries, for every request; the backoff is fixed (retries and backoff)
testops.project (QASE_TESTOPS_PROJECT)	projectId (PROBARA_PROJECT)
testops.uploadAttachments (QASE_TESTOPS_UPLOAD_ATTACHMENTS)	uploadAttachments (PROBARA_UPLOAD_ATTACHMENTS)
testops.attachments.concurrency, .timeout	attachmentConcurrency; uploads get at least 120 s (timeouts)
testops.run.id (QASE_TESTOPS_RUN_ID)	run.ulid (PROBARA_RUN_ULID)
testops.run.title, .description, .tags	run.name, run.description, run.tags (PROBARA_RUN_*)
testops.run.complete (QASE_TESTOPS_RUN_COMPLETE)	closeRun (PROBARA_CLOSE_RUN); a created run is closed by default
testops.plan.id (QASE_TESTOPS_PLAN_ID)	run.plan (PROBARA_PLAN): the plan’s display id or name
testops.configurations.values (group=value)	run.configurations (PROBARA_CONFIGURATIONS=Group=Name)
testops.statusFilter (QASE_TESTOPS_STATUS_FILTER)	statusFilter (PROBARA_STATUS_FILTER)
testops.batch.size (QASE_TESTOPS_BATCH_SIZE)	chunkSize (1 to 500)
(none)	run.milestone (PROBARA_MILESTONE): Qase’s reporter has no milestone

Qase settings without a counterpart are in not ported. Every Probara option is on the configuration page.

The ctx.qase calls

Qase	Probara
test(addQaseId('pays', [12]), ...), (Qase ID: 12) in a title	test('SHOP-12 pays', ...): an id in the title or a describe (linking)
addQaseId('pays', [12, 13])	'SHOP-12 SHOP-13 pays', or probara.id(['SHOP-12', 'SHOP-13'])
addQaseProjects('pays', { WEB: [3] }), (Qase WEB: 3) in a title	WEB-3 in the title, with projects: ['WEB'] (several projects)
withQase(async ({ qase }) => {...})	The test function itself: probara finds the running test on its own
await qase.title('...')	probara.title('...')
await qase.suite('Payments - Cards')	probara.suite(['Payments', 'Cards'])
await qase.fields({ severity: 'critical' })	probara.fields({ severity: 'critical' })
await qase.tags('smoke', 'checkout')	probara.tags('smoke', 'checkout')
await qase.comment('...')	probara.comment('...')
await qase.parameters({...}), await qase.groupParameters({...})	probara.parameters({...})
await qase.step('Pay', fn, 'Paid', 'card=visa')	await probara.step('Pay', fn, { expected: 'Paid', data: 'card=visa' })
await qase.attach({ paths: [a, b] })	await probara.attach({ name, path }), once per file
await qase.attach({ name, content, type })	await probara.attach({ name, body, contentType })
await qase.annotate(message, options)	await context.annotate(message, options): Vitest’s own; its file is sent (Vitest annotations)
(none: Qase’s Vitest reporter has no ignore)	probara.ignore()
(none)	probara.link() and probara.issue() (links and issues)

Remove the (Qase ID: 12) markers from the titles: the reporter reads them as text, so they would stay in the automation key.

Before:

tests/checkout.test.ts

import { test } from 'vitest';
import { addQaseId, withQase } from 'vitest-qase-reporter/vitest';

test(
  addQaseId('pays with a card', [12]),
  withQase(async ({ qase }) => {
    await qase.fields({ severity: 'critical' });
    await qase.tags('smoke');
    await qase.step('Pay', async () => {}, 'The order is paid');
    await qase.attach({ name: 'order.json', content: '{"id":1042}', type: 'application/json' });
  }),
);

After:

tests/checkout.test.ts

import { probara } from '@probara/vitest-reporter';
import { test } from 'vitest';

test('SHOP-12 pays with a card', async () => {
  probara.fields({ severity: 'critical' }).tags('smoke');
  await probara.step('Pay', async () => {}, { expected: 'The order is paid' });
  await probara.attach({
    name: 'order.json',
    body: '{"id":1042}',
    contentType: 'application/json',
  });
});

[
  {
    "caseDisplayId": "SHOP-12",
    "automationKey": "tests/checkout.test.ts > pays with a card",
    "status": "passed",
    "steps": [{ "action": "Pay", "expected": "The order is paid", "status": "passed" }],
    "case": {
      "tags": ["smoke"],
      "fields": { "severity": "critical" },
      "steps": [{ "action": "Pay", "expected": "The order is paid" }]
    }
  }
]

The result also carries order.json (application/json). The id leaves the title and the key: adding or removing it never changes which case the test matches (automation keys).

Documented, but not in the code

Qase’s README and guides show calls and settings that version 1.5.1 does not have, or has with another behavior. Code copied from them fails or does something else:

Qase’s documentation	Qase’s code (1.5.1)	With Probara
test(qase(12, 'pays'), ...), with qase imported from vitest-qase-reporter or vitest-qase-reporter/vitest	No qase is exported: only withQase, addQaseId and addQaseProjects	test('SHOP-12 pays', ...)
qase.title(), qase.fields(), qase.suite() on that import, in any test	Only ctx.qase, in a function withQase() wraps	probara.title(), probara.fields(), probara.suite(), in any test
qase.ignore()	Not in the code	probara.ignore()
qase.mute()	Not in the code	Not ported (below)
setupFiles: ['vitest-qase-reporter/setup'], for the network profiler	Not exported by the package: Vitest cannot load it	Nothing to add: the reporter adds its own setup file
qase.suite('Payments\tCards'), a tab between suites	Split only on a dash with a space on each side ('Payments - Cards')	probara.suite(['Payments', 'Cards'])
Steps nested in steps	A step that holds steps is lost; only the inner ones are kept	Nested as they ran (steps)
An attachment of a step	Always the test’s	probara.attach() inside a step (files of a step)
The reporter’s options win over QASE_* variables, which win over qase.config.json	Variables win over the file, which wins over the options	Options, then variables (precedence)
testops.run.complete is true by default	Unset leaves the run open	A created run is closed by default (closing)
report.connection.*	report.connections.*	resultsFile
Watch mode reports when you exit or run every test again	Each re-run creates a new run, unless run.id is set	One run for the whole session (watch mode)
api.token per project of testops_multi	One token for every project	One app token for every project of the organization

What is different from Qase, and why

Topic	Qase	Probara
Precedence	Variables win over qase.config.json, which wins over the reporter options	Options in the Vitest config, then PROBARA_* variables, then defaults: what the config says is what runs
Config file	qase.config.json or .qaserc	None: a config file holding a token could be committed. Options live in the Vitest config
Case ids	Numbers, in the test’s own title only	Display ids (SHOP-12) in the title or any describe: the Project ID routes each id to its project
The helpers	ctx.qase, inside withQase(), until the test body ends: Vitest refuses its annotations in afterEach	probara, imported once, works in the body, beforeEach, afterEach, onTestFinished and the functions they call (where to call them)
Steps	Flat; the expected result and data are encoded into the step name; no duration	Nested as they ran, each with its status, duration and error; the outermost become a new case’s steps (steps)
Statuses	invalid for a failure whose error does not look like an assertion (a guess from the message)	passed, failed, skipped; blocked only through statusMapping. No guessing from error text (statuses)
Errors	The first error’s message, which comment() replaces; every stack in a field of its own	Every error of the attempt in the notes, expect.soft failures included; a comment comes first and hides none
fields() and parameters() calls	The last call replaces the others	Merged by name
Case fields	Sent with every result	Only when the report creates the case: an existing case keeps what people wrote (created cases only)
Retries	The final attempt only	Every attempt is a result, with no flaky flag: the case’s history shows the flakiness (retries)
Closing the run	Only with run.complete: true; unset leaves it open	A run the reporter creates is closed at the end, once everything was sent; a reused one stays open (closing)
Shards	Each shard makes its own run unless it gets a run id; closing a shared run is left to another step	One run: vitest --merge-reports over the shards’ blobs, or probara run create with PROBARA_RUN_ULID (sharding)
When reporting fails	fallback to a local report	A results file, sent later by probara import results into the same runs (results file)
Strings in attach()	A string that looks like base64 is decoded ('Hello' uploads garbage)	A string is sent as it is; pass a Buffer or a Uint8Array for binary content (probara.attach())
attach({ paths })	A list of paths; the content type is type, not contentType	One file per call, path or body, and contentType

Not ported from Qase

Qase feature	Why
Public report link (showPublicReportLink)	A public link to test results is a sharing decision for the Probara app, not something a CI job should turn on
Automatic defects (testops.defect)	Filing a defect for every failure floods the defect list; report defects from the result in Probara
Network profiler (profilers: ['network'], networkProfiler)	Its steps are noise, up to the 200 steps of a result, and cannot be told apart when tests run in parallel; attach what matters with probara.attach() (no network profiler)
Jira link on runs (run.externalLink)	Probara links Jira issues to cases and defects, where traceability lives; probara.issue() links a result to an issue
qase.mute()	Documented by Qase, but not in its code; triage lives in Probara, on the results
Creating configurations (createIfNotExists)	Configurations are part of a project’s planning; an unknown one is refused, so a typo never creates a new one
The invalid status	It is a guess from the error text; Probara keeps four statuses and the whole error in the notes
Logging to a file (logging.file)	Logs go to stderr, where CI keeps them; redirect stderr to keep a file
Per-project plan and environment (testops_multi.projects[].plan, .environment)	Planning entities belong to one project, where an unknown name would refuse the report: create that project’s run first with probara run create and pass it in run.ulids (one run per project)
A config file (qase.config.json, .qaserc)	A file holding a token could be committed: the token stays in the environment, the options in the Vitest config

From Allure

allure-vitest is a reporter that adds its setup file and its runner to each Vitest project, and writes result files into allure-results/; a separate step turns them into a report. The reporter sends the results to Probara instead, and adds its own setup file and runner the same way: swap the entries.

// vitest.config.mts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: [
      'default',
      [
        'allure-vitest/reporter',
        {
          resultsDir: 'allure-results',
          links: { issue: { urlTemplate: 'https://jira.example.com/browse/%s' } },
        },
      ],
    ],
  },
});

Allure settings

Allure	Probara
resultsDir	Nothing: results go to Probara. A results file keeps what could not be sent
links.issue.urlTemplate	issueUrlTemplate (PROBARA_ISSUE_URL_TEMPLATE) (probara.issue())
environmentInfo	run.environment (a name) and the run source, detected from the CI (the run)
globalLabels, ALLURE_LABEL_*	run.tags (PROBARA_RUN_TAGS) for the run; the tags of a new case come from probara.tags()
ALLURE_TESTPLAN_PATH	runCasesOnly with PROBARA_RUN_ULID (run selection)
reportMatchers, categories, listeners	Not ported (not supported)

Allure calls

Allure’s calls are async (await allure.tags('smoke')); Probara’s return at once, except probara.attach() and a probara.step() with an async body, which you await.

Allure (allure-js-commons)	Probara (probara.*)
@allure.id:12 in a title, allureId('12')	SHOP-12 in the title or a describe, or id('SHOP-12')
displayName('...')	title('...')
description('...')	fields({ description: '...' })
severity('critical'), layer('api')	fields({ severity: 'critical', layer: 'api' })
tag('smoke'), tags(...)	tags('smoke')
epic, feature, story, label(name, value)	fields({ Feature: 'Checkout' }): a custom field of that name, or tags(...)
parentSuite, suite, subSuite	suite(['Payments', 'Cards'])
parameter('user', 'ana')	parameters({ user: 'ana' })
step('Pay', async () => {...})	step('Pay', async () => {...})
logStep('Paid')	step('Paid'): a step without a body, passed
attachment(name, content, { contentType })	attach({ name, body: content, contentType })
attachmentPath(name, path)	attach({ name, path })
link(url, name)	link(url, name)
issue('PAY-7')	issue('PAY-7'), with issueUrlTemplate
tms('12')	id('SHOP-12')

reporters: [
  'default',
  ['@probara/vitest-reporter', { issueUrlTemplate: 'https://jira.example.com/browse/%s' }],
],

tests/payments.test.ts

import { probara } from '@probara/vitest-reporter';
import { test } from 'vitest';

test('refunds an order', async () => {
  probara.fields({ severity: 'critical' }).issue('PAY-7');
  await probara.step('Open the order', async () => {});
  await probara.step('Refund it', async () => {});
});

[
  {
    "automationKey": "tests/payments.test.ts > refunds an order",
    "steps": [
      { "action": "Open the order", "status": "passed" },
      { "action": "Refund it", "status": "passed" }
    ],
    "links": [{ "url": "https://jira.example.com/browse/PAY-7", "name": "PAY-7" }],
    "case": {
      "fields": { "severity": "critical" },
      "steps": [{ "action": "Open the order" }, { "action": "Refund it" }]
    }
  }
]

What works differently from Allure

Topic	Allure	Probara
Statuses	failed for an assertion, broken for any other error	failed for both, with the error in the notes: no guess from the error’s type (statuses)
test.skip, a skipped describe	Not reported	Skipped results, as Vitest’s JUnit output has them
Assertions	Each expect is a step (reportMatchers, on by default)	The steps are the probara.step() calls
A call where no test runs	A call in beforeAll or a describe body goes to every test of the suite	Left out, with one warning: no single test runs there (where to call the helpers)
A hook that throws	Its error is a global error of the report, beside the tests	Vitest’s verdict: a failing beforeEach or afterEach fails its test, a failing beforeAll or afterAll is a failed result for its describe or file (errors outside a test)
Retries	One result per test, the final attempt	Every attempt is a result of the run (retries)
Watch mode	New result files on every re-run, which pile up in resultsDir	One run for the whole session (watch mode)
Case fields	Labels on every result	Fields and tags of the case a report creates; an existing case keeps its own (created cases only)
test.concurrent in browser mode	A call can go to the wrong test after an await (its README says so)	A call is dropped with a warning, never recorded in another test (browser mode)

From ReportPortal

@reportportal/agent-js-vitest is a reporter object in reporters, with a setup file you list in setupFiles for its ReportingApi. Replace both with the reporter entry above:

// vitest.config.mts
import RPReporter from '@reportportal/agent-js-vitest';
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    setupFiles: ['@reportportal/agent-js-vitest/setup'],
    reporters: [
      'default',
      new RPReporter({
        apiKey: process.env.RP_API_KEY,
        endpoint: 'https://rp.example.com/api/v2',
        project: 'shop',
        launch: 'Nightly',
      }),
    ],
  },
});

ReportPortal settings

ReportPortal (variable)	Probara (variable)
apiKey, oauth	PROBARA_API_TOKEN: an app token from the Vitest card (get started)
endpoint	baseUrl (PROBARA_BASE_URL), the app’s URL without a path
project	projectId (PROBARA_PROJECT)
launch	run.name (PROBARA_RUN_NAME)
description	run.description (PROBARA_RUN_DESCRIPTION)
attributes ([{ key: 'build', value: '42' }])	run.tags (PROBARA_RUN_TAGS=build:42,nightly): tags are plain names, a colon included
launchId (RP_LAUNCH_ID)	run.ulid (PROBARA_RUN_ULID)
launchUuidPrint	probara run create prints the ULID every shard reports into (an existing run)
restClientConfig (timeout, retries, proxy)	timeoutMs, maxRetries; a proxy through Node’s variables (network)
headers	A fetch of your own (your own HTTP client)
debug	debug (PROBARA_DEBUG)
extendTestDescriptionWithLastError	Nothing to set: every error of an attempt is in its notes
skippedIssue, rerun, rerunOf, mode: 'DEBUG'	Not ported (not supported)

ReportPortal calls

ReportPortal (ReportingApi)	Probara (probara.*)
ReportingApi.attachment(task, { name, type, content })	attach({ name, body, contentType }), with a Buffer or text, or attach({ name, path })
ReportingApi.description(task, text)	fields({ description: text }), for a new case
ReportingApi.attributes(task, [{ key, value }])	parameters({ [key]: value }) on the result, or tags(value) on a new case
ReportingApi.testCaseId(task, id)	Not ported: link a case by its id instead (linking)
ReportingApi.log(task, message, level)	Not ported: comment() for a line in the notes, captureOutput for the console
console.log(), console.error(), as logs	captureOutput: stdout.log and stderr.log of each attempt (console output)
The global ReportingApi of the setup file	probara, imported: there is no task to pass

A base64 string as content would be sent as text: give attach() the file’s path, or its bytes as a Buffer.

tests/checkout.test.ts

import { probara } from '@probara/vitest-reporter';
import { test } from 'vitest';

test('pays with a card', async () => {
  probara.fields({ description: 'Pays with a saved card' }).parameters({ browser: 'chromium' });
  await probara.attach({
    name: 'response',
    body: '{"status":"paid"}',
    contentType: 'application/json',
  });
});

[
  {
    "automationKey": "tests/checkout.test.ts > pays with a card",
    "parameters": { "browser": "chromium" },
    "case": { "description": "Pays with a saved card" }
  }
]

The result also carries response.json (application/json).

What works differently from ReportPortal

Topic	ReportPortal	Probara
In-test API	ReportingApi, global from its setup file, or with the test’s task passed in	probara, imported; each call belongs to the test that made it (metadata)
test.concurrent	The global ReportingApi follows the last test that started, so a call can go to another test	Each call goes to its own test
Retries	The final attempt only	Every attempt is a result (retries)
test.todo	Skipped, with the attribute todo	Skipped, with the note Todo
A skipped describe	One skipped suite item; its tests are not reported	Each test is a skipped result
Errors	The first error, as a log and in the description	Every error of the attempt, in the notes of its result
Structure	A suite item per describe	Each new case in a suite named after its file (missing cases)
Watch mode	The launch ends with the first run, and the re-runs have none	One run for the whole session (watch mode)

From TestRail

TestRail has no Vitest reporter: a pipeline writes Vitest’s JUnit output, then TestRail’s Python CLI trcli parse_junit sends it after the run. The reporter replaces both steps, and sends what the XML cannot hold: steps, files, comments and every attempt.

npx vitest run --reporter=junit --outputFile=reports/vitest.xml
trcli -h https://example.testrail.io --project "Shop" -u ci@example.com -k "$TESTRAIL_KEY" \
  parse_junit --title "Nightly" --case-matcher name -f reports/vitest.xml --close-run

With the reporter, the test step is enough:

npx vitest run

trcli settings

trcli	Probara
-h (host), -u, -p, -k	baseUrl for a self-hosted Probara; PROBARA_API_TOKEN
--project (a name)	projectId (PROBARA_PROJECT), the Project ID
--title, --run-description	run.name, run.description
--run-id	run.ulid (PROBARA_RUN_ULID)
--plan-id, --config-ids, --milestone-id	run.plan, run.configurations, run.milestone, by display id or name (the run)
--close-run	The default for a run the reporter creates; closeRun for a reused one
--case-matcher name (C123 in the name)	SHOP-12 in the title or a describe (linking)
--case-matcher auto (classname.name)	The automation key: the file, the describes and the title
-y (create sections and cases)	createMissingCases, on by default; suites from the test file
--suite-id, --section-id	suiteUlid (PROBARA_SUITE_ULID): the suite created cases go under
--case-fields type_id:3	probara.fields({ type: 'Regression' }) in the test, by name (metadata)
-a, --assign (emails)	assignFailedTo (PROBARA_ASSIGN_FAILED_TO) (assign failed results)
-b, --batch-size	chunkSize (1 to 500)
-t, --timeout (seconds)	timeoutMs (milliseconds, for each attempt of a request)
--proxy, --noproxy	HTTPS_PROXY and NO_PROXY with NODE_USE_ENV_PROXY=1 (proxies)
-v (verbose)	debug (PROBARA_DEBUG)
testrail_result_comment property	probara.comment()
testrail_attachment property	probara.attach()
testrail_result_step property	probara.step()

What works differently from trcli

Topic	trcli on Vitest’s JUnit output	Probara
Skipped tests	Retest	Skipped
test.todo	Retest: Vitest writes it as skipped	Skipped, with the note Todo
Retries	One result per test	Every attempt (retries)
Nested describes	One section per test file, flat: Vitest writes a suite per file	The suite of a created case is its file; probara.suite() names nested suites
Durations	Rounded to seconds (--allow-ms for milliseconds)	Milliseconds, and the start of each attempt
An unknown case id	Aborts the upload	That result is not recorded, and the log names it; the others are
A failed upload	Rolls back the run, cases and sections it created	Nothing is rolled back; a results file keeps what was not sent
Shards	One upload of every shard’s file, with a glob	vitest --merge-reports over the blobs, or probara run create, PROBARA_RUN_ULID in every shard, probara run close (sharding)

Not supported

Each of these is left out, with the reason:

Feature	In	Why
Updating existing cases	trcli (--update-existing-cases, --update-strategy)	A report never changes an existing case: people own what is written there (created cases only)
A flaky flag	Vitest (diagnostic().flaky)	Every attempt is a result: a case whose history mixes failures and passes on one run is flaky, without a flag (retries)
Assertions as steps	Allure (reportMatchers)	Each expect would be a step, up to the 200 steps of a result, crowding out the steps that say what the test does: wrap what matters in probara.step()
The broken and invalid statuses	Allure (broken), Qase (invalid)	Both are a guess from the error’s type or text; Probara keeps four statuses (passed, failed, skipped, blocked) and the whole error in the notes
Masked, hidden and excluded parameters	Allure (the modes of parameter())	Parameters are shown with the result and never part of the key: leave a secret out rather than masking it
Log levels and a log API	ReportPortal (ReportingApi.log()), Allure (logStep() with a status)	Probara has no log entity: captureOutput keeps each attempt’s console output, probara.comment() writes a line in the notes, and a step’s status is what its body did
Defects, Jira links on runs, public report links	Qase (testops.defect, run.externalLink, showPublicReportLink)	Defects and their triage live in Probara, on the results; probara.issue() links a result to an issue; a public link to results is a sharing decision, not a CI setting
A network profiler	Qase (profilers: ['network'])	Its steps are noise, up to the 200 steps of a result, and cannot be told apart when tests run in parallel (no network profiler)
Categories, known issues, muted tests	Allure (categories), ReportPortal (skippedIssue), Qase (qase.mute(), documented only)	Triage lives in Probara, on the results and defects, not in a CI setting
Overriding the automation key	Allure (historyId(), testCaseId()), ReportPortal (testCaseId()), trcli (custom_automation_id)	The key comes from the file and the names, the same as probara import junit’s: link a case by its id instead (automation keys)
An owner	Allure (owner())	A Probara case has no owner field: assign failed results instead (assign failed results)
Result fields	trcli (--result-fields, testrail_result_field)	Probara results have no custom fields: parameters (probara.parameters()) and links (probara.link()) carry the extra data
Steps and calls from suite hooks	Allure (a call in beforeAll goes to every test of the suite)	A call where no single test runs belongs to no test: it is left out with a warning, not copied into each one. Make the steps that matter in the test or its beforeEach; a hook that throws still shows (errors outside a test)
A config file	Qase (qase.config.json, .qaserc), trcli (config.yml)	A file holding a token could be committed: the token stays in the environment, the options in the Vitest config (configuration)
Benchmarks and coverage	Vitest (vitest bench, --coverage)	Probara has no benchmark or coverage results: the reporter sends test results
Type tests	Vitest (vitest typecheck)	No test of the reporter covers them, so nothing is promised for them
Rerun and debug launches	ReportPortal (rerun, rerunOf, mode: 'DEBUG')	Report again into the same run with run.ulid: each run case keeps the last outcome and its history. To try a setup, check what would be sent
Signing in with OAuth	ReportPortal (oauth)	CI reports with an app token of the organization, which uses no seat and outlives the person who made it
Global files and errors	Allure (globalAttachment(), globalError())	Files and errors belong to a result: attach them from the test that produced them
HTML descriptions	Allure (descriptionHtml())	The description is text; Probara formats it
Listeners	Allure (listeners)	No hook changes a result before it is sent; statusMapping and statusFilter cover statuses (status mapping)
Turning TLS verification off	trcli (--insecure)	It cannot be turned off: give Node your certificate authority instead (custom certificate authorities)
Rolling back a failed upload	trcli	A result recorded is kept: rolling back would lose the ones that went through
References on runs	trcli (--test-run-ref)	The run records its CI build URL, branch and commit; issues link to results with probara.issue()

Upgrading the reporter

@probara/vitest-reporter follows semantic versioning. Read the changelog before you upgrade it. A new Vitest major can change the names it gives tests, and so their automation keys: before you upgrade Vitest, read from Vitest 4 to 5 and compare the keys.

• 0.x (now): the reporter is young. A minor version (0.2.0) may change options, log lines or defaults; a patch version (0.1.1) only fixes bugs. Pin the version in package.json (or keep the lockfile) and upgrade on purpose.
• 1.0 and later: breaking changes only in a major version.
• @probara/vitest-reporter depends on @probara/core of the same release line; install the reporter alone and let it bring its core.

What is a breaking change

Any of these needs a major version from 1.0 on, and a minor one before:

• An option or a PROBARA_* variable removed or renamed, or its meaning changed.
• A default changed (closing runs, creating missing cases, keyIncludesFile, what gets attached).
• A probara.* helper removed, or its behavior changed.
• A status an attempt is sent with changed for the same attempt.
• Any change to the automation key a test gets, with keyIncludesFile on or off. A different key unlinks the test from its case: the next run creates a new one.
• Dropping a Vitest or Node.js version that is still supported upstream.

Adding an option, a helper, a log line or a file uploaded is not breaking. Log lines on stderr are for people: do not parse them.

The automation key contract

The key algorithm is automation key v1, frozen in @probara/core and pinned by golden vectors: no version of the reporter changes it. What the reporter adds is how a Vitest test becomes the key’s parts, the way probara import junit reads Vitest’s JUnit output (keys):

• the test file relative to rootDir (the Vitest root), when keyIncludesFile is on;
• the describe titles and the test title, each a part, as Vitest’s JUnit output names them, the case ids removed;
• a test.each or test.for row by the title Vitest formats for it.

That mapping is part of the contract too: a version of the reporter that changes the key of a test is a breaking change, named as such in the changelog. A version of Vitest that formats titles differently changes keys as well (below), for the reporter and for probara import junit alike.

To check an upgrade of either, compare what both versions would send: run the suite with reporting off once per version, each with its own results file, and diff the dry runs (check what would be sent). Any changed key is a test that would get a new case.

Vitest and Node.js versions

Needs	Supported
vitest	3.2 and later (the peer range ^3.2.0 || ^4.0.0 || ^5.0.0): tested on 3.2.7, 4.1.11, 5.0.2
Browser mode	The provider of your vitest version: tested on 3.2.7, 4.0.18, 4.1.11 and 5.0.2
Node.js	22.12 and later (the engines field)

Vitest 3.2 is the first version with test annotations, which the reporter reads, hence the peer range. Browser mode has its own setup per Vitest line (browser mode). 0.1.0 is the first version of the reporter: there is nothing to migrate from yet, and coming from probara import junit on Vitest’s JUnit output, the keys stay the same.

From Vitest 4 to 5

Some test.each and test.for titles change, and with them their keys. Vitest 5 formats the values it puts in a title in another way:

The test	Vitest 3 and 4	Vitest 5
test.each([{ name: 'Ana' }])('greets $name')	greets 'Ana'	greets Ana
test.each([{ note }])('saves $note'), a note of 48 characters	saves 'Leave the parcel with the neighbour a…'	saves Leave the parcel with the neighbour at …
test.each([[{ note }]])('stores %o')	stores { note: 'Leave the parcel with the neighbour at number 12' }	stores { …(1) }
test.each([[new Map([['eur', 12]])]])('reads %o')	reads Map{ 'eur' => 12 }	reads Map { 'eur' => 12 }
Numbers, short objects and arrays, %s of a string	The same	The same

The keys of those rows, as Vitest 5 names them:

tests/rows.test.ts

import { test } from 'vitest';

const note = 'Leave the parcel with the neighbour at number 12';

test.each([{ name: 'Ana' }])('greets $name', () => {});

test.each([[{ note }]])('stores %o', () => {});

[
  { "automationKey": "tests/rows.test.ts > greets Ana" },
  { "automationKey": "tests/rows.test.ts > stores { …(1) }" }
]

• A changed key is a new case. The old case keeps its history, and the new key’s results go to a new case. Compare the keys before you upgrade, as above: each changed line is such a row.
• Rows can collide. { …(1) } stands for any object of one key that is too long to print, so two such rows get the same title and share one case. Put a short value of each row in its title ('stores $id').
• probara import junit changes the same way: it reads the names Vitest 5 writes in its JUnit output, so both paths keep giving a test the same key.
• The default folders move: attachmentsDir from .vitest-attachments to .vitest/attachments (where the copies are kept), and the blobs of a sharded job from .vitest-reports to .vitest/blob, which vitest --merge-reports reads by default. A CI job that carries them, or cleans them, needs the new paths (carry the blobs and the attachments).

From Vitest 3 to 4

• Browser mode has a new provider package, @vitest/browser-playwright, with provider: playwright() (browser mode).
• Vitest tags (4.1 and later) join the tags of the cases a report creates; Vitest refuses a tag its config does not declare (Vitest tags).
• Text bodies of context.annotate() need bodyEncoding: 'utf-8' from Vitest 4.1.3 on: Vitest labels a body base64 otherwise (Vitest annotations).

Any upgrade of a sharded pipeline

vitest --merge-reports reads only the blobs of its own Vitest version: upgrade the shards and the merge job together, in one change of the pipeline (the same Vitest everywhere).

Related: Get started · Linking · CLI · Source: migrating-from-junit.md, migrating-from-qase.md, coming-from-other-tools.md, upgrade.md