Cypress steps and attachments

A result can carry steps and files. Cypress has no steps of its own: write them with probara.step(). Cypress takes a screenshot of each failure by itself, and the reporter attaches it to the attempt it belongs to; probara.attach() is for everything else.

`probara.step()`

describe('Cart', () => {
  it('checks out', () => {
    probara.step(
      'Open the cart',
      () => {
        cy.get('.cart').click();
        probara.step('The cart holds two items', () => {
          cy.get('.cart-item').should('have.length', 2);
        });
      },
      { expected: 'The cart shows two items' },
    );

    probara.step('Pay by card', () => {
      cy.get('.pay').click();
    });
  });
});

Argument	What it is
`title`	What the step did. It is the `action` of the step in Probara.
`body()`	The Cypress commands of the step. Without it, a step that passed.
`options.expected`	The expected result, for the case the report creates.
`options.data`	The data the step uses, for the case the report creates.

The steps show under the result, nested as they ran, each with its status, duration and error. The outermost steps are also the steps of the case the report creates, with what expected and data declare; the ones nested in them belong to their step only. An existing case keeps its own steps.

What a Cypress step is

It returns nothing. probara.step() is not a promise: its body queues Cypress commands, and the step ends once Cypress has run them. It never yields the result of its body.
Nesting is by call order. A probara.step() called inside the body of another is its child, and a file attached inside a body belongs to that step. A body that awaits between its commands interleaves them with the steps after it: write a body without awaiting, or split it into two steps.
A command that fails fails the step. An assertion inside a step, or a cy.get() of something that is not there, is reported as a failed step, with the error the step had not finished when the test ended; the result carries the error of the command.
A body that throws fails the test. Cypress fails a test whose body throws before it runs the commands that body queued: the test is reported failed with the error, and the step itself is not.

Commands are not steps

Cypress commands are not turned into steps: a step is what a test says it did with probara.step(), not every command its framework queued. Every click and assertion as a step would multiply the steps of a result, and Probara keeps 200 per result.

`probara.attach()`

it('writes the cart as CSV', () => {
  probara.attach({ name: 'cart.csv', body: 'sku,qty\nA-1,2\n' });
  probara.attach({ name: 'report.png', path: 'cypress/fixtures/report.png' });
});

Argument	What it is
`name`	The name the file shows under in Probara. Required.
`body`	The content: a string, a `Uint8Array` or an `ArrayBuffer`.
`path`	A file of the project, read by the plugin.
`contentType`	The type of the file; without one it is taken from the extension of its name.

A body is attached as it is; bytes are converted by the browser and written by the plugin. The call returns nothing.
A path is read from the project root, as cy.readFile() reads one, not from the folder of the spec. The plugin reads it, so a missing file never fails the test: it is left out, with one warning per path. The call returns the chain of the cy.task() that hands the path over: wait for it when the test needs the file attached before it goes on.

Files of a step

A file attached inside a probara.step() belongs to that step; any other belongs to the result, beside the steps. A name with an extension keeps it; a name without one takes the extension of the file it came from.

Screenshots

Cypress takes a screenshot of a failed attempt by itself, and the reporter attaches it to the result of that attempt, as an image/png. Nothing is matched by searching a folder: the plugin receives each screenshot from Cypress with its exact path, and the reporter reads the name Cypress gave it:

What Cypress names it	What it is
`<titles joined by ' -- '> (failed)`	The first failed attempt of a test
`<titles joined by ' -- '> (failed) (attempt N)`	A retry, from the second attempt on
`<titles joined by ' -- '> -- <hook> hook (failed) (…)`	A failing `beforeEach` or `before`, named after the hook and the test it was running

So each screenshot lands on the attempt it was taken for, and a test that fails twice has two.

Setting	What happens
`attachScreenshots: true` (the default)	The screenshot of each failed attempt is attached
`attachScreenshots: false`	None is attached; Cypress still takes them where it always does
No plugin (`setupNodeEvents`)	None is attached: only the plugin receives the screenshots

Not attached, on purpose:

A screenshot you took yourself, such as cy.screenshot('my-name'): its name matches no failed attempt, so it is left out and named once at PROBARA_DEBUG. To keep it, attach it with probara.attach().
A screenshot of an attempt that did not fail: only failures are attached.

The video of a spec

With video: true in the Cypress config and attachVideos: true in the reporter options, the video of a spec is attached to every failed result of that spec (a blocked one too), under the name of the spec file (cart.cy.js.mp4), as a video/mp4. A spec with no failed result sends no video. It needs the plugin, which is what receives the video from Cypress.

attachVideos is off by default because of the cost: one upload per failed result of the spec, and video files are large.

Console output

captureOutput: true (PROBARA_CAPTURE_OUTPUT, off by default) attaches what each test writes to the browser console, as two files of its result:

Stream	What goes into it	Attached as
stdout	`console.log`, `console.info`, `console.debug`	`stdout.log` (`text/plain`)
stderr	`console.warn`, `console.error`	`stderr.log` (`text/plain`)

The browser console keeps printing everything, captured or not. Each stream is cut at 32 MiB, and the attachment then ends with:

[probara] The output of this test was cut here: an attachment holds at most 32 MiB

What is not captured:

Any other console method (console.table, console.dir, console.group, console.count, console.time).
A console method the test replaced.
Anything written outside a test.
Everything, without the support file: the capture is installed by that file.

Limits and what is skipped

Probara holds every attachment to these caps; the reporter skips a file rather than fail a result, and names each skipped file in the run’s log:

Cap	What happens
20 files per result	More than 20 attachments in one result are left out
32 MiB per file	The captured console output is cut at this size, with the line above
10 MiB per image	A larger image is left out
8192 px per side of an image	A larger image is left out (a full-page screenshot of a long page easily is)
Executables and scripts by content type	A refused content type (`application/x-msdownload`, `application/x-sh`, `application/x-bat`, …) is left out
255 characters in a stored file name	A longer name is truncated
200 steps per result, 10 levels deep	What the steps of a result are held to

uploadAttachments: false (PROBARA_UPLOAD_ATTACHMENTS=false) uploads no file at all: steps, links and parameters are still sent.

When steps or files are missing

What you see	Why
No step, no file, no `probara.*` effect at all	The plugin or the support file is missing (registration)
Results, but no screenshot	No plugin, `attachScreenshots: false`, or the screenshot was your own
`probara.attach() could not read "<path>" (<reason>): it is not attached`	The file of a `path` is missing or unreadable
`probara.attach() takes { name, path } or { name, body }`	A wrong argument; the call is ignored and the test goes on
`probara.step() takes a function as its body`	The body was not a function
`Skipped an attachment: <reason>`	A file over a limit

Every other warning is in troubleshooting.

Related: Metadata and links · Configuration · Troubleshooting · Source: steps.md, attachments.md, specs.md