Skip to content

Custom reporters can't access per-attempt details for retries #10303

Description

@aslushnikov

Clear and concise description of the problem

Custom reporters receive a single TestCase result per test, even when the test ran multiple times via retry. There's no way to access the individual attempts — their status, duration, errors, annotations, or stdio.

Repro: a test with retry: 3 that fails twice then passes — the reporter only sees one result with state: 'passed'.

Suggested solution

It would be useful for Vitest's test result to expose an attempts array with per-attempt details of execution: duration, errors, annotations, stdio, etc.

Alternative

No response

Additional context

  1. This was discovered while building @flakiness/vitest
  2. For reference, Playwright Test exposes this in its reporter API: each retry comes through as a separate TestResult with its own status, duration, errors, attachments, and stdio.

Validations

Activity

  1. AriPerkkio commented on May 9, 2026

    @AriPerkkio
    Member

    There's no way to access the individual attempts — their status, duration, errors, annotations, or stdio.

    Errors are present in errors:

    /**
    * Errors that were thrown during the test execution.
    *
    * **Note**: If test was retried successfully, errors will still be reported.
    */
    readonly errors: ReadonlyArray<TestError> | undefined

    But I don't think other metrics are stored.

    the reporter only sees one result with state: 'passed'.

    From diagnostic().retryCount you can see how many times test was retried. So it kind of indicates there were 2x state: 'failed' before the last result.

  2. aslushnikov commented on May 9, 2026

    @aslushnikov
    Author

    From diagnostic().retryCount you can see how many times test was retried. So it kind of indicates there were 2x state: 'failed' before the last result.

    Indeed, that's precisely what we do:

    https://github.com/flakiness/vitest/blob/c67b1196918a1aed10df1145b6c710732bd48f67/src/reporter.ts#L259-L282

    Errors are present in errors:

    While they are stored there, it's hard to assign them to individual attempts since each attempt might have more than one error.

    A contrived example:

    import { expect, it } from 'vitest';
    
    it('fate', async () => {
      expect.soft(0.5).toBeLessThan(Math.random());
      expect.soft(0.5).toBeGreaterThan(Math.random());
    });
  3. sheremet-va commented on May 10, 2026

    @sheremet-va
    Member

    I like attempts concept, it could be defined on task.result, I think. It should also probably include repeats runs though.

  4. puneetdixit200 commented on May 16, 2026

    @puneetdixit200
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions