Skip to content

Smart way to detect benchmarks without a steady state #594

Open
@AndreyAkinshin

Description

@AndreyAkinshin

Sometimes, benchmarks don't have a steady state. For example, each benchmark iteration can take more time than the previous one. In this case, our default run strategy will run new iterations again and again (because it tries to find a steady state). We should have a smart way to detect such kind of situation, terminate the process, and warn users about it. We should also beware of false alarms: in some cases, it's a legal situation (the main logic of a benchmark can require dozens of warmup iteration before it reaches the steady state).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @AndreyAkinshin@morgan-kn

      Issue actions

        Smart way to detect benchmarks without a steady state · Issue #594 · dotnet/BenchmarkDotNet