BenchmarkDotNet should warn if there are no benchmarks

[richardcocks]

I've got a small benchmark that I was about to open a bug report because it appeared to "crash" during Benchmark Validation.

It shows // Validating benchmarks: and then the process exits without any error or warning.

I'm an idiot, and it turns out I forgot to annotate any methods at all with [Benchmark].

A warning that "No benchmarks found" would help idiots like me realise their mistake without thinking that validation is crashing.

[CodingwithKetan]

@timcassell, can I work on this issue?

[timcassell]

Yes

[CodingwithKetan]

Thank you @timcassell !

[AvishaiDotan]

@CodingwithKetan Have you had a chance to make any progress on this?

It looks like checking whether supportedBenchmarks.Length == 0 could be a good way to handle this. If there are no supported benchmarks, logging an informative message and returning a summary array might be sufficient. Reference:

BenchmarkDotNet/src/BenchmarkDotNet/Running/BenchmarkRunnerClean.cs

Line 63 in 21da339

compositeLogger.WriteLineInfo("// Validating benchmarks:");

if (supportedBenchmarks.Length == 0)
{
    compositeLogger.WriteLineError("// No benchmarks have been found");
    return new[] { Summary.ValidationFailed(title, resultsFolderPath, logFilePath,
        ImmutableArray.Create(new ValidationError(true, "No benchmarks have been found"))) };
}

If you're not actively working on this, I’d be happy to pick it up — just let me know.

[CodingwithKetan]

@AvishaiDotan, as you got solution, so you can work on this.