Recursive Suite Traits must conform to TestTrait or else they crash

[KentLottis-jwn]

Description

If a recursive SuiteTrait type fails to also conform to TestSuite, you get a runtime crash (apparently an assertion failure)

See this Swift.org discussion

Reproduction

// DANGER: missing TestSuite conformance
struct TestConfiguration: SuiteTrait, TestScoping {
    let isRecursive: Bool = true
    func provideScope(for test: Test, testCase: Test.Case?, performing function: @Sendable () async throws -> Void) async throws {
        try await function()
    }
}

extension Trait where Self == TestConfiguration {
    static var testConfiguration: Self { Self() }
}

@Suite(.testConfiguration)
struct DemoSuite {
    @Test func demo1() {}

Expected behavior

Ideally, some sort of catastrophic but non-crashing result. Or, at least, a clearer indication as to what's wrong

Environment

Xcode 16.3

Testing Library Version: 124

Additional information

No response

[grynspan]

As I wrote there:

@briancroom, @smontgomery, and I need to squabble over discuss it a bit more, but IMHO it is likely that we would treat the trait you have as recursively applying to suites nested in the outermost suite, but not to tests, and that you'd need to add TestTrait conformance to make it also apply to tests in that suite.

[KentLottis-jwn]

Right. So the docs would read something like this:

If you want the trait to apply to sub-suites, set isRecursive to true. If you want the trait to apply to individual test functions within each suite, add TestTrait conformance

[grynspan]

You'd need to set isRecursive to true to have anything at all inherit the trait, but what inherits the trait would be dependent on which protocols the type conforms to. So something like:

If you want the trait to apply to sub-suites, set isRecursive to true. If you want the trait to apply to individual test functions within each suite, set isRecursive to true and add TestTrait conformance

[grynspan]

Which then may mean we want a tri- or quad-state here, not a boolean:

enum Inheritance {
case none
case nestedTests
case nestedSuites
case allNestedThingies
}

var inheritance: Inheritance { get } // default .none

Or:

enum InheritedBy: Hashable {
case tests
case suites
}

var inheritedBy: Set<InheritedBy> { get } // default []