Revisit structure at runtime of allOf/oneOf/anyOf

[czechboy0]

Structure at runtime of allOf/oneOf/anyOf - today, we use nesting of structs to represent the nested schemas.

Could be represented as a single flattened struct that conforms to multiple protocols, one for each nested type, or some other way.

Things to keep in mind:

• lack of nested protocols in Swift
• cost of using existentials vs concrete types

[iDevPro]

allOf - is like inheritance but with struct it would be like big struct with embed fields from parent struct :)

[acecilia]

👋 Would anybody be able to share some context about why when using allOf, the properties are generated in such an unexpected way? Example:

For this spec:

components:
  schemas:
    MyType:
      allOf:
        - $ref: "#/components/schemas/Property1"
        - type: object
          properties:
            property2:
              type: string
    Property1:
      type: object
      properties:
        property1:
          type: string

The generated code is:

/// - Remark: Generated from `#/components/schemas/MyType`.
internal struct MyType: Codable, Hashable, Sendable {
    /// - Remark: Generated from `#/components/schemas/MyType/value1`.
    internal var value1: Components.Schemas.Property1
    /// - Remark: Generated from `#/components/schemas/MyType/value2`.
    internal struct Value2Payload: Codable, Hashable, Sendable {
        /// - Remark: Generated from `#/components/schemas/MyType/value2/property2`.
        internal var property2: Swift.String?
        /// Creates a new `Value2Payload`.
        ///
        /// - Parameters:
        ///   - property2:
        internal init(property2: Swift.String? = nil) {
            self.property2 = property2
        }
        internal enum CodingKeys: String, CodingKey {
            case property2
        }
    }
    /// - Remark: Generated from `#/components/schemas/MyType/value2`.
    internal var value2: Components.Schemas.MyType.Value2Payload
    /// Creates a new `MyType`.
    ///
    /// - Parameters:
    ///   - value1:
    ///   - value2:
    internal init(
        value1: Components.Schemas.Property1,
        value2: Components.Schemas.MyType.Value2Payload
    ) {
        self.value1 = value1
        self.value2 = value2
    }
    internal init(from decoder: any Decoder) throws {
        value1 = try .init(from: decoder)
        value2 = try .init(from: decoder)
    }
    internal func encode(to encoder: any Encoder) throws {
        try value1.encode(to: encoder)
        try value2.encode(to: encoder)
    }
}
/// - Remark: Generated from `#/components/schemas/Property1`.
internal struct Property1: Codable, Hashable, Sendable {
    /// - Remark: Generated from `#/components/schemas/Property1/property1`.
    internal var property1: Swift.String?
    /// Creates a new `Property1`.
    ///
    /// - Parameters:
    ///   - property1:
    internal init(property1: Swift.String? = nil) {
        self.property1 = property1
    }
    internal enum CodingKeys: String, CodingKey {
        case property1
    }
}

But my expectation was for allOf to be just syntactic sugar of the yaml spec to be able to reuse properties instead of copy/paste them, and be equivalent to:

components:
  schemas:
    MyType:
      type: object
      properties:
        property1:
          type: string
        property2:
          type: string

That generates:

/// - Remark: Generated from `#/components/schemas/MyType`.
internal struct MyType: Codable, Hashable, Sendable {
    /// - Remark: Generated from `#/components/schemas/MyType/property1`.
    internal var property1: Swift.String?
    /// - Remark: Generated from `#/components/schemas/MyType/property2`.
    internal var property2: Swift.String?
    /// Creates a new `MyType`.
    ///
    /// - Parameters:
    ///   - property1:
    ///   - property2:
    internal init(
        property1: Swift.String? = nil,
        property2: Swift.String? = nil
    ) {
        self.property1 = property1
        self.property2 = property2
    }
    internal enum CodingKeys: String, CodingKey {
        case property1
        case property2
    }
}

[czechboy0]

Hi @acecilia,

the reason the generated code closely mirrors the source OpenAPI doc is due to our first principle, to faithfully represent it in Swift.

The generator aims to generate what the OpenAPI doc says, rather than to try to guess what the author of the OpenAPI doc meant. The OpenAPI specification is quite flexible and general, and there are many different patterns that adopters use that look similar in the OpenAPI document, but are used differently from adopters' code - so we don't want to overly optimize for one pattern, which could complicate/prevent other patterns.

Your example is a good one, and in your case, you just want a single flat object of all the properties collected from the allOf's subschemas.

However, let's imagine a different, but similar, example:

components:
  schemas:
    Special1:
      allOf:
        - $ref: "#/components/schemas/Common"
        - type: object
          properties:
            property2:
              type: string
    Special2:
      allOf:
        - $ref: "#/components/schemas/Common"
        - type: object
          properties:
            property3:
              type: string
    Common:
      type: object
      properties:
        property1:
          type: string

In this example, we don't know whether Common is ever used as its own type, or is only ever used as a way to share properties between Special1 and Special2.

So the most accurate representation of the above is to generate the subschemas as individual properties of the allOf schemas, and let adopters use them as they need them.

If we flatten them, and the adopter has a function that takes Common, suddenly they can't just pass Special1.value1 directly to the function, but they have to write custom code to extract those values back out into an instance of Common.

[acecilia]

Understood, thanks so much for the context 🙏

About the example above, if the author of the spec wanted to have Common as its own type, I think the spec would need to reflect that (instead of just being an implementation detail in the generator), and the more reasonable way to do that would be to assign it a field name:

components:
  schemas:
    Special1:        
      type: object
      properties:
        property2:
          type: string
        common:
          $ref: "#/components/schemas/Common"
    Special2:
      type: object
      properties:
        property3:
          type: string
        common:
          $ref: "#/components/schemas/Common"
    Common:
      type: object
      properties:
        property1:
          type: string

I would argue that current implementation of allOf is not aligned very much with what the OpenAPI says. For example in here is mentioned that:

it does not imply a hierarchy between the models. For that purpose, you should include the discriminator

allOf acts as a tool for combining schemas used in specific cases with the general one

And here:

In your API, you may have model schemas that share common properties. Instead of describing these properties for each schema repeatedly, you can describe the schemas as a composition of the common property set and schema-specific properties. In OpenAPI version 3, you do this with the allOf keyword

Also, looking at the Codable representation, the generated json matches 100% what a flattened struct would produce. Seems to me the generated struct is not aligned with the OpenAPI spec, and as a result the struct requires the custom Codable implementation.

But I also understand that:

• Above is somewhat opinionated
• That the struct is somewhat an internal implementation detail, and itself does not need to match the OpenAPI spec
• That there are probably other considerations explaining why above was done in the way it is today

Considering the opinionated nature of the solution above, would you be open to the introduction of a configuration that users can control, so the generation of allOf can be customized to produce a flattened structure?

[czechboy0]

I'm not ready to commit to accepting a configuration option for this, because to make this work in the general case is likely to require a lot of work and might run against our principle of keeping the generator relatively simple and predictable.

There are tricky cases, such as the same property existing in two separate subschemas, with different comments, for example. There's also the problem that the process of "flattening" would require a lot of judgement calls of how exactly to do that (I've played with it in the past and backed away from it).

That the struct is somewhat an internal implementation detail, and itself does not need to match the OpenAPI spec

I think this is the key point - the question of how to project allOf (and anyOf) into Swift isn't a settled question, and we chose the solution that works in all cases, without us having to bake in too many opinions. A flattened solution will require being more opinionated, and we didn't see much benefit. Ultimately, an adopter can always flatten the properties in their own code.

I won't stop you from experimenting, but just wanted to provide my honest take on the likelihood that we'd actually accept the complexity increase without a clear benefit. 🙂