"Behavior"

[benjie]

In PostGraphile v5 we've moved away from the introspection-based approach of V4 to a DataSource oriented approach. This gives us a lot more control over what happens and why and how to change it. Essentially we think that the schema build process can be based on these 4 topics:

• sources: tables, views, functions, etc
• types: the data type of something (we call these "codecs" in @dataplan/pg)
  • scalar: string, int, etc
  • composite: a type that has "attributes" which then have their own types
  • list: a list of another type
• relationships: how do the sources relate to one another - user has many posts
• behaviors: how do the different sources, types, relationships behave? What features are enabled for them?

Smart tags/smart comments should be factored into the production of the above, and generally shouldn't leak through to the schema build phase. Except sometimes they do/should... maybe? This separation is not at all clear to me yet.

Currently "behavior" is very loosely defined and has been done in a very ad-hoc manner without a central repository of behavior strings or meanings. If you search over the codebase you'll see it being used in a decent number of ways, from a quick scan I see at least the following:

• 'connection' - should be output as a connection (as opposed to, or maybe in addition to, a list)
• 'list' - reverse of above
• 'single' - represents a single thing (rather than many)
• 'query_field' - should be added as a field to the query type
• 'type_field' - should be added as a field to a type (other than Query)
• 'create' - supports 'create' mutation (table)
• 'insert' - as above?
• 'insertable' - can be included in a 'create' mutation (column)
• 'node' - supports the Node interface (i.e. Global Object Identification spec compliance)
• 'order' - is orderable
• 'filter' - is filterable
• 'filterable' - as above (I said this was inconsistent!)
• 'filterBy' - should be able to filter by this column/function
• 'orderBy' - should be able to order by this column/function
• 'jwt' - this is a JWT type and should be converted to a JWT
• 'all' - should get the "allFoos" entry in the root Query type
• 'select' - should be allowed to be seen by the user
• 'selectable' - something to do with the conditions plugin... should probably have been 'select'?

Already this has a number of issues:

• not type safe
• inconsistent
• ad-hoc
• no central registration of "defaults"
• poor cohesion

By "poor cohesion" I mean that behaviours typically interact. Take an example of a "user_emails" table - at the root you might want cursor pagination so admins can page over the huge number of emails across your codebase (this would be the all and connection behavior on the user_emails source); but when it comes to getting the list of emails for a particular user, you probably just want a straight array without pagination (type User { emails: [UserEmail!] }) - this would correlate with the list behaviour. So it's clear the behaviour is dependent on context, one way to think of this is that the source should have all:connection and the relationship should have list behaviours - i.e. the source should allow all, and when it does so it should use connection (and not list); the relationship should just use the list directly (no connection).

We also have the concept of "default behaviors" - these are the behaviors that should apply if the source/relationship/type/etc have not defined a behavior. Currently these are defined in a per-plugin basis - you request the behavior for the relevant entity, and if it does not have one then you locally treat it as having your plugin's defaults. But what this means is that if you do add a behavior to something explicitly, then all the other defaults get immediately turned off. This seems sub-optimal. I've been considering an explicit syntax such as -all or -connection to disable the given behavior versus the defaults.

When we try and combine these default behaviors we get interesting interactions, for example if you have -all all:connection does that result in all being disabled or not? I think it's equivalent to -all:* +all:connection - i.e. disable all all behaviours, except enable the all:connection behavior. But I'm not sure if this is too confusing, and I'm also not sure if it covers all eventualities.

So my current thinking is:

• there should be a central repository of behaviours, what they apply to, and what their default state is
• plugins should be able to add to this central repository
• we should be able to scope a behaviour to being within another behaviour (all:connection) and we should know which behaviours can be nested in which other behaviors
• we should have a special syntax for expressing all of this, probably as a text string for convenience

Anyway, I've rambled enough... what do you think? How do you want to define the behaviors of your schema?

[benjie]

Excerpt from my notes:

Can we boil everything down to these 4 concepts?

• SOURCE - the PgSource
• RELATIONSHIP - PgSource::relationships
• BEHAVIOUR - smart tags-y
• PRESENTATION/REPRESENTATION/TYPE/NATURE/something - e.g. the PgTypeCodec (PgSource::codec, PgSource::columns[].codec)

[...]

Need to categorise the sources; e.g.:

• behavior: table
• behavior: enum

classification -> behavior

codecs have the behaviours:

enum plugin would need to track down all relationships and remove the relation behavior - enum instead

• filter
• order
• condition
• type gen (e.g. table vs enum (or both))

Then e.g. the enum tables plugin could REMOVE the "table" useAs and instead say that this should be used as an enum. Or it could allow both.

• sources
  • behaviours - govern appearance as fields
    • "all"
    • "byUnique" (based on uniques)
• codecs
  • responsible for:
    • type name
    • connections
  • behaviours - govern field arguments and return types
    • filter
    • order
    • enum
• relationships
  • behaviours - govern fields (but also e.g. filter properties)
    • single forward
    • single backward
    • many backward
    • enum

Behavior has classification and metadata.

Is behavior scoped to context (source/codec/relationship)?

[jemgillam]

• 'connection' - should be output as a connection (as opposed to, or maybe in addition to, a list)

  • 'list' - reverse of above

  • 'single' - represents a single thing (rather than many)

  • 'query_field' - should be added as a field to the query type

  • 'type_field' - should be added as a field to a type (other than Query)

  • 'create' - supports 'create' mutation (table)

  • 'insert' - as above?

  • 'insertable' - can be included in a 'create' mutation (column)

  • 'node' - supports the Node interface (i.e. Global Object Identification spec compliance)

  • 'order' - is orderable

  • 'filter' - is filterable

  • 'filterable' - as above (I said this was inconsistent!)

  • 'filterBy' - should be able to filter by this column/function

  • 'orderBy' - should be able to order by this column/function

  • 'jwt' - this is a JWT type and should be converted to a JWT

  • 'all' - should get the "allFoos" entry in the root Query type

  • 'select' - should be allowed to be seen by the user

  • 'selectable' - something to do with the conditions plugin... should probably have been 'select'?

All of these are "Should I add this thing (field, argument, etc)?" except for the JWT one, which is "What shape should this thing be?"

But defining the context is tricky

[benjie]

Imagine we have a many to many relationship between User and Forum, so we might have:

create table users ( ... );
create table forums ( ... );
create table forum_users (
  id int primary key,
  user_id int not null references users,
  forum_id int not null references forums,
  unique (user_id, forum_id)
);

Looking at this many to many join table, PostGraphile will currently generate (depending on settings, plugins, etc):

• Query.allForumUsers(first: offset: order: condition: filter: last: before: after:)
• Query.allForumUsersList(first: offset: order: condition: filter:)
• Query.forumUser(id: ID!)
• Query.forumUserByRowId(rowId: Int!)
• Query.forumUserByUserIdAndForumId(userId: forumId:)
• Mutation.createForumUser
• Mutation.updateForumUser(input: {id: ID!, patch:})
• Mutation.updateForumUserByRow(input: {rowId: Int!, patch:})
• Mutation.updateForumUserByUserIdAndForumId(input: {userId: forumId: patch:})
• Mutation.deleteForumUser(input: {id: ID!})
• Mutation.deleteForumUserByRow(input: {rowId: Int!})
• Mutation.deleteForumUserByUserIdAndForumId(input: {userId: forumId:})
• User.forumUsers(first: offset: order: condition: filter: last: before: after:)
• User.forumUsersList(first: offset: order: condition: filter:)
• Forum.forumUsers(first: offset: order: condition: filter: last: before: after:)
• Forum.forumUsersList(first: offset: order: condition: filter:)
• UserFilter.forumUsers (for filtering 'users' by related forum_users)
• ForumFilter.forumUsers (for filtering 'forums' by related forum_users)
• UserOrderBy.FORUM_USERS_COUNT_ASC (order users by the number of forums they're in)
• ForumOrderBy.FORUM_USERS_COUNT_ASC (order forums by the number of users in the forum)
• etc

This is generated from:

• The table
• The table's primary key constraint
• The table's unique constraint (user_id, forum_id)
• The table's two foreign key constraints

We want to give users the flexibility to easily control which of these entities (fields, enum values, arguments, etc) is generated. For example:

• Don't add any root level fields for this, but keep the relations
• Never allow filtering or ordering on this table
• Never allow filtering or ordering by this table (e.g. don't filter users by the forums they're in)
• Don't allow update mutations
• Don't include the root-level single accessor fields
• For the root-level single accessor fields, only include the id: ID one and omit the others
• Only allow deletion using the primary key, drop the other mutations

When doing this, we should factor in that the behaviour can come from (from least to most specific):

• Behavioural defaults in the plugins themselves
• Behavioural defaults configured globally
• Behaviour defined on the source itself
• Behaviour defined on the relevant constraint

It would be nice to combine all these.

[benjie]

• Don't add any root level fields for this, but keep the relations

-root:*

• Never allow filtering or ordering on this table

-*:filter -*:order

• Never allow filtering or ordering by this table (e.g. don't filter users by the forums they're in)

-*:filterBy -*:orderBy

• Don't allow update mutations

-*:update

• Don't include the root-level single accessor fields

-root:single

• For the root-level single accessor fields, only include the id: ID one and omit the others

-root:single +root:single:nodeId

• Only allow deletion using the primary key, drop the other mutations

-root:delete +root:delete:pk

Use connection for root level, but simple array without pagination/filtering for relations:

-root:list +root:connection -relation:connection +relation:list -relation:list:order -relation:list:pagination -relation:list:condition

[benjie]

I'm thinking that we split on : and the rightmost thing is the "behaviour" and the rest are the context. We then search back through the behaviour string for the rightmost match, and apply that.

We can then just concatenate the strings in order: the plugin default, the global (user) default, the source behaviour and the behaviour of any relevant constraint (relation, unique, etc).

e.g.: we want to know whether or not to define the list field for a relation:

1. Establish the behaviour string:
   • global defaults: +connection +list
   • user defaults: +connection -list -list:order -list:filter
   • source behavior: -root:*
   • constraint: +list -connection -pagination
   • RESULT (concatenate, join with spaces): behaviorSpecsString = "+connection +list +connection -list -list:order -list:filter -root:* +list -connection -pagination"
2. Determine whether or not to add the "list" field for a relation:
   • hasBehavior(behaviorSpecsString, 'relation:list')
3. If yes, add the list field, but when adding the arguments check:
   • hasBehavior(behaviorSpecsString, 'relation:list:pagination')
   • hasBehavior(behaviorSpecsString, 'relation:list:filter')
   • hasBehavior(behaviorSpecsString, 'relation:list:order')

hasBehavior(behaviorSpecsString, filter) will do the following:

1. Let behaviorSpecs be the result of splitting behaviorSpecsString on spaces (trimming empty entries)
2. For each behaviorSpec in behaviorSpecs looping backwards:
   1. Let positive be false if behavior starts with -, true otherwise
   2. Let behavior be the remainder of behaviorSpec after removing any leading + or -
   3. If behaviorMatches(behavior, filter):
      1. Return positive.
3. Return false.

behaviorMatches(behavior, filter) is the fuzzy part... Perhaps something like:

1. If behavior is filter return true.
2. If filter doesn't contain a : return false.
3. Let nextFilter be filter with the text up to and including the first : removed.
4. Return behaviorMatches(behavior, nextFilter).