TypeForm[T]: Spelling for regular types (int, str) & special forms (Union[int, str], Literal['foo'], etc)

[davidfstr]

(An earlier version of this post used TypeAnnotation rather than TypeForm as the initially proposed spelling for the concept described here)

Feature

A new special form TypeForm[T] which is conceptually similar to Type[T] but is inhabited by not only regular types like int and str, but also by anything "typelike" that can be used in the position of a type annotation at runtime, including special forms like Union[int, str], Literal['foo'], List[int], MyTypedDict, etc.

Pitch

Being able to represent something like TypeForm[T] enables writing type signatures for new kinds of functions that can operate on arbitrary type annotation objects at runtime. For example:

# Returns `value` if it conforms to the specified type annotation using typechecker subtyping rules.
def trycast(typelike: TypeForm[T], value: object) -> Optional[T]: ...

# Returns whether the specified value can be assigned to a variable with the specified type annotation using typechecker subtyping rules.
def isassignable(value: object, typelike: TypeForm[T]) -> bool: ...

Several people have indicated interest in a way to spell this concept:

• @ltworf: Type[T] -> T has a strange behaviour #9003
• @glyph: Type[T] -> T has a strange behaviour #9003 (comment)
• @davidfstr (yours truly): Type[T] -> T has a strange behaviour #9003 (comment)
• @Nico-Kialo: Version 0.780 does not recognize python internal types as typing.Type anymore #8992
• @hmvp: Version 0.780 does not recognize python internal types as typing.Type anymore #8992 (comment)

For a more in-depth motivational example showing how I can use something like TypeForm[T] to greatly simplify parsing JSON objects received by Python web applications, see my recent thread on typing-sig:

• Spelling the type of a runtime type annotation object (typing-sig thread)

If there is interest from the core mypy developers, I'm willing to do the related specification and implementation work in mypy.

[hauntsaninja]

Why not do something like:

from __future__ import annotations
from typing import *

T = TypeVar("T")

class TypeAnnotation(Generic[T]):
    @classmethod
    def trycast(cls, value: object) -> T:
        ...

reveal_type(TypeAnnotation[Optional[int]].trycast(object()))

[JelleZijlstra]

Why can't we make Type[T] also work for other special forms?

@hauntsaninja's workaround is useful, but it would be better to have a feature in the core type system for this.

[davidfstr]

@hauntsaninja , using your workaround I am unable to access the passed parameter at runtime from inside the wrapper class.

The following program:

from __future__ import annotations
from typing import *

T = TypeVar('T')

class TypeAnnotation(Generic[T]):
    @classmethod
    def trycast(cls, value: object) -> Optional[T]:
        Ts = get_args(cls)
        print(f'get_args(cls): {Ts!r}')
        return None

ta = TypeAnnotation[Union[int, str]]
print(f'get_args(ta): {get_args(ta)!r}')
result = ta.trycast('a_str')

prints:

get_args(ta): (typing.Union[int, str],)
get_args(cls): ()

[davidfstr]

@JelleZijlstra commented:

Why can't we make Type[T] also work for other special forms?

@hauntsaninja's workaround is useful, but it would be better to have a feature in the core type system for this.

I agree that having Type[T] be widened to mean "anything typelike, including typing special forms" would be an alternate solution. A very attractive one IMHO.

However it appears that there was a deliberate attempt in mypy 0.780 to narrow Type[T] to only refer to objects that satisfy isinstance(x, type) at runtime. I don't understand the context for that decision. If however that decision was reversed and Type[T] made more general then there would be no need for the additional TypeAnnotation[T] syntax I'm describing in this issue.

[gvanrossum]

I believe the issue is that Type is used for things that can be used as the second argument of isinstance(). And those things must be actual class objects (or tuples of such) -- they cannot be things like Any, Optional[int] or List[str].

So if this feature is going to happen I think it should be a separate thing -- and for the static type system it probably shouldn't have any behavior, since such objects are only going to be useful for introspection at runtime. (And even then, how are you going to do the introspection? they all have types that are private objects in the typing module.)

[ltworf]

Well I wrote this module with various checks and add every new major py version to the tests to see that it keeps working: https://github.com/ltworf/typedload/blob/master/typedload/typechecks.py

Luckily since py3.6 it has not happened that the typing objects change between minor upgrades of python.

[davidfstr]

I believe the issue is that Type is used for things that can be used as the second argument of isinstance(). And those things must be actual class objects (or tuples of such) -- they cannot be things like Any, Optional[int] or List[str].

Makes sense.

for the static type system it probably shouldn't have any behavior, since such objects are only going to be useful for introspection at runtime.

Agreed.

how are you going to do the introspection? they all have types that are private objects in the typing module.

The typing module itself provides a few methods that can be used for introspection. typing.get_args and typing.get_origin come to mind.

[davidfstr]

I could see a couple of possible spellings for the new concept:

• TypeAnnotation (as proposed earlier by the issue title)
• TypeForm -- as in a "typing special form", which is consistent with the following existing documentation:
  • "special forms" -- https://docs.python.org/3/library/typing.html#special-forms
  • "special typing form" -- https://docs.python.org/3/library/typing.html#typing.get_origin
• TypeHint -- as in the original PEP 484 "Type Hints", although I still lean toward "type annotation" as more of the proper full name

Personally I'm now leaning toward TypeForm (over TypeAnnotation) because it is consistent with prior documentation and is more succinct to type. It does sound a bit abstract but I expect only relatively advanced developers will be using this concept anyway.

(Let the bikeshedding begin. :)

[gvanrossum]

I like TypeForm.

[davidfstr]

Okay I'll go with TypeForm then, for lack of other input.

Next steps I expect are for me to familiarize myself with the mypy codebase again since I'm a bit rusty. Hard to believe it's been as long as since 2016 I put in the first version of TypedDict. Rumor is it that the semantic analyzer has undergone some extensive changes since then.

[gvanrossum]

Good. And yeah, a lot has changed. Once you have this working we should make a PEP out of it.

[davidfstr]

Once you have this working we should make a PEP out of it.

Yep, will do this time around. :)