Loop Variable over Literal List Not Recognised as Having Literal Type

[macdjord]

Bug Report

When a for-loop is used to iterate over a list-literal (or other collection-literal) which contains only scalar literals, then MyPy should be smart enough to recognize that the implicit type of the loop variable is not merely the type of the values in the list but the specific values in that list.

To Reproduce

import typing as _tp

MyLiteral: _tp.TypeAlias = _tp.Literal["foo", "bar", "baz"]

def func(a1: MyLiteral):
    ...

def func2() -> None:
    a: MyLiteral
    a = "foo"
    func(a)
    a = "bar"
    func(a)

    b: MyLiteral
    for b in ["foo", "bar"]:
        func(b)

    for c in ["foo", "bar"]:
        func(c)

Expected Behavior

• At a minimum, there should be no warnings emitted on the first loop: MyPy should be smart enough to identify that all the values the loop is assigning to b are compatible with its explicit type of MyLiteral, the same way it identifies that the values being assigned to a on lines 10 & 12 are legal.
• Ideally, there should also be no warnings emitted on the second loop: it would be nice if MyPy correctly identified the implicit type of c as Literal["foo", "bar"], which is compatible with the first argument of func()

Actual Behavior

$ mypy [REDACTED]/tmp.py
[REDACTED]\tmp.py:16: error: Incompatible types in assignment (expression has type "str", variable has type "Literal['foo', 'bar', 'baz']")  [assignment]
[REDACTED]\tmp.py:20: error: Argument 1 to "func" has incompatible type "str"; expected "Literal['foo', 'bar', 'baz']"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

• Mypy version used: mypy 1.5.1 (compiled: yes)
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: Python 3.11.2

Related Issues

• Looping through literals not typed correctly #9230: Issue complaining about the same problem, in the specific case where the loop variable is being used to access members of TypedDict. A fix was applied which solved the problem, but only for the special-case of accessing members of TypedDict, not for other, similar cases such as using the loop variable as a function argument as in the above example

[asottile-sentry]

use a tuple instead of a list

[A5rocks]

I agree the first should work but IMO the second is bad -- remember that list[T] is invariant so if mypy infers list["foo" | "bar" | "baz"] that's not assignable to list[str]. So the consequences of guessing wrong are really quite bad.

[macdjord]

use a tuple instead of a list

As a general rule, I try to maintain a semantic distinction between tuples and lists: tuples are used to collect unlike elements of related data together so they can be manipulated as a single object, whereas lists are used to collect a sequence of common items for iteration. I'll use the 'wrong' type if I have to, e.g. converting a list to a tuple if I need to use it as a dictionary key, or using a list to assemble the pieces of related data one-by-one before I then convert them to a tuple once complete.

While, in this case, a tuple and a one-off list literal are functional identical, a list is clearly semantically more appropriate, so I'd prefer to use one.

I agree the first should work but IMO the second is bad -- remember that list[T] is invariant so if mypy infers list["foo" | "bar" | "baz"] that's not assignable to list[str]. So the consequences of guessing wrong are really quite bad.

In the general case, yes. I would absolutely not expect this to pass muster:

ls = ["foo", "bar"]
d: MyLiteral
for d in ls:
    func(d)

But in the specific case where a list-literal is being used as the iterable of a for-loop, it makes more sense to infer the type as specifically as possible. It doesn't matter that list[T] is invariant when the list in question cannot be modified.

There's already precedent for making exactly this kind of distinction:

def func3(l1: list[MyLiteral]):
    ...

func3(["foo", "bar"])
fs1: frozenset[MyLiteral] = frozenset(["foo", "bar"])

l = ["foo", "bar"]
func3(l)
fs2: frozenset[MyLiteral] = frozenset(l)

The first call to func3() does not raise an error, nor does the assignment to fs1, while the second call and the assignment to fs2 do raise arg-type errors. Clearly MyPy is willing to consider the context a list-literal is being used in when deciding what its implicit type is. Why should use in a for-statement not be similarly considered?

Hell, if we just assumed that the implicit type of ["foo", "bar"] is always list[str], then by the strictest reading, l: list[MyLiteral] = ["foo", "bar"] would be an error! The type hint applies to the variable, not the literal on the other side of the assignment, so that statement would be assigning a value of implicit type list[str] to a variable of explicit type list[MyLiteral], and these are not compatible types.

[macdjord]

Now, I can see one potential issue with my proposal:

for phonetic_letter in [
        "Alpha",
        "Bravo",
        "Charlie",
        "Delta",
        "Echo",
        "Foxtrot",
        "Golf",
        "Hotel",
        "India",
        "Juliet",
        "Kilo",
        "Lima",
        "Mike",
        "November",
        "Oscar",
        "Papa",
        "Quebec",
        "Romeo",
        "Sierra",
        "Tango",
        "Uniform",
        "Victor",
        "Whiskey",
        "X-ray",
        "Yankee",
        "Zulu",
]:
    print(f"The NATO phonetic name for {phonetic_letter[1]} is {phonetic_letter!r}")
    _tp.reveal_type(phonetic_letter)

What should reveal_type() print? Because this is not a particularly useful response:

note: Revealed type is "Union[Literal['Alpha'], Literal['Bravo'], Literal['Charlie'], Literal['Delta'], Literal['Echo'], Literal['Foxtrot'], Literal['Golf'], Literal['Hotel'], Literal['India'], Literal['Juliet'], Literal['Kilo'], Literal['Lima'], Literal['Mike'], Literal['November'], Literal['Oscar'], Literal['Papa'], Literal['Quebec'], Literal['Romeo'], Literal['Sierra'], Literal['Tango'], Literal['Uniform'], Literal['Victor'], Literal['Whiskey'], Literal['X-ray'], Literal['Yankee'], Literal['Zulu']]"

I can see 3 general solutions:

• Limit the inference to list-literals of no more than some specific number of items
• Some sort of backtracking, where MyPy checks whether there is actually any code within the loop where the literal values matter, and if not it retroactively decides the variable's type was just builtins.str all along
• Just have reveal_type() say builtins.str while MyPy internally still keeps track of the literals and considers them before generating any errors. After all, that's what it displays right now, even though since v1.14 MyPy has actually secretly kept track of the potential literal values to allow their use as typed-dict indices as per Looping through literals not typed correctly #9230.

(Side note: Anybody else wish MyPy was better about using concise forms for type names? Because that entire 482-char Union[Literal['Alpha'], ..., Literal['Zulu']] could instead be expressed as Literal["Alpha", "Bravo", ..., "Zulu"], which is much more readable and only 250 chars.)

[A5rocks]

even though since v1.14 MyPy has actually secretly kept track of the potential literal values to allow their use as typed-dict indices as per

I think you're misunderstanding that issue. It's about iterating over specifically tuples for literal keys.

[macdjord]

even though since v1.14 MyPy has actually secretly kept track of the potential literal values to allow their use as typed-dict indices as per

I think you're misunderstanding that issue. It's about iterating over specifically tuples for literal keys.

Okay, fair enough, I missed that in that case it is looping over a tuple rather than over a list. Option 3 on what to do about reveal_type() no longer applies.

However, I feel my original point stands: the implicit type of a loop variable when iterating over a list-literal should use as narrow a type as possible. for b in ["foo", "bar"]: and for b in ("foo", "bar"): are exactly functionally equivalent, except that the first version is, IMO, more natural and readable. So why should MyPy narrow the type of the loop variable in the second case but not in the first?