Incorrect mapping of actual to formal arguments.

[randolf-scholz]

def takes_at_least3(a1: str, a2: str, a3: str, *args: str) -> None: ...

def test(
    x0: tuple[str, ...],
    x1: tuple[str, *tuple[str, ...]],
    x2: tuple[str, str, *tuple[str, ...]],
    x3: tuple[str, str, str, *tuple[str, ...]],
) -> None:
    takes_at_least3(*x0)  # no error
    takes_at_least3(*x1)  # Missing positional arguments "x2", "x3"
    takes_at_least3(*x2)  # no error
    takes_at_least3(*x3)  # no error

https://mypy-play.net/?mypy=latest&python=3.12&gist=34662974552c517610691a6b5a792ba2

There are only 2 logically sensible results here:

1. All of x0-x3 should be accepted
2. Only x3 should be accepted, since it is the only type that is guaranteed to provide at least 3 items

The choice depends on whether we consider tuple[str, ...] as equal to

1. The union type tuple[()] | tuple[str] | tuple[str, str] | ...
2. The gradual AnyOf[tuple[()], tuple[str], tuple[str, str], ... ] (AnyOf - Union for return types typing#566)

[randolf-scholz]

This is causes by bad formal_to_actual mapping:

mypy/mypy/argmap.py

Lines 57 to 66 in 0d23c61

	if isinstance(actualt, TupleType):
	# A tuple actual maps to a fixed number of formals.
	for _ in range(len(actualt.items)):
	if fi < nformals:
	if formal_kinds[fi] != nodes.ARG_STAR2:
	formal_to_actual[fi].append(ai)
	else:
	break
	if formal_kinds[fi] != nodes.ARG_STAR:
	fi += 1

For input x1, this just sees tuple[str, *tuple[str, ...]] and then maps str -> a1 and *tuple[str, ...] -> a2, and then assumes argument for a3 is missing.

[sterliakov]

This is a bug, and IMO all those calls should be accepted. We accept unpacked variadic tuples everywhere when they can possibly be expanded into a correct sequence - this aims at maintaining the balance between correctness and usability.

[randolf-scholz]

A second example: https://mypy-play.net/?mypy=latest&python=3.12&gist=b13968d8aa8c885711e80ff40abf5074

def demo(x1: int, x2: str) -> None: ...

def test(
    x0: tuple[int, ...],
    x1: tuple[str, ...],
    y0: tuple[*tuple[int, ...]],
    y1: tuple[*tuple[str, ...]],
) -> None:
    demo(*x0, *x1)  # Argument 1 has incompatible type "*tuple[int, ...]"
    demo(*y0, *y1)  # No error

here, two equivalent types lead to different inference.

[randolf-scholz]

And here's if we also check a function that takes exactly 3 arguments using the original setup:

def takes_exactly_3(a1: str, a2: str, a3: str, /) -> None: ...
def takes_at_least3(a1: str, a2: str, a3: str, /, *args: str) -> None: ...

def test_takes_at_least3(
    x0: tuple[str, ...],
    x1: tuple[str, *tuple[str, ...]],
    x2: tuple[str, str, *tuple[str, ...]],
    x3: tuple[str, str, str, *tuple[str, ...]],
    x4: tuple[str, str, str, str, *tuple[str, ...]],
) -> None:
    takes_at_least3(*x0)  # no error
    takes_at_least3(*x1)  # too few arguments
    takes_at_least3(*x2)  # no error
    takes_at_least3(*x3)  # no error
    takes_at_least3(*x4)  # no error

def test_takes_exactly_3(
    x0: tuple[str, ...],
    x1: tuple[str, *tuple[str, ...]],
    x2: tuple[str, str, *tuple[str, ...]],
    x3: tuple[str, str, str, *tuple[str, ...]],
    x4: tuple[str, str, str, str, *tuple[str, ...]],
) -> None:
    takes_exactly_3(*x0)  # no error
    takes_exactly_3(*x1)  # too few arguments
    takes_exactly_3(*x2)  # # no error
    takes_exactly_3(*x3)  # too many arguments
    takes_exactly_3(*x4)  # too many arguments

https://mypy-play.net/?mypy=1.17.1&python=3.12&gist=4837507130a3fba367e8458dde74851d

@sterliakov So, If we treat tuple[T, ...] as AnyOf rather than Union, in this extended example the only thing that should fail is takes_exactly_3(*x4), right?

[randolf-scholz]

I have thought about this some more, and I am starting to believe this is generally not feasible due to combinatorial explosion, especially when the variable tuples have different types.

But I came up with a general approach that limits the computational complexity and is sufficiently close to what mypy is already doing, so that I believe it could be integrated.

Setup

1. Assume we are only dealing with positional arguments

2. Assume that all positional arguments are passed in the form of *args¹

   So w.l.o.g. our function signature is f(*x1, *x2, ...., *xn)

3. Define the Tuple Normal Form (TNF):
   Given any tuple tuple[T1, ..., Tn] or tuple[T, ...] define the TNDF as the unique tuple tuple[P1, ..., Pn, *Vs?, S1, ..., Sm] that splits the tuple into 3 parts: a maximal fixed size finite prefix P1-Pn, a potentially non-existent variable part *Vs?, and a maximal finite size suffix S1-Sm (which is always empty if there is no variable part)

4. Convert each argument x to TNF. Let c denote the first argument with a variable length part. So w.l.o.g. out call signature is:

   f(*a1, ..., *an, *c, *b1, ..., *bm)

   where each a is a statically known fixed length tuple, and c is a variable length tuple.

Algorithm

1. Perform a split at c:

   • Let tuple[P1, ..., Pn, *Vs, S1, ..., Sm] be the TNF of c.
   • Map the prefix to the remaining formal arguments as usual. Then there are 3 options:
     1. We exhausted the formal positional args before reaching *Vs -> we know there are too many arguments!
     2. We exhaust the prefix before reaching an ARG_STAR formal -> tail_slice=[*Vs, S1, ..., Sm].
     3. We reached an ARG_STAR formal before exhausting the prefix -> tail_slice=[Pk, ..., Pn, *Vs, S1, ..., Sm].

2. Recombine the tail slice with the remaining arguments into a single *args, and express it in TNF:
   *[Pk, ..., Pn, *Vs, S1, ..., Sm], *b1, ...., *bm ⟶ [P1', ..., Pr', *Vs', S1', ..., St']
   call this the tail_arg. So we have reinterpreted our original call signature as:

   f(p1, p2, ..., pn, *tail_args)

   where each p is a regular (non star) positional argument, and tail_args is tuple[P1', ..., Pr', *Vs', S1', ..., St']
   by method of statically extracting the maximal fixed length prefix.

3. Map the tail_args onto the remaining formal positional arguments.

4. Type check the prefix part as usual element-wise.

5. To type check the tail, we know *tail_args maps to some formal arguments with kinds: (ARG_POS, ...., ARG_POS, ARG_STAR) with types T1, ..., Tn and some unpacked tuple *Ts for ARG_STAR. (ignore paramspec for now)

   1. combine all these types into a tuple tuple[T1, ..., Tn, *Ts]
   2. express this in TNF.
   3. compare the result with tail_args

Finally, regarding the TNF, one needs to see how multiple variable length parts need to be combines. For example:

tuple[int, *tuple[int, ...], None, *tuple[str, ...], str]

Then the TNF is tuple[int, *VS, str], where Vs is one of:

1. tuple[Any, ...] (fallback)
2. tuple[object, ...] (join)
3. tuple[int | None | str] (union)
4. tuple[*tuple[int, ...], None, *tuple[str, ...]] (disallowed by spec - only one variable length part allowed).

Application to the original example

Considering the example #19692 (comment), this should give us the desired behavior, since all the tuple argument types are str, the recombination into TNF of the tail does not produce any unexpected types.

Footnotes

1. We can always interpret a regular positional argument as a *args by simply wrapping it in a tuple: f(x) ⟺ f(*(x,)) ↩

[sterliakov]

the only thing that should fail is takes_exactly_3(*x4), right?

Yes, I think so - we really don't want to restrict unpacking variadic tuples into fixed arguments, this would break too may programs in wild (though maybe it can be made configurable via a separate flag, --strict-tuple-unpacking?)