Not able to accomplish a mapping protocol with contravariant key type

[Azureblade3808]

I have been building a mapping protocol which is expected to have contravariant key type, unlike collections.abc.Mapping whereas the key type is invariant.

My current code is something like -

from typing import Protocol, overload

class MapLike[K, V](Protocol):
    def __getitem__(self, key: K, /) -> V: ...

    @overload
    def get(self, key: K, /) -> V | None: ...
    @overload
    def get[D](self, key: K, default: V | D, /) -> V | D: ...

It is expected that any dict object should conform to this protocol with compatible parameter types. It works most of time, such as -

from typing import Any

a: MapLike[str, Any] = {}
b: MapLike[str, int] = dict[str, int]()
c: MapLike[str, int] = {"a": 0}

But sometimes, it fails, such as -

d: MapLike[str, int] = {}  # Error

def _[K, V](k: K, v: V) -> tuple[K, V]:
    e: MapLike[K, V] = dict()  # Error
    _ = e

    return (k, v)

The two assignments above are both valid in my mind though.

With previous versions of Pyright (up till 1.1.375), I swapped the order of two overloads of MapLike.get, and the errors with d and e were magically gone (not sure why). But this trick no longer fully works with recent versions (assignment to d would still get reported).

I also see that if I removed a redundant overload (the 2nd overload) of dict.get from typeshed, these assignments would pass Pyright checks.

Am I supposed to try to file a PR on typeshed, or wait for some changes of Pyright?

[erictraut]

I don't think that the second overload in typeshed is redundant. There are cases where the presence of this overload will produce different results than if it were absent.

If you want your protocol to work with the definition of dict, you'll need to add a third overload that mirrors the dict.get method definition.

Code sample in pyright playground

from typing import Protocol, overload, Any


class MapLike[K, V](Protocol):
    def __getitem__(self, key: K, /) -> V: ...

    @overload
    def get(self, key: K, /) -> V | None: ...
    @overload
    def get(self, key: K, default: V, /) -> V: ...
    @overload
    def get[D](self, key: K, default: D, /) -> V | D: ...


a: MapLike[str, Any] = {}
b: MapLike[str, int] = dict[str, int]()
c: MapLike[str, int] = {"a": 0}
d: MapLike[str, int] = {}


def _[K, V](k: K, v: V) -> tuple[K, V]:
    e: MapLike[K, V] = dict()
    _ = e

    return (k, v)

Incidentally, mypy generates the same errors as pyright on your code sample, and it produces no errors in my modified sample.

[Azureblade3808]

If MapLike is implemented as such, it can not longer cover collections.abc.Mapping objects.

from collections.abc import Mapping
from typing import Protocol, overload


class MapLike[K, V](Protocol):
    def __getitem__(self, key: K, /) -> V: ...

    @overload
    def get(self, key: K, /) -> V | None: ...
    @overload
    def get(self, key: K, default: V, /) -> V: ...
    @overload
    def get[D](self, key: K, default: D, /) -> V | D: ...


def _[K, V](some_map_like: MapLike[K, V], some_dict: dict[K, V], some_mapping: Mapping[K, V]) -> None:
    some_map_like = some_dict
    some_map_like = some_mapping  # Error
    _ = some_map_like

[Azureblade3808]

There are cases where the presence of this overload will produce different results than if it were absent.

I've made some experiments on this one, and have got -

from collections.abc import Iterable
from typing import Any, Generic, Never, TypeVar, overload
from typing_extensions import reveal_type


K = TypeVar("K")
V = TypeVar("V")
D = TypeVar("D")


class Some: pass


class MyDict0(Generic[K, V]):
    # Following two methods will help to ensure that both K and V are invariant.
    def keys(self, /) -> Iterable[K]: raise
    def __setitem__(self, key: K, value: V, /) -> None: raise

    @overload
    def get(self, key: K, /) -> V | None: raise
    @overload
    def get(self, key: K, default: D, /) -> V | D: raise
    def get(self, *args: Any) -> Any: raise


class MyDict1(Generic[K, V]):
    # Following two methods will help to ensure that both K and V are invariant.
    def keys(self, /) -> Iterable[K]: raise
    def __setitem__(self, key: K, value: V, /) -> None: raise

    @overload
    def get(self, key: K, /) -> V | None: raise
    @overload
    def get(self, key: K, default: V, /) -> V: raise
    @overload
    def get(self, key: K, default: D, /) -> V | D: raise
    def get(self, *args: Any) -> Any: raise


def _(
    dict_: dict[K, V],
    my_dict_0: MyDict0[K, V],
    my_dict_1: MyDict1[K, V],
    key: K,
    value: V,
    any_: Any,
    some: Some,
    top: object,
    bottom: Never,
) -> None:
    _ = reveal_type(dict_.get(key))  # Mypy: `V | None`  # Pyright: `V | None`
    _ = reveal_type(my_dict_0.get(key))  # Mypy: `V | None`  # Pyright: `V | None`
    _ = reveal_type(my_dict_1.get(key))  # Mypy: `V | None`  # Pyright: `V | None`

    _ = reveal_type(dict_.get(key, None))  # Mypy: `V | Any`  # Pyright: `V | None`
    _ = reveal_type(my_dict_0.get(key, None))  # Mypy: `V | Any`  # Pyright: `V | None`
    _ = reveal_type(my_dict_1.get(key, None))  # Mypy: `V | Any`  # Pyright: `V | None`

    _ = reveal_type(dict_.get(key, key))  # Mypy: `V | Any`  # Pyright: `V | K`
    _ = reveal_type(my_dict_0.get(key, key))  # Mypy: `V | Any`  # Pyright: `V | K`
    _ = reveal_type(my_dict_1.get(key, key))  # Mypy: `V | Any`  # Pyright: `V | K`

    _ = reveal_type(dict_.get(key, value))  # Mypy: `V`  # Pyright: `V`
    _ = reveal_type(my_dict_0.get(key, value))  # Mypy: `V | Any`  # Pyright: `V`
    _ = reveal_type(my_dict_1.get(key, value))  # Mypy: `V``  # Pyright: `V`

    _ = reveal_type(dict_.get(key, any_))  # Mypy: `Any`  # Pyright: `V``
    _ = reveal_type(my_dict_0.get(key, any_))  # Mypy: `V | Any`  # Pyright: `V | Any``
    _ = reveal_type(my_dict_1.get(key, any_))  # Mypy: `Any`  # Pyright: `V``

    _ = reveal_type(dict_.get(key, some))  # Mypy: `V | Any`  # Pyright: `V | Some`
    _ = reveal_type(my_dict_0.get(key, some))  # Mypy: `V | Any`  # Pyright: `V | Some`
    _ = reveal_type(my_dict_1.get(key, some))  # Mypy: `V | Any`  # Pyright: `V | Some`

    _ = reveal_type(dict_.get(key, top))  # Mypy: `V | Any`  # Pyright: `V | object`
    _ = reveal_type(my_dict_0.get(key, top))  # Mypy: `V | Any`  # Pyright: `V | object`
    _ = reveal_type(my_dict_1.get(key, top))  # Mypy: `V | Any`  # Pyright: `V | object`

    _ = reveal_type(dict_.get(key, bottom))  # Mypy: `V`  # Pyright: `V`
    _ = reveal_type(my_dict_0.get(key, bottom))  # Mypy: `V | Any`  # Pyright: `V`
    _ = reveal_type(my_dict_1.get(key, bottom))  # Mypy: `V`  # Pyright: `V`

On Pyright side, the only difference is between my_dict_0.get(key, any_) being V | Any and my_dict_1.get(key, any_) being V, in which the former is more correct in my mind.

But on Mypy side, my_dict_0.get(key, ...) always being V | Any is very problematic, and I believe this is why dict.get is declared in its current form.

So, the real problem lies in dict.get not being fully compatible with Mapping.get, and it doesn't seem feasible to modify typeshed to erase this incompatibility. Sigh.