Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interpret type annotations as type[Any] annotations #16366

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a430c12
Interpret `type` annotations as `type[Any]` annotations.
tyralla Oct 29, 2023
56fb2bd
Replace `type` with `type[object]` in Mypy's code.
tyralla Nov 13, 2023
fe0cd0a
Find other places where `analyze_annotation` should eventually be set…
tyralla Nov 15, 2023
28ee26b
* Rename `analyze_annotation` to `builtin_type_is_type_type` and mak…
tyralla Nov 21, 2023
84d06cb
Merge branch 'master' into fix/narrow_type_vs_Type
tyralla Nov 21, 2023
dcf1f95
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 21, 2023
b1bb26c
small fixes
tyralla Nov 21, 2023
339f0b9
Merge branch 'fix/narrow_type_vs_Type' of https://github.com/tyralla/…
tyralla Nov 21, 2023
80a8c67
minor fixes after review
tyralla Nov 21, 2023
ab7322c
Add type[ignore] to tuple.pyi to satisfy testDisallowAnyExplicitGener…
tyralla Nov 21, 2023
aa0cd97
Add error codes to the type: ignores in tuple.pyi (and tuple.pyi) to …
tyralla Nov 21, 2023
dcd12ba
Consider TypeType as hashable if (if no metaclass sets __hash__ to None)
tyralla Nov 24, 2023
e25f8e8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 24, 2023
ed0f652
Look for __hash__ instead of checking the name of the potential Hashable
tyralla Nov 24, 2023
a46e5d9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 24, 2023
721465a
Merge branch 'master' into fix/narrow_type_vs_Type
tyralla Nov 26, 2023
7d1d3f2
Add another `TypeVar`-related test to `testBuiltinTypeType`.
tyralla Nov 26, 2023
7107fb8
Add more test cases to `testBuiltinTypeType`.
tyralla Nov 27, 2023
3858928
Check more thoroughly if the `Protocol` is `Hashable`-like.
tyralla Nov 27, 2023
a09c6e2
Check more thoroughly if the signatures agree.
tyralla Nov 27, 2023
1a2202f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 27, 2023
4330ea0
remove unused local symtab variable
tyralla Dec 11, 2023
d6e4338
Merge remote-tracking branch 'upstream/master' into fix/narrow_type_v…
hauntsaninja Nov 3, 2024
c7dbe6a
fix merge
hauntsaninja Nov 3, 2024
5a3e028
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mypy/fastparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ def __init__(
self.type_ignores: dict[int, list[str]] = {}

# Cache of visit_X methods keyed by type of visited object
self.visitor_cache: dict[type, Callable[[AST | None], Any]] = {}
self.visitor_cache: dict[type[object], Callable[[AST | None], Any]] = {}

def note(self, msg: str, line: int, column: int) -> None:
self.errors.report(line, column, msg, severity="note", code=codes.SYNTAX)
Expand Down
4 changes: 4 additions & 0 deletions mypy/message_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,10 @@ def with_additional_msg(self, info: str) -> ErrorMessage:
IMPLICIT_GENERIC_ANY_BUILTIN: Final = (
'Implicit generic "Any". Use "{}" and specify generic parameters'
)
BUILTIN_TYPE_USED_AS_GENERIC = (
'"builtins.type" is indexable as a type hint but neither a generic class nor a generic '
"function"
)
INVALID_UNPACK: Final = "{} cannot be unpacked (must be tuple or TypeVarTuple)"
INVALID_UNPACK_POSITION: Final = "Unpack is only valid in a variadic position"
INVALID_PARAM_SPEC_LOCATION: Final = "Invalid location for ParamSpec {}"
Expand Down
9 changes: 8 additions & 1 deletion mypy/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,13 @@ def has_no_attr(
# Indexed get.
# TODO: Fix this consistently in format_type
if isinstance(original_type, FunctionLike) and original_type.is_type_obj():
if (
isinstance(original_type, CallableType)
and isinstance(ret_type := get_proper_type(original_type.ret_type), Instance)
and (ret_type.type.fullname == "builtins.type")
):
self.fail(message_registry.BUILTIN_TYPE_USED_AS_GENERIC, context)
return None
self.fail(
"The type {} is not generic and not indexable".format(
format_type(original_type, self.options)
Expand Down Expand Up @@ -2113,7 +2120,7 @@ def report_protocol_problems(
# note: method, attr
MAX_ITEMS = 2 # Maximum number of conflicts, missing members, and overloads shown
# List of special situations where we don't want to report additional problems
exclusions: dict[type, list[str]] = {
exclusions: dict[type[object], list[str]] = {
TypedDictType: ["typing.Mapping"],
TupleType: ["typing.Iterable", "typing.Sequence"],
}
Expand Down
2 changes: 1 addition & 1 deletion mypy/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -3509,7 +3509,7 @@ class FakeInfo(TypeInfo):
def __init__(self, msg: str) -> None:
self.msg = msg

def __getattribute__(self, attr: str) -> type:
def __getattribute__(self, attr: str) -> type[object]:
# Handle __class__ so that isinstance still works...
if attr == "__class__":
return object.__getattribute__(self, attr) # type: ignore[no-any-return]
Expand Down
30 changes: 25 additions & 5 deletions mypy/semanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -2511,7 +2511,10 @@ def analyze_base_classes(

try:
base = self.expr_to_analyzed_type(
base_expr, allow_placeholder=True, allow_type_any=True
base_expr,
allow_placeholder=True,
allow_type_any=True,
builtin_type_is_type_type=not refers_to_fullname(base_expr, "builtins.type"),
)
except TypeTranslationError:
name = self.get_name_repr_of_expr(base_expr)
Expand Down Expand Up @@ -2561,10 +2564,14 @@ def configure_base_classes(
elif isinstance(base, TypedDictType):
base_types.append(base.fallback)
else:
msg = "Invalid base class"
name = self.get_name_repr_of_expr(base_expr)
if name:
msg += f' "{name}"'
if isinstance(base_expr, IndexExpr) and refers_to_fullname(
base_expr.base, "builtins.type"
):
msg = message_registry.BUILTIN_TYPE_USED_AS_GENERIC
else:
msg = "Invalid base class"
if name := self.get_name_repr_of_expr(base_expr):
msg += f' "{name}"'
self.fail(msg, base_expr)
info.fallback_to_any = True
if self.options.disallow_any_unimported and has_any_from_unimported_type(base):
Expand Down Expand Up @@ -3827,6 +3834,7 @@ def analyze_alias(
name: str,
rvalue: Expression,
allow_placeholder: bool = False,
builtin_type_is_type_type: bool = False,
declared_type_vars: TypeVarLikeList | None = None,
all_declared_type_params_names: list[str] | None = None,
python_3_12_type_alias: bool = False,
Expand Down Expand Up @@ -3876,6 +3884,7 @@ def analyze_alias(
in_dynamic_func=dynamic,
global_scope=global_scope,
allowed_alias_tvars=tvar_defs,
builtin_type_is_type_type=builtin_type_is_type_type,
alias_type_params_names=all_declared_type_params_names,
python_3_12_type_alias=python_3_12_type_alias,
)
Expand Down Expand Up @@ -3995,6 +4004,9 @@ def check_and_set_up_type_alias(self, s: AssignmentStmt) -> bool:
lvalue.name,
rvalue,
allow_placeholder=True,
builtin_type_is_type_type=(
(s.type is not None) or not refers_to_fullname(s.rvalue, "builtins.type")
),
declared_type_vars=type_params,
all_declared_type_params_names=all_type_params_names,
)
Expand Down Expand Up @@ -7252,6 +7264,7 @@ def expr_to_analyzed_type(
allow_unbound_tvars: bool = False,
allow_param_spec_literals: bool = False,
allow_unpack: bool = False,
builtin_type_is_type_type: bool = True,
) -> Type | None:
if isinstance(expr, CallExpr):
# This is a legacy syntax intended mostly for Python 2, we keep it for
Expand Down Expand Up @@ -7283,6 +7296,9 @@ def expr_to_analyzed_type(
allow_unbound_tvars=allow_unbound_tvars,
allow_param_spec_literals=allow_param_spec_literals,
allow_unpack=allow_unpack,
builtin_type_is_type_type=(
builtin_type_is_type_type or not refers_to_fullname(expr, "builtins.type")
),
)

def analyze_type_expr(self, expr: Expression) -> None:
Expand All @@ -7308,6 +7324,7 @@ def type_analyzer(
report_invalid_types: bool = True,
prohibit_self_type: str | None = None,
allow_type_any: bool = False,
builtin_type_is_type_type: bool = True,
) -> TypeAnalyser:
if tvar_scope is None:
tvar_scope = self.tvar_scope
Expand All @@ -7327,6 +7344,7 @@ def type_analyzer(
allow_unpack=allow_unpack,
prohibit_self_type=prohibit_self_type,
allow_type_any=allow_type_any,
builtin_type_is_type_type=builtin_type_is_type_type,
)
tpan.in_dynamic_func = bool(self.function_stack and self.function_stack[-1].is_dynamic())
tpan.global_scope = not self.type and not self.function_stack
Expand All @@ -7351,6 +7369,7 @@ def anal_type(
report_invalid_types: bool = True,
prohibit_self_type: str | None = None,
allow_type_any: bool = False,
builtin_type_is_type_type: bool = True,
) -> Type | None:
"""Semantically analyze a type.

Expand Down Expand Up @@ -7386,6 +7405,7 @@ def anal_type(
report_invalid_types=report_invalid_types,
prohibit_self_type=prohibit_self_type,
allow_type_any=allow_type_any,
builtin_type_is_type_type=builtin_type_is_type_type,
)
tag = self.track_incomplete_refs()
typ = typ.accept(a)
Expand Down
12 changes: 8 additions & 4 deletions mypy/stubgenc.py
Original file line number Diff line number Diff line change
Expand Up @@ -758,7 +758,7 @@ def generate_property_stub(

rw_properties.append(f"{self._indent}{name}: {inferred_type}")

def get_type_fullname(self, typ: type) -> str:
def get_type_fullname(self, typ: type[object]) -> str:
"""Given a type, return a string representation"""
if typ is Any: # type: ignore[comparison-overlap]
return "Any"
Expand All @@ -769,7 +769,7 @@ def get_type_fullname(self, typ: type) -> str:
typename = f"{module_name}.{typename}"
return typename

def get_base_types(self, obj: type) -> list[str]:
def get_base_types(self, obj: type[object]) -> list[str]:
all_bases = type.mro(obj)
if all_bases[-1] is object:
# TODO: Is this always object?
Expand All @@ -781,14 +781,18 @@ def get_base_types(self, obj: type) -> list[str]:
# remove the class itself
all_bases = all_bases[1:]
# Remove base classes of other bases as redundant.
bases: list[type] = []
bases: list[type[object]] = []
for base in all_bases:
if not any(issubclass(b, base) for b in bases):
bases.append(base)
return [self.strip_or_import(self.get_type_fullname(base)) for base in bases]

def generate_class_stub(
self, class_name: str, cls: type, output: list[str], parent_class: ClassInfo | None = None
self,
class_name: str,
cls: type[object],
output: list[str],
parent_class: ClassInfo | None = None,
) -> None:
"""Generate stub for a single class using runtime introspection.

Expand Down
2 changes: 1 addition & 1 deletion mypy/stubutil.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ def __init__(
name: str,
self_var: str,
docstring: str | None = None,
cls: type | None = None,
cls: type[object] | None = None,
parent: ClassInfo | None = None,
) -> None:
self.name = name
Expand Down
24 changes: 24 additions & 0 deletions mypy/subtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -1098,6 +1098,30 @@ def visit_type_type(self, left: TypeType) -> bool:
# of x is Type[int]. It's unclear what's the right way to address this.
return True
item = left.item
if (
right.type.is_protocol
and len(right.type.protocol_members) == 1
and right.type.protocol_members[0] == "__hash__"
and (symtab := right.type.get("__hash__")) is not None
and isinstance(hash_ := get_proper_type(symtab.type), CallableType)
and len(hash_.arg_names) == 1
and hash_.arg_names[0] == "self"
and isinstance(ret := get_proper_type(hash_.ret_type), Instance)
and ret.type.fullname == "builtins.int"
):
if isinstance(item, AnyType):
return True
if isinstance(item, Instance):
if (mtype := item.type.metaclass_type) is None or (
mtype.type.get("__hash__") is None
):
return True
supertype = get_proper_type(find_member("__hash__", right, mtype))
assert supertype is not None
subtype = mypy.typeops.get_protocol_member(mtype, "__hash__", False)
assert subtype is not None
if is_subtype(subtype, supertype, ignore_pos_arg_names=True):
return True
if isinstance(item, TypeVarType):
item = get_proper_type(item.upper_bound)
if isinstance(item, Instance):
Expand Down
4 changes: 2 additions & 2 deletions mypy/test/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -313,14 +313,14 @@ def assert_equal(a: object, b: object, fmt: str = "{} != {}") -> None:
raise AssertionError(fmt.format(good_repr(a), good_repr(b)))


def typename(t: type) -> str:
def typename(t: type[object]) -> str:
if "." in str(t):
return str(t).split(".")[-1].rstrip("'>")
else:
return str(t)[8:-2]


def assert_type(typ: type, value: object) -> None:
def assert_type(typ: type[object], value: object) -> None:
__tracebackhide__ = True
if type(value) != typ:
raise AssertionError(f"Invalid type {typename(type(value))}, expected {typename(typ)}")
Expand Down
8 changes: 7 additions & 1 deletion mypy/typeanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ def analyze_type_alias(
in_dynamic_func: bool = False,
global_scope: bool = True,
allowed_alias_tvars: list[TypeVarLikeType] | None = None,
builtin_type_is_type_type: bool = True,
alias_type_params_names: list[str] | None = None,
python_3_12_type_alias: bool = False,
) -> tuple[Type, set[str]]:
Expand All @@ -175,6 +176,7 @@ def analyze_type_alias(
allow_placeholder=allow_placeholder,
prohibit_self_type="type alias target",
allowed_alias_tvars=allowed_alias_tvars,
builtin_type_is_type_type=builtin_type_is_type_type,
alias_type_params_names=alias_type_params_names,
python_3_12_type_alias=python_3_12_type_alias,
)
Expand Down Expand Up @@ -231,6 +233,7 @@ def __init__(
prohibit_self_type: str | None = None,
allowed_alias_tvars: list[TypeVarLikeType] | None = None,
allow_type_any: bool = False,
builtin_type_is_type_type: bool = True,
alias_type_params_names: list[str] | None = None,
) -> None:
self.api = api
Expand Down Expand Up @@ -279,6 +282,7 @@ def __init__(
self.allow_type_any = allow_type_any
self.allow_type_var_tuple = False
self.allow_unpack = allow_unpack
self.builtin_type_is_type_type = builtin_type_is_type_type

def lookup_qualified(
self, name: str, ctx: Context, suppress_errors: bool = False
Expand Down Expand Up @@ -645,7 +649,9 @@ def try_analyze_special_unbound_type(self, t: UnboundType, fullname: str) -> Typ
and (self.always_allow_new_syntax or self.options.python_version >= (3, 9))
):
if len(t.args) == 0:
if fullname == "typing.Type":
if fullname == "typing.Type" or (
self.builtin_type_is_type_type and (fullname == "builtins.type")
):
any_type = self.get_omitted_any(t)
return TypeType(any_type, line=t.line, column=t.column)
else:
Expand Down
9 changes: 4 additions & 5 deletions test-data/unit/check-classes.test
Original file line number Diff line number Diff line change
Expand Up @@ -2287,7 +2287,7 @@ tmp/foo.pyi:8: note: def __add__(self, int, /) -> A
tmp/foo.pyi:8: note: @overload
tmp/foo.pyi:8: note: def __add__(self, str, /) -> A
tmp/foo.pyi:8: note: @overload
tmp/foo.pyi:8: note: def __add__(self, type, /) -> A
tmp/foo.pyi:8: note: def __add__(self, Type[Any], /) -> A
tmp/foo.pyi:8: note: Overloaded operator methods can't have wider argument types in overrides

[case testOverloadedOperatorMethodOverrideWithSwitchedItemOrder]
Expand Down Expand Up @@ -3580,7 +3580,7 @@ def foo(arg: Type[Any]):
from typing import Type, Any
def foo(arg: Type[Any]):
reveal_type(arg.__str__) # N: Revealed type is "def () -> builtins.str"
reveal_type(arg.mro()) # N: Revealed type is "builtins.list[builtins.type]"
reveal_type(arg.mro()) # N: Revealed type is "builtins.list[Type[Any]]"
[builtins fixtures/type.pyi]
[out]

Expand Down Expand Up @@ -3920,7 +3920,7 @@ def f(a: type) -> None: pass
f(3) # E: No overload variant of "f" matches argument type "int" \
# N: Possible overload variants: \
# N: def f(a: Type[User]) -> None \
# N: def f(a: type) -> None
# N: def f(a: Type[Any]) -> None
[builtins fixtures/classmethod.pyi]
[out]

Expand Down Expand Up @@ -5651,8 +5651,7 @@ def f() -> type: return M
class C1(six.with_metaclass(M), object): pass # E: Unsupported dynamic base class "six.with_metaclass"
class C2(C1, six.with_metaclass(M)): pass # E: Unsupported dynamic base class "six.with_metaclass"
class C3(six.with_metaclass(A)): pass # E: Metaclasses not inheriting from "type" are not supported
@six.add_metaclass(A) # E: Metaclasses not inheriting from "type" are not supported \
# E: Argument 1 to "add_metaclass" has incompatible type "Type[A]"; expected "Type[type]"
@six.add_metaclass(A) # E: Metaclasses not inheriting from "type" are not supported

class D3(A): pass
class C4(six.with_metaclass(M), metaclass=M): pass # E: Multiple metaclass definitions
Expand Down
2 changes: 1 addition & 1 deletion test-data/unit/check-generic-alias.test
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ reveal_type(t6) # N: Revealed type is "Tuple[builtins.int, builtins.str]"
reveal_type(t7) # N: Revealed type is "builtins.tuple[builtins.int, ...]"
reveal_type(t8) # N: Revealed type is "builtins.dict[Any, Any]"
reveal_type(t9) # N: Revealed type is "builtins.dict[builtins.int, builtins.str]"
reveal_type(t10) # N: Revealed type is "builtins.type"
reveal_type(t10) # N: Revealed type is "Type[Any]"
reveal_type(t11) # N: Revealed type is "Type[builtins.int]"
[builtins fixtures/dict.pyi]

Expand Down
4 changes: 1 addition & 3 deletions test-data/unit/check-generics.test
Original file line number Diff line number Diff line change
Expand Up @@ -515,9 +515,7 @@ Alias[int]("a") # E: Argument 1 to "Node" has incompatible type "str"; expected
[out]

[case testTypeApplicationCrash]
import types
type[int] # this was crashing, see #2302 (comment) # E: The type "Type[type]" is not generic and not indexable
[builtins fixtures/tuple.pyi]
type[int] # this was crashing, see #2302 (comment) # E: "builtins.type" is indexable as a type hint but neither a generic class nor a generic function
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved


-- Generic type aliases
Expand Down
2 changes: 1 addition & 1 deletion test-data/unit/check-isinstance.test
Original file line number Diff line number Diff line change
Expand Up @@ -1907,7 +1907,7 @@ if issubclass(y): # E: Missing positional argument "t" in call to "issubclass"

[case testIsInstanceTooManyArgs]
isinstance(1, 1, 1) # E: Too many arguments for "isinstance" \
# E: Argument 2 to "isinstance" has incompatible type "int"; expected "Union[type, Tuple[Any, ...]]"
# E: Argument 2 to "isinstance" has incompatible type "int"; expected "Union[Type[Any], Tuple[Any, ...]]"
x: object
if isinstance(x, str, 1): # E: Too many arguments for "isinstance"
reveal_type(x) # N: Revealed type is "builtins.object"
Expand Down
4 changes: 2 additions & 2 deletions test-data/unit/check-literal.test
Original file line number Diff line number Diff line change
Expand Up @@ -1486,13 +1486,13 @@ Alias = Literal[3]

isinstance(3, Literal[3]) # E: Cannot use isinstance() with Literal type
isinstance(3, Alias) # E: Cannot use isinstance() with Literal type \
# E: Argument 2 to "isinstance" has incompatible type "<typing special form>"; expected "Union[type, Tuple[Any, ...]]"
# E: Argument 2 to "isinstance" has incompatible type "<typing special form>"; expected "Union[Type[Any], Tuple[Any, ...]]"
isinstance(3, Renamed[3]) # E: Cannot use isinstance() with Literal type
isinstance(3, indirect.Literal[3]) # E: Cannot use isinstance() with Literal type

issubclass(int, Literal[3]) # E: Cannot use issubclass() with Literal type
issubclass(int, Alias) # E: Cannot use issubclass() with Literal type \
# E: Argument 2 to "issubclass" has incompatible type "<typing special form>"; expected "Union[type, Tuple[Any, ...]]"
# E: Argument 2 to "issubclass" has incompatible type "<typing special form>"; expected "Union[Type[Any], Tuple[Any, ...]]"
issubclass(int, Renamed[3]) # E: Cannot use issubclass() with Literal type
issubclass(int, indirect.Literal[3]) # E: Cannot use issubclass() with Literal type
[builtins fixtures/isinstancelist.pyi]
Expand Down
4 changes: 2 additions & 2 deletions test-data/unit/check-lowercase.test
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ x = 3 # E: Incompatible types in assignment (expression has type "int", variabl

[case testTypeLowercaseSettingOff]
# flags: --python-version 3.9 --no-force-uppercase-builtins
x: type[type]
x: type[type] # E: Type[...] can't contain another Type[...]
y: int

y = x # E: Incompatible types in assignment (expression has type "type[type]", variable has type "int")
y = x # E: Incompatible types in assignment (expression has type "type[Any]", variable has type "int")

[case testLowercaseSettingOnTypeAnnotationHint]
# flags: --python-version 3.9 --no-force-uppercase-builtins
Expand Down
Loading
Loading