Repository navigation
Assignment of Any does not get rid of Optional (add AnyUnion type?) #3526
Description
Activity
powwill no longer need to returnAnyin builtins. Instead, it could
return the much less broadAnyUnion[int, float].But doesn't #3501 (or similar) provides a better solution?
AnyUnion[X, None]would provide a per-variable opt-out of strict None
checking. This could be helpful for classes with delayed initialization where
a lot of effort would be needed to refactor them into a state that would work
properly with strict None checking.Do I understand correctly that this has been solved by PEP 526?
In general, I don't think that we really need this feature. What worries me is that in addition to making type system more complex it also can break transitivity of subtyping (or maybe better call this compatibility). IIUC your proposal, then
int <: AnyUnion[int, str] <: str. Currently, we have only one special type that does this --Any. Having many (user defined) types that break transitivity might lead to unexpected consequences.Do I understand correctly that this has been solved by PEP 526?
Probably it was only partially fixed by PEP 526. But for remaining cases one can just do something like:
UNINITIALIZED: Any = None class C: def __init__(self) -> None: self.complicated: int = UNINITIALIZED # existing code below while self.complicated is None: ...
I'm not sure this is needed all that often, but if we do, we could just spell it
Any--Any[A, B, C]is acceptable as a A, B or C, while plainAnyis unchanged. Alternatively, maybe this type is really the same asIntersection(python/typing#213)?Regardless I'm not sure that in your original motivating example we should do anything different. If you want to signal at the call site that
f()returns anint, you can writedef g(x: Optional[int]) -> int if x is None: y = f() # type: int x = y return x # OK
This is probably the same feature as 'unsafe unions' that have been discussed before (though I can't find the discussion -- maybe it was in person or over email). Unsafe unions are different from
Intersection. For example, unlike intersections, compatibility goes both ways:intis compatible withAnyUnion[int, str]and vice versa.My take is that this feature would complicate mypy significantly. Normal union types are already a very complicated feature, and they still aren't even fully implemented. It would be a likely cause of some user confusion, since we'd have two pretty similar but different concepts -- union types and these any/unsafe unions. If we decide to add intersection types at some point, there would be even more potential for confusion, since they are also similar in some ways to any unions. All in all, I don't think that there is enough evidence of the usefulness of the feature to support implementing it.
- changed the title
[-]Add `AnyUnion` type[/-][+]Assignment of Any does not get rid of Optional (add `AnyUnion` type?)[/+]on Jun 23, 2017 Solving the original issue is a priority because this may come up often when using ignored imports. I'm not convinced that adding
AnyUnionis the right course of action, though.Concerning the original example:
from typing import Optional def f(): # untyped function; often from silent import return 0 # always returns an int def g(x: Optional[int]) -> int if x is None: x = f() return x
what is wrong with
xhaving typeUnion[Any, int]after theifblock? Thus seems logical, initially it wasUnion[None, int]but then theNoneis replaced byAnyin theifblock byx = f(). Fortunately, such unions are not simplified anymore:x: Union[int, Any] reveal_type(x) # Revealed type is 'Union[builtins.int, Any]' x.whatever # Error: Item "int" of "Union[int, Any]" has no attribute "whatever"
Also,
Union[int, Any]is a (non-proper) subtype ofintso that there should be no error in the original example.@ilevkivskyi: That doesn't work, as variables don't change type in general. In particular, we want to maintain this behavior:
x = 0 # type: int x = any() reveal_type(x) # int
as variables don't change type in general
But there is the type binder that can already do this:
x: Optional[int] if x is None: x = int() reveal_type(x) # Revealed type is 'builtins.int'
Why can't it be special-cased to re-bind to
Anyinside unions? (I agree that binding toAnyin general is clearly bad.)Unless there's a really good reason, Unions should work as similarly as possible to non-Unions. I don't think this is a strong enough reason.
The binder only binds to subtypes of the declared type. If we change that, the declared type starts to become pretty meaningless.
- Nevertheless the bar for introducing a new concept like AnyUnion is way higher.
14 remaining items
We should check how many more Anys the simple proposal would introduce into real world codebases before merging it.
And yes, this would conflict with proposal 3 in #2008 -- it would need tweaks.
The option number 3 already mentions
Unclear interactions with
Anytypes.as a downside. Maybe I am missing something, but this could be a minor tweak. Something like: types are always bound on initial assignments unless r.h.s. is
Any.It's not that simple. Take this circumstance, for example:
x = [1, 2, 3] # type: Sequence[Any]We don't want to bind
List[int]forx, because we want to preserveAnys (as otherwise Anys in generics would be useless). We don't want to leave it asSequence[Any], because that would make it behave inconsistently from all other list assignments. We'd have to bindList[Any]as the type forx. This isn't necessarily terrible, but is at a minimum a little tricky.We'd have to bind
List[Any]as the type forx. This isn't necessarily terrible, but is at a minimum a little tricky.I think bind
List[Any]in this case is a reasonable thing to do. Indeed, general interactions withAnymay be complex. However, I would still propose to go with Jukka's simple solution (if it doesn't explode the number ofAnys) and figure out the detailed rules aboutAnylater, when we get to #2008.This seems to happen frequently enough that I'm bumping priority to "high". Also removing the "needs discussion" label -- let's special case
Nonefor now, as discussed above.I'm not 100% sure what the proposal is. From "discussed above" link it seems that we should make this work?
def f():... def g(x: Optional[int]) -> int: if x is None: x = f() reveal_type(x) # Union[int, Any] # <---- this is new (currently Optional[int]) reveal_type(x) # Union[int, Any] # Currently also Optional[int] return x # Ok # Currently an error (Incompatible return value type ...)
But how? And should the fix take into account that f() is unannotated, or should it work for anything of type
Any? Let's assume the latter. At the pointx = f()we do something special if certain conditions are met. What are those conditions? I'm guessing all of the following:- the target's type is known to be
Nonedue to aNonecheck recorded in the binder overriding someOptional[t] - the value's type is
Any
The action is then apparently to set the target's type to
Union[t, Any]. The rest will follow.Have I got that?
- the target's type is known to be
But how?
I think not much have to be done, the binder would do the things, but it is explicitly prohibited to do many things with
AnyType, for example inConditionalTypeBinder.assign_type:# Assigning an Any value doesn't affect the type to avoid false negativesI think we can just add an exception there, so that when
declared_typeis a union containingNoneandenclosing_typeisNone, we stillputthetypeeven if it is anAnyType.- added a commit that references this issue
on Sep 17, 2018 - added a commit that references this issue
on Sep 18, 2018
Consider this code (run with
--strict-optional):Mypy doesn't infer that
xis notNoneing. I think this a bug -- mypyshould be permissive here.
If we wanted to fix this, we'd run into a problem: what type should
xhaveafter
x = f()? We don't want it to beAny, because we don't want thatbehavior in the general case. It shouldn't be
int--f()could be intendedto unconditionally return
Noneinstead. To fix this, I think we need tointroduce a new special type, which I will call
AnyUnion.AnyUnion[A, B, C]would represent a type which is allowed to be used as any ofA,B, orC(similarly to howAnyis allowed to be used as any type). In this case,we could have
x's type bound toAnyUnion[int, None], which would fix thiserror.
Other helpful uses of
AnyUnion:powwill no longer need to returnAnyin builtins. Instead, it couldreturn the much less broad
AnyUnion[int, float].AnyUnion[X, None]would provide a per-variable opt-out of strict Nonechecking. This could be helpful for classes with delayed initialization where
a lot of effort would be needed to refactor them into a state that would work
properly with strict None checking.
Cons:
makes things slightly harder for users to understand.
currently works), but
AnyUniondoesn't help much with that.