Repository navigation
update the import machinery to only use __spec__ #65961
Description
Activity
With PEP-451, Python 3.4 introduced module specs to encapsulate the module's import-related information, particularly for loading. While __loader__, __file__, and __cached__ are no longer used by the import machinery, in a few places it still uses __name__, __package__, and __path__.
Typically the spec and the module attrs will have the same values, so it would be a non-issue. However, the import-related module attributes are not read-only and the consequences of changing them (i.e. accidentally or to rely on an implementation detail) are not clearly defined. Making the spec strictly authoritative reduces the likelihood accidental changes and gives a better focus point for a module's import behavior (which was kind of the point of PEP-451 in the first place). Furthermore, objects in sys.modules are not required to be modules. By relying strictly on __spec__ we likewise give a more distinct target (of import-related info) for folks that need to use that trick.
I don't recall the specifics on why we didn't change those 3 attributes for PEP-451 (unintentional or for backward compatibility?). At one point we discussed the idea that a module's spec contains the values that *were* used to load the module. Instead, each spec became the image of how the import system sees and treats the module.
So unless there's some solid reason, I'd like to see the use of __name__, __package__, and __path__ by the import machinery eliminated (and accommodated separately if appropriate). Consistent use of specs in the import machinery will help limit future surprises.
Here are the specific places:
__name__
--------mod.__repr__() ExtensionFileLoader.load_module() importlib._bootstrap._handle_fromlist() importlib._bootstrap._calc___package__() importlib._bootstrap.__import__()
__package__
-----------importlib._bootstrap._calc___package__()
__path__
--------importlib._bootstrap._find_and_load_unlocked() importlib._bootstrap._handle_fromlist() importlib._bootstrap._calc___package__()
__file__
--------mod.__repr__()
Note that I'm not suggesting the module attributes be eliminated (they are useful for informational reasons). I would just like the import system to stop using them. I suppose they could be turned into read-only properties, but anything like that should be addressed in a separate issue.
If we do make this change, the language reference, importlib docs, and inspect docs should be updated to clearly reflect the role of the module attributes in the import system.
I have not taken into account the impact on the standard library. However, I expect that it will be minimal at best. (See issue bpo-21736 for a related discussion).
Manipulating name, package and path at runtime is fully supported, and the
module level attributes accordingly take precedence over the initial import
time spec.There may be some test suite gaps and documentation issues around the
behaviour, but it's definitely intentional (things like runpy,
"pseudo-modules", third party namespace package support and workarounds for
running modules inside packages correctly rely on it).Thanks for clarifying. I remembered discussing it but couldn't recall the details. Documenting the exact semantics, use cases, and difference between spec and module attrs would be help. I'll look into updating the language reference when I have some time.
It would still be worth it to find a way to make __spec__ fully authoritative, but I'll have to come up with a solution to the current use cases before that could go anywhere. :)
The spec is authoritative for "how was this imported?". The differences between that and the module attributes then provide a record of any post-import modifications.
So I am going to disagree with Nick about the module attributes and their usefulness (partially because I just made __spec__.parent take precedence over __package__ in issue bpo-25791). While I get the idea of wanting a history of what did (not) change since import, keeping the duplicate information around is annoying. And I don't know how truly useful it is to know what things were compared to what they became.
If we shift to preferring specs compared to module attributes we can then begin to clean up __import__ itself by no longer grabbing the globals() and locals() and instead simply pass in the module's __spec__ object. It also simplifies the documentation such that we don't have to explain everything twice. If people really want to track what a value was relating to import before mutation they can simply store it themselves instead of making us do the bookkeeping for them. It would also make things such as module_from_spec() or loader.create_module() simpler since they only have to worry about setting __spec__ instead of that attribute plus a bunch of other ones.
My concern is more about backwards compatibility - at the moment, you can alter the behaviour of import, pickle, and other subsystems by modifying the module level attributes, and if we switch to preferring the __spec__ attributes, then that kind of code will break (I added an import specific example related to __main__ module relative imports to the linked issue).
That's not to say it shouldn't be done - as you say, it would be nice to eventually get to a point where the import system only needs access to the module spec and not to the runtime state, and there are also cases where the __spec__ information will be more correct (e.g. pickling objects in __main__).
However, it needs to be in such a way that there are appropriate porting notes that explain to people why their state mutations stopped having the desired effect, and what (if anything) they can do instead.
I totally agree proper notes in the What's New doc need to be in place to explain that people need to update.
How about I tweak the __package__ change to continue to prefer __package__ over __spec__.parent, but raise an ImportWarning when they differ? It can also fall back to __spec__.parent if __package__ isn't defined and simply not worry about the lack of __package__? Then we can do an ImportWarning for all of the other attributes when we discover a difference so people have time to migrate to updating both locations, and then eventually we can invert the priority and then after that drop the module attributes.
That approach sounds good to me.
The main problem cases I'm aware of are:
__name__:
- reliably wrong in __main__
- the attribute you mess with if you want __qualname__ on functions and classes to be different so that pickle will import them from somewhere else (e.g. a parent package)
__path__:
- used for dynamic package definitions (including namespace package emulation)
__package__:
- AFAIK, mainly useful as a workaround for other people doing "bad things" (TM), like running package submodules directly as __main__ rather than using the -m switch
Yeah, which is why it will take a transition to get people to start mucking with spec instead of the module attributes for their legitimate/questionable needs (although the whole
__name__ == '__main__'idiom means name will never go away while path, package, loader, file, and cached could all slowly go away).__package__ != __spec__.parent now triggers an ImportWarning.
I think that leaves the following attributes to be updated/checked for dependencies in importlib (and if they are found, raise ImportWarning when they differ):
- __path__
- __loader__
- __file__
- __cached__
It turns out that the import system itself doesn't use
__loader__(it does set it), but various parts of the stdlib do use__loader__. bpo-42133 updates a bunch of stdlib modules to use__spec__.loaderas a fallback. bpo-42132 tracks the fact that there's C code which isn't setting or using__spec__(let alone__spec__.loader).30 remaining items
Way back when, I had a branch that proxied the reads of the old attributes through the
__spec__. I'm sure it's woefully out of date (if I can even find it again), but IIRC it did work for reading the old attributes. I got hung up on setting through the attributes, but we probably wouldn't need that. I wonder if it's worth trying to resurrect that?I wonder if it's worth trying to resurrect that?
I think it's worth looking at, but I'm afraid the point @ncoghlan made about how reading
__package__within the module itself circumvents attribute lookup on a module object as it comes directly from the dict as passed intoexec(). I think we would need to have a custom namespace object for__dict__or something in order to make something work in all scenarios.I wonder if it's worth trying to resurrect that?
I think we would need to have a custom namespace object for
__dict__or something in order to make something work in all scenarios.And the reasons the lazy import PEP ruled that out as not being worth the complexity unfortunately apply here as well.
__cached__is now gone!Hello, I was going through the pending removal in 3.15 file, and I found one thing was not completed, that is:
cpython/Doc/deprecations/pending-removal-in-3.15.rst
Lines 11 to 14 in 46a2c11
I wrote #149594 to move it 3.16.
I was going through the pending removal in 3.15 file, and I found one thing was not completed
It's not not completed either. 😉 While
__package__is still read as a fallback, it is no longer set.Should we add warnings._deprecated() landmines in the code so we don't forget to remove them again? 😄
As the person who had to deal with those land mines (i.e., the tests failing when we hit the set version) during the recent 3.16 bump, please don’t, a landmine shouldn’t take out a whole corps ;-)
As the person who had to deal with those land mines (i.e., the tests failing when we hit the set version) during the recent 3.16 bump, please don’t, a landmine shouldn’t take out a whole corps ;-)
Sorry about that! I know it's a PITA, but also kind of the point 😜. Maybe there's a better way to kick ourselves during each new release branch.
- added a commit that references this issue
on Aug 27, 2026
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
__package__and__cached__#124377__package__and__cached__(GH-124377) #124380__package__on modules #142046__package__on modules #142055__cached__on modules #142165