Repository navigation
IDLE is unable to open any .py files #104719
Description
With a fresh CPython build (be0c106), IDLE is unable to open any .py files.
To reproduce:
- Create an empty
.pyfile with the namerepro.py - Run
python -m idlelib repro.py
IDLE still seems able to create new .py files and save them; it just can't open pre-existing .py files right now.
Traceback observed
C:\Users\alexw\coding\cpython>python -m idlelib repro.py
Running Debug|x64 interpreter...
Traceback (most recent call last):
File "C:\Users\alexw\coding\cpython\Lib\runpy.py", line 198, in _run_module_as_main
return _run_code(code, main_globals, None,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\runpy.py", line 88, in _run_code
exec(code, run_globals)
File "C:\Users\alexw\coding\cpython\Lib\idlelib\__main__.py", line 7, in <module>
idlelib.pyshell.main()
File "C:\Users\alexw\coding\cpython\Lib\idlelib\pyshell.py", line 1640, in main
if flist.open(filename) is None:
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\idlelib\filelist.py", line 37, in open
edit = self.EditorWindow(self, filename, key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\idlelib\pyshell.py", line 135, in __init__
EditorWindow.__init__(self, *args)
File "C:\Users\alexw\coding\cpython\Lib\idlelib\editor.py", line 289, in __init__
self.set_indentation_params(is_py_src)
File "C:\Users\alexw\coding\cpython\Lib\idlelib\editor.py", line 1327, in set_indentation_params
i = self.guess_indent()
^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\idlelib\editor.py", line 1574, in guess_indent
opener, indented = IndentSearcher(self.text, self.tabwidth).run()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\idlelib\editor.py", line 1646, in run
save_tabsize = tokenize.tabsize
^^^^^^^^^^^^^^^^
AttributeError: module 'tokenize' has no attribute 'tabsize'Environment
(Given the cause of the bug, the environment details shouldn't really be relevant; but they're included here anyway, for completeness.)
Python 3.12.0a7+ (heads/main:be0c106789, May 21 2023, 12:00:27) [MSC v.1932 64 bit (AMD64)] on win32
Reproduces on a debug and non-debug build, FWIW.
Linked PRs
- gh-104719: Restore Tokenize module constants #104722
- gh-104719: IDLE - delete useless monkeypatch of tokenize #104726
- [3.11] gh-104719: IDLE - delete useless monkeypatch of tokenize (GH-104726) #104727
- gh-104719: IDLE - test existence of all tokenize references. #104767
- [3.12] gh-104719: IDLE - test existence of all tokenize references. (GH-104767) #104844
- [3.11] gh-104719: IDLE - test existence of all tokenize references. (GH-104767) #104845
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
AlexWaygood commentedon May 21, 2023
Bisects to 6715f91 (unsurprisingly 😄)
Cc. @mgmacias95, @pablogsal, @lysnikolaou, @isidentical
pablogsal commentedon May 21, 2023
Will check this today
pablogsal commentedon May 21, 2023
Looks like IDLE is using a bunch of non documented attributes of the tokenize module that are no longer used internally. The easier course of action is probably just restore these but note that there will be no guarantee that whatever idle is going with them will continue working.
I ran the buildbots and the CI before merge so another question is why CI didn't catch this.
AlexWaygood commentedon May 21, 2023
Do any of the buildbots run the test suite with
-u gui(or-u all), I wonder? A bunch of IDLE's tests actually open and close GUI windows, so they're not run unless you run the test suite with that option.AlexWaygood commentedon May 21, 2023
That doesn't seem relevant;
python -m test test_idle -u guipasses onmain, on both debug and non-debug builds. So it does seem like there's some missing test coverage here.pablogsal commentedon May 21, 2023
I checked a bit more how the constant that was eliminated is being used and seems that IDLE is monkey patching the value to modify the behaviour of the now deleted
_tokenizeprivate function to adapt something regarding tabsize.Unfortunately I don't think this is going to work and was out of contract anyway so I think the best course of action here is to eliminate that code from
idlelibas restoring the constant will not restore whatever was happening before.AlexWaygood commentedon May 21, 2023
FWIW, I can find other places in prominent open-source projects where these undocumented constants (which have now been removed) are used:
So it might be worth adding them back anyway, to limit the disruption caused by this change.
33 remaining items
pythongh-104719: IDLE - test editor.IndentSearcher
terryjreedy commentedon May 22, 2023
I decided that it would in general be a good idea to have test coverage of references to non-idlelib modules. I have a toktest branch that adds coverage of the class containing the failure and all tokenize references in editor.py. I will make a PR tomorrow with or without coverage of tokenize references in 2 other idlelib modules.
Yhg1s commentedon May 22, 2023
FWIW, whether to deprecate or document the unused globals isn't really a RM question, but considering the difficulty of emitting warnings for them, I'd rather not do that right now (but for 3.13 it's fine). Documenting them as for internal use only and/or deprecated can be done after b1.
gh-104719: IDLE - test existence of all tokenize references. (#104767)
pythongh-104719: IDLE - test existence of all tokenize references. (p…
pythongh-104719: IDLE - test existence of all tokenize references. (p…
[3.12] gh-104719: IDLE - test existence of all tokenize references. (G…
[3.11] gh-104719: IDLE - test existence of all tokenize references. (G…
terryjreedy commentedon May 24, 2023
IDLE now has a regression test that will fail if any tokenizer names change. I suggest we close this and someone open a new issue about the now obsolete constants.
AlexWaygood commentedon May 24, 2023
That sounds good to me. I'll try to open an issue later today.
Thanks @terryjreedy, @pablogsal and @mgmacias95!