Repository navigation
sys.monitoring PY_UNWIND not being delivered when a generator is closed/garbage collected #140373
Description
Bug report
Bug description:
With Python 3.12, this code generates a PY_UNWIND event; with 3.13 and later, it no longer does.
import sys
def gen():
yield 1
yield 2
def callback(code, off, exc):
if code is gen.__code__:
print("got unwind!")
ID=3
sysmon = sys.monitoring
sysmon.use_tool_id(ID, "test")
sysmon.register_callback(ID, sysmon.events.PY_UNWIND, callback)
sysmon.set_events(ID, sysmon.events.PY_UNWIND)
g = gen()
next(g)
del g # or g.close()CPython versions tested on:
3.13, 3.14
Operating systems tested on:
macOS, Linux
Activity
efimov-mikhail commentedon Oct 20, 2025
Thanks for the report, confirmed on 3.13, 3.14 and main.
Zheaoli commentedon Oct 20, 2025
Seems related with https://github.com/python/cpython/pull/111069/files
Zheaoli commentedon Oct 20, 2025
https://github.com/python/cpython/blob/main/Objects/genobject.c#L413-L420
Seems we add a short path to just return instead of raise an exception when there is only one level generator
markshannon commentedon Oct 21, 2025
What happening is that in #111069 we skipped the expensive raise then catch if the generator is not in a try block, as there would (or so we thought) be no observable effect.
But, if there is no exception raised, there is no unwinding, and no
PY_UNWINDevent.This is fine for finalization, as the language is quite clear that finalization may never happen,
but the docs explicitly state that an exception will be raised when
generator.close()is called.If we are to fix this for
generator.close()we might as well do it for finalization for consistency.My intended purpose of
PY_UNWINDis that profilers and other tools can balance entries and exits to functions: for everyPY_STARTorPY_RESUMEthere should be aPY_RETURN,PY_YIELDorPY_UNWIND, but it does seem that you ought to be able to use the event to detect generator close/finalization.So, do we add the additional check for instrumentation in
generator.close()or do we update the docs to say that the exception may not be raised if the generator is not currently executing in atryblock?@jaltmayerpizzorno how did you notice this, and is it breaking anything?
jaltmayerpizzorno commentedon Oct 21, 2025
Hi @markshannon! RightTyper uses
PY_STARTandPY_UNWIND; thePY_UNWINDis useful for detecting (in the absence ofPY_RETURN) when a function doesn't return a value, or when a stack frame is done, so that it doesn't possibly take it for another call that happened to reuse the frame.To observe types, RightTyper executes the target program (using
runpy). Once that execution ends, generators (but apparently not async generators or coroutines) may remain running. RightTyper used toclose()them to havePY_UNWINDprocess normally. I've now changed that to athrow(GeneratorExit), which works around the issue for us... but I thought I'd raise the issue anyway (pun partly intended, sorry).I think generally it's useful to be able to tell when a call is done (by getting either
PY_RETURNandPY_UNWIND), but you'd know best what it costs to support it.markshannon commentedon Oct 22, 2025
It sounds like it is being relied on, and it is in the spec. So, we'd better fix it.
It's not a complicated fix and I expect the performance impact will not be measurable, and definitely not significant.
efimov-mikhail commentedon Oct 22, 2025
I'd like to prepare a fix for this.
jaltmayerpizzorno commentedon Oct 27, 2025
In short: yes; thank you!
gh-140373: Correctly emit `PY_UNWIND` event when generator is closed (G…
pythongh-140373: Correctly emit `PY_UNWIND` event when generator is c…
pythongh-140373: Correctly emit `PY_UNWIND` event when generator is c…
efimov-mikhail commentedon Nov 6, 2025
cc @hugovk
Could you please merge backports for this issue?
The CI is green and changes are little.
[3.14] gh-140373: Correctly emit `PY_UNWIND` event when generator is …
pythongh-140373: Correctly emit `PY_UNWIND` event when generator is c…
[3.13] gh-140373: Correctly emit PY_UNWIND event when generator is cl…