3.11.5 regression: StreamWriter.__del__ fails if event loop is already closed

Bug report

Bug description:

PR #107650 added a StreamWriter.__del__ that emits a ResourceWarning if a StreamWriter is not closed by the time it is garbage collected, and it has been backported as 3.11.5. However, if the event loop has already been closed by the time this happens, it causes a RuntimeError message to be displayed. It's non-fatal because exceptions raised by __del__ are ignored, but it causes an error message to be displayed to the user (as opposed to ResourceWarning, which is only shown when one opts in).

Code like the following used to run without any visible error, but now prints a traceback (and does not report the ResourceWarning for the unclosed StreamWriter when run with -X dev):

#!/usr/bin/env python3

import asyncio

async def main():
    global writer
    reader, writer = await asyncio.open_connection("127.0.0.1", 22)

asyncio.run(main())

Output in 3.11.5:

Exception ignored in: <function StreamWriter.__del__ at 0x7fd54ee11080>
Traceback (most recent call last):
  File "/usr/lib/python3.11/asyncio/streams.py", line 396, in __del__
  File "/usr/lib/python3.11/asyncio/streams.py", line 344, in close
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 860, in close
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs

Make unit tests pass on Python 3.11.5

Same problem here

I have the same problem with 3.11.6. Thanks for reporting it.

Is there a workaround?

Is there a workaround?

Use gc.collect() may help

Here is my workaround.

@pytest.fixture
def event_loop():
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    loop.set_debug(True)
    yield loop
    gc.collect()
    loop.close()

@dpr-0

@dpr-0 Thanks, but just adding that to my program didn't prevent the messages.

@dpr-0

@dpr-0 Thanks, but just adding that to my program didn't prevent the messages.

Hope you can find the problem T^T

Same problem for python 3.12.0

Sorry for the inconvenience. Can you (or anyone else reading this) submit a PR for the main branch? We will then backport the PR to 3.12 and 3.11.

Sorry for the inconvenience. Can you (or anyone else reading this) submit a PR for the main branch? We will then backport the PR to 3.12 and 3.11.

What is the desired behaviour if the event loop is already closed when the StreamWriter is destroyed? Should __del__ just become a no-op?

[3.12] pythongh-109538: Avoid RuntimeError when StreamWriter is delet…

[3.12] gh-109538: Catch closed loop runtime error and issue warning (G…

[3.11] gh-109538: Catch closed loop runtime error and issue warning (G…

Did anyone find a solution?

Did anyone find a solution?

It's fixed in the main, 3.12 and 3.11 branches. It will be fixed in the next official releases of 3.11 and 3.12 from python.org (within 1-2 months).

Downgrade to python 11.4 due to regression python/cpython#109538

fix(tests): Fix hanging tests

Did anyone find a solution?

It's fixed in the main, 3.12 and 3.11 branches. It will be fixed in the next official releases of 3.11 and 3.12 from python.org (within 1-2 months).

Is this fix included in 3.11.7?
Because I'm still able to reproduce it with Python 3.11.7

Edit: nevermind, I used 3.11.6 by accident. All good

pythongh-109538: Avoid RuntimeError when StreamWriter is deleted with…

Just so you know, on python 3.11.9 with this setting for pytest:

[tool.pytest.ini_options]
asyncio_mode = "auto"
filterwarnings = ["error"]

the suite still raises an error from ignored warning:

    def _warn_teardown_exception(
        hook_name: str, hook_impl: HookImpl, e: BaseException
    ) -> None:
        msg = "A plugin raised an exception during an old-style hookwrapper teardown.\n"
        msg += f"Plugin: {hook_impl.plugin_name}, Hook: {hook_name}\n"
        msg += f"{type(e).__name__}: {e}\n"
        msg += "For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning"  # noqa: E501
>       warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5)
E       pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
E       Plugin: unraisableexception, Hook: pytest_runtest_setup
E       PytestUnraisableExceptionWarning: Exception ignored in: <function StreamWriter.__del__ at 0xffffb9f4d620>
E       
E       Traceback (most recent call last):
E         File "/usr/local/lib/python3.11/asyncio/streams.py", line 411, in __del__
E           warnings.warn("loop is closed", ResourceWarning)
E       ResourceWarning: loop is closed
E       
E       For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning

../virtualenvs/project-bTydf30B-py3.11/lib/python3.11/site-packages/pluggy/_callers.py:50: PluggyTeardownRaisedWarning

    def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            yield
            if cm.unraisable:
                if cm.unraisable.err_msg is not None:
                    err_msg = cm.unraisable.err_msg
                else:
                    err_msg = "Exception ignored in"
                msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                msg += "".join(
                    traceback.format_exception(
                        cm.unraisable.exc_type,
                        cm.unraisable.exc_value,
                        cm.unraisable.exc_traceback,
                    )
                )
>               warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function StreamWriter.__del__ at 0xffff878ed620>
E               
E               Traceback (most recent call last):
E                 File "/usr/local/lib/python3.11/asyncio/streams.py", line 411, in __del__
E                   warnings.warn("loop is closed", ResourceWarning)
E               ResourceWarning: loop is closed

../virtualenvs/project-bTydf30B-py3.11/lib/python3.11/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning

Is it expected? Should I add those warnings as ignored to my pytest setup?

@mdczaplicki

@mdczaplicki That error might be due to pytest -- I cannot tell from the output you're posting. If you really think this is still not fixed in CPython 3.11.9, can you open a new issue with complete instructions for reproducing the problem? (Ideally also testing in 3.12.)

pythongh-109538: Avoid RuntimeError when StreamWriter is deleted with…

Navigation Menu

Repository navigation

3.11.5 regression: StreamWriter.del fails if event loop is already closed #109538

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

8 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

3.11.5 regression: StreamWriter.__del__ fails if event loop is already closed #109538

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

dpr-0 commented on Sep 27, 2023

smirnoffs commented on Oct 6, 2023

pfps commented on Oct 25, 2023

dpr-0 commented on Oct 26, 2023

pfps commented on Oct 26, 2023

dpr-0 commented on Oct 26, 2023

dtatarkin commented on Nov 9, 2023

gvanrossum commented on Nov 9, 2023

bmerry commented on Nov 10, 2023

8 remaining items

RoTorEx commented on Nov 22, 2023

gvanrossum commented on Nov 22, 2023

LostInDarkMath commented on Jan 23, 2024

mdczaplicki commented on May 15, 2024

gvanrossum commented on May 16, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

3.11.5 regression: StreamWriter.del fails if event loop is already closed #109538