SIGSEGV with generators in free-threaded build

Crash report

What happened?

Trying to iterate on a generator from multiple threads under the free-threaded build results in a segmentation fault.

Here is a minimal repro:

import concurrent.futures


def gen():
    while True:
        yield


it = gen()
with concurrent.futures.ThreadPoolExecutor() as executor:
    while True:
        _ = executor.submit(lambda: next(it))

The issue seems to be specific to generators as other kinds of iterators work well in parallel.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a0 experimental free-threading build (heads/main:c3b6dbff2c, Jun 10 2024, 16:54:16) [GCC 11.4.0]

Linked PRs

Thanks for the repro. Looking into this.

Confirmed, with pydebug and tsan.

May I take a chance to dive into this issue?

@Zheaoli

@Zheaoli sorry I assigned myself on this one I have a fix already and will put it up shortly. Do you have any specific issues in mind you would like to work on? I could help mentor/guide you on stuff you're interested in if I can help!

sorry I assigned myself on this one I have a fix already and will put it up shortly

No problem!

Do you have any specific issues in mind you would like to work on? I could help mentor/guide you on stuff you're interested in if I can help!

For now, I'm a fresh man try to be more familiar with the CPython codebase.

I'm now working on #104280 to make dtrace more functional to help debug the process. I'm also interested in JIT and Tier2(especially to enhance the observerability for the Python process). But I'm still trying to find the issue I can fix and the thing I can do(lol

Thanks for knowing that you will be willing to mentor/guide me lol!

@Zheaoli

@Zheaoli please send me an email I'll help give some pointers privately

pythongh-120321: Use atomics for PyGenObject.gi_frame_state

fusil has hit this one during fuzzing, it results in different segfaults and an abort, here's the MRE I got:

from threading import Thread

gen = (x for x in range(1_000_000))
def run_gen():
    for n in gen:
        pass

[Thread(target=run_gen, args=()).start() for x in range(20)]

gh-120321: Add gi_state, cr_state, and ag_state attributes (gh-144409)

pythongh-120321: Make gen.gi_frame.clear() thread-safe (pythongh-143112)

pythongh-120321: Add missing "return false" in gen_try_set_executing (p…

pythongh-120321: Make gi_yieldfrom thread-safe in free-threading build (

pythongh-120321: Add gi_state, cr_state, and ag_state attributes (pyt…

This fails with asserts on current 3.14:

Assertion failed: frame == &entry.frame, file X:\Work\cpython\3.14\Python\generated_cases.c.h, line 7694

And works well on current main.

Should we close this now since it's not planned for backport?

Reopening for async generators which are still not thread safe.

gh-120321: fix thread safety of concurrently iterating over async gen…

pythongh-120321: Make gi_frame, cr_frame and ag_frame thread-safe in …

pythongh-120321: Fix thread safety of gi_frame, cr_frame and ag_frame…

Navigation Menu

Repository navigation

SIGSEGV with generators in free-threaded build #120321

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

37 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

SIGSEGV with generators in free-threaded build #120321

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

Fidget-Spinner commented on Jun 10, 2024

Fidget-Spinner commented on Jun 10, 2024

Zheaoli commented on Jun 10, 2024

Fidget-Spinner commented on Jun 10, 2024

Zheaoli commented on Jun 10, 2024

Fidget-Spinner commented on Jun 11, 2024

devdanzin commented on Apr 21, 2025

37 remaining items

sergey-miryanov commented on May 4, 2026

kumaraditya303 commented on Aug 1, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions