Repository navigation
Use stop-the-world to make fork and interpreter shutdown thread-safe #116522
Description
Feature or enhancement
In the free-threaded build, we should use a stop-the-world call to make fork() 1 and shutdown (i.e, Py_FinalizeEx) thread-safe.
The two operations are similar in that they both involve deleting thread states for other threads.
Fork
Fork and in multithreaded environments is notorious for the problems it causes, but we can make the free-threaded build at least as safe as the default (GIL) build by pausing other threads with a stop-the-world call before forking. In the parent, we can use a normal start-the-world call to resume other threads after fork. In the child, we want to call start the world after other thread states are removed, but before we execute python code. This may require some refactoring.
Shutdown
The interpreter waits for threading module created non-daemon threads during shutdown, but this can be canceled by a ctrl-c and there may still be active thread states for daemon threads and threads created from C outside of the threading module. We should use a stop-the-world call to bring these threads to a consistent state before we try deleting their thread states.
Linked PRs
- gh-116522: Stop the world before fork() and during shutdown #116607
- gh-116522: Refactor
_PyThreadState_DeleteExcept#117131
Footnotes
-
or at least less thread-unsafe... ↩
Activity
Rajendra1308 commentedon Mar 9, 2024
Hey @colesbury I would like to work on this, can I get more info. And what file to modify. I am a beginner to this repo, please help.
colesbury commentedon Mar 9, 2024
Hi @Rajendra1308, thanks for your interest but I'm already working on this.
Rajendra1308 commentedon Mar 10, 2024
Hey @colesbury , alright. Good Luck!
pythongh-116522: Stop the world before fork() and during shutdown
pythongh-116522: Stop the world before fork() and during shutdown
Merge branch 'pythongh-116522-fork-shutdown' into nogil-integration
Merge branch 'pythongh-116522-fork-shutdown' into nogil-integration
Merge branch 'pythongh-116522-fork-shutdown' into nogil-integration
Merge branch 'pythongh-116522-fork-shutdown' into nogil-integration
Merge branch 'pythongh-116522-fork-shutdown' into nogil-integration
gh-116522: Stop the world before fork() and during shutdown (#116607)
pythongh-116522: Refactor `_PyThreadState_DeleteExcept`
gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)
pythongh-116522: Stop the world before fork() and during shutdown (py…
pythongh-116522: Refactor `_PyThreadState_DeleteExcept` (python#117131)
Squashed commit of the following:
Squashed commit of the following:
pythongh-116522: Stop the world before fork() and during shutdown (py…
pythongh-116522: Refactor `_PyThreadState_DeleteExcept` (python#117131)