Repository navigation
Add terminate_workers to ProcessPoolExecutor #128041
Description
Activity
- added a commit that references this issue
on Mar 3, 2025 thanks for the feature contribution!
Re-opening because we have some build bots failing (I'm not sure whether the macOS failure is related but the other seems so)
https://github.com/python/cpython/actions/runs/13631446353/job/38099943353#step:11:686
https://buildbot.python.org/#/builders/568/builds/8398Thanks! @colesbury. I'll dig more to figure out the issue before opening a fresh pr.
Current issue:
test_force_shutdown_workers_stops_pool (test.test_concurrent_futures.test_process_pool.ProcessPoolForkserverProcessPoolExecutorTest.test_force_shutdown_workers_stops_pool) ... 0.07s Warning -- reap_children() reaped child process 33328 ... 0:00:09 load avg: 2.32 [1/1/1] test.test_concurrent_futures.test_process_pool failed (env changed) == Tests result: ENV CHANGED == 1 test altered the execution environment (env changed): test.test_concurrent_futures.test_process_pool Total duration: 9.0 sec Total tests: run=84 skipped=9 Total test files: run=1/1 env_changed=1 Result: ENV CHANGEDInterestingly enough it was suggested that commenting out
self.assertRaises(RuntimeError, executor.submit, time.sleep, 0)makes the test pass. It does.. most of the time. It still transiently fails the same way with a leaked child process (though much rarer).
So I guess the timing changed caused by that line make it more likely to happen, but we're still dealing with a race condition.
I think I figured it out. Will open a PR in a few minutes.
FWIW on the original PR while I suspected we might have flakiness issues... I was way underestimating that apparently. I should've does the buildbot tests before merging. sorry for the churn! :)
- added a commit that references this issue
on Mar 5, 2025 merged again with the improvements. 🤞🏾
Feature or enhancement
Proposal:
This is an interpretation of the feature ask in: https://discuss.python.org/t/cancel-running-work-in-processpoolexecutor/58605/1. It would be a way to stop all the workers running in a
ProcessPoolExecutorPreviously the way to do this was to use to loop through the
._processesof theProcessPoolExecutorthough, preferably this should be possible without accessing implementation details.Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
https://discuss.python.org/t/cancel-running-work-in-processpoolexecutor/58605/1
Linked PRs
terminate_workersandkill_workersmethods to ProcessPoolExecutor (GH-128043)" #130838terminate_workersandkill_workersmethods to ProcessPoolExecutor #130849