Repository navigation
Race in concurrent list mutation and item retrieval #129069
Description
Activity
I'm a bit confused by the "memory writes to the newly allocated areas happen with memcpy/memmove" part of the issue description.
The data races with
memcpyI've seen involve writes to the existing item array. For example, inlist_ass_slice_lock_held, we havememmovecalls withself->ob_itemandrecycleas the destinations. The ones with the newly allocatedrecycleas the destination are okay, but the ones withself->ob_itemare potentially a problem.Yes, you're right, 'newly allocated' isn't really relevant. I just meant that the calls that move items around before/after list_resize() are potentially problematic. I have to admit my head hurts every time I try to trace through list_ass_slice_lock_held.
- added a commit that references this issue
on Mar 29, 2025
Bug report
Bug description:
A data race exists in lists when they are concurrently updated and indexed (or otherwise accessed): for slice assignment and other ways of adding or removing multiple items (list.extend, in-place multiplication), memory writes to the newly allocated areas happen with memcpy/memmove, which have no write size/atomicity guarantees (and it's not unlikely that they will perform jagged writes). These reallocations happen with the lock held, but since we're relying on QSBR, concurrent reads of those memory areas are technically possible.
The tests in PR #128798 (test_free_threading.test_iteration), when run under ThreadSanitizer, expose those races. (They are suppressed in Tools/tsan/suppressions_free_threading.txt).
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
list(GH-143019) #158488