Repository navigation
Memory leak in test_sys with subinterpreters creation (AddressSanitizer detection) #140067
Description
Activity
@ericsnowcurrently It appears we still have a leak in subinterpreters. It also looks like similar to #110411 but not really a duplicate considering the traceback.
I'll try to fix this leak, but no promises.
Let us first try to check if this is something that is known to Eric or not before jumping onto a PR. It would avoid having to make unnecessary work. It's fine to report an issue, wait for an expert to have a look at it, and only then consider a PR.
Reacted by yihong and ShamilAlright, got it, I’ll do that. Thanks!
- added a commit that references this issue
on Oct 14, 2025 I think this is a duplicate of #109700, I was looking into this a while ago I'll check if I had reached a fix.
I had git bisected it and I remember that #132428 was the first bad commit in my testing.
@kumaraditya303 I solved this problem with these changes.
ashm-dev@42ff41aAh yes, the problem is that malloced pointer is being overwritten here, I came up with a different solution just now but it does the same as yours.
diff --git a/Include/internal/pycore_interp_structs.h b/Include/internal/pycore_interp_structs.h index 2124e76514f..badc97808c6 100644 --- a/Include/internal/pycore_interp_structs.h +++ b/Include/internal/pycore_interp_structs.h @@ -769,12 +769,6 @@ struct _is { * and should be placed at the beginning. */ struct _ceval_state ceval; - /* This structure is carefully allocated so that it's correctly aligned - * to avoid undefined behaviors during LOAD and STORE. The '_malloced' - * field stores the allocated pointer address that will later be freed. - */ - void *_malloced; - PyInterpreterState *next; int64_t id; diff --git a/Python/pystate.c b/Python/pystate.c index dbed609f29a..fb7f8177ab2 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -457,16 +457,19 @@ _PyInterpreterState_Enable(_PyRuntimeState *runtime) static PyInterpreterState * alloc_interpreter(void) { + // Aligned allocation for PyInterpreterState. + // the first word of the memory block is used to store + // the original pointer to be used later to free the memory. size_t alignment = _Alignof(PyInterpreterState); - size_t allocsize = sizeof(PyInterpreterState) + alignment - 1; + size_t allocsize = sizeof(PyInterpreterState) + sizeof(void*) + alignment - 1; void *mem = PyMem_RawCalloc(1, allocsize); if (mem == NULL) { return NULL; } - PyInterpreterState *interp = _Py_ALIGN_UP(mem, alignment); - assert(_Py_IS_ALIGNED(interp, alignment)); - interp->_malloced = mem; - return interp; + void *ptr = _Py_ALIGN_UP((char *)mem + sizeof(void*), alignment); + ((void **)ptr)[-1] = mem; + assert(_Py_IS_ALIGNED(ptr, alignment)); + return ptr; } static void @@ -481,7 +484,7 @@ free_interpreter(PyInterpreterState *interp) interp->obmalloc = NULL; } assert(_Py_IS_ALIGNED(interp, _Alignof(PyInterpreterState))); - PyMem_RawFree(interp->_malloced); + PyMem_RawFree(((void **)interp)[-1]); } }
@ashm-dev I have slight preference to my fix as it avoids the malloced pointer altogether so in future there would not be such mistake of overwriting the pointer. Would you like to create a PR based on the changes?
17 remaining items
Hum. I was sure to be careful here but looks like not. However be sure to also run the UB sanitizer combined with ASAN on whatever fix you suggest (I am travelling and won't be back until Monday)
Should not the fix be backported to 3.14 and 3.13?
Should not the fix be backported to 3.14 and 3.13?
I backported this to 3.14 in #140118, haven't checked if this affects 3.13 but should be backported if it does.
Indeed, there were leaks in #109700 in 3.13, but they were fixed, and only new leaks remained in 3.14+.
I see that there were two commits in main, one of them was reverted, but only one commit in 3.14. Should it it also be reverted and replaced with the backport of the new version?
It was reverted in main because it broke CI, 3.14 backport was redone with the fix hence a single PR.
- added 3 commits that reference this issue
on Dec 6, 2025
Bug report
Bug description:
Bug Description:
I've discovered a memory leak when running
test_syswith AddressSanitizer enabled. The leak occurs during subinterpreter creation and the allocated memory is never freed.Environment:
CC=clang CXX=clang++ ./configure \ --disable-optimizations \ --with-valgrind \ --with-pydebug \ --enable-pystats \ --with-address-sanitizerSteps to Reproduce:
./python -m test test_sysExpected Behavior:
No memory leaks should be detected by AddressSanitizer.
Actual Behavior:
AddressSanitizer reports approximately 4.5 MB leaked across 20 allocations. All allocations trace back to
_PyInterpreterState_New→alloc_interpreter→_PyMem_DebugRawCalloc.The test itself passes successfully, but the leak persists:
Reproduction on Multiple Versions:
Additional Context:
The leak appears to be related to subinterpreter lifecycle management. The stack trace shows memory is allocated through:
interp_create(Modules/_interpretersmodule.c)_PyXI_NewInterpreter(Python/crossinterp.c:3204)Py_NewInterpreterFromConfig(Python/pylifecycle.c)new_interpreter(Python/pylifecycle.c)_PyInterpreterState_New(Python/pystate.c)alloc_interpreter(Python/pystate.c)This issue appears to be related to #110411, which remains open with the latest report from August 2025 stating "This is still a problem in 3.13.7 and 3.14.0rc2". Issue #113055 was closed as resolved in February 2025, but the leak still occurs on current 3.14 and main branches.
Complete AddressSanitizer Output (3.15 main branch)
Complete AddressSanitizer Output (3.14 branch)
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs