accessing mmap of file that is overwritten causes bus error #84897

Activity

1. mhvk commented on May 21, 2020

   

   mhvkmannequin

   on May 21, 2020

   MannequinAuthor

   More actions

   While debugging a strange failure with tests and np.memmap, I realized that the following direct use of mmap reliably leads to a bus error. Here, obviously mmap'ing a file, closing it, opening the file for writing but not writing anything, and then again accessing the mmap is not something one should do (but a test case did it anyway), but it would nevertheless be nice to avoid a crash!

   import mmap
   
   
   with open('test.dat', 'wb') as fh:
       fh.write(b'abcdefghijklmnopqrstuvwxyz')
   
   
   with open('test.dat', 'rb') as fh:
       mm = mmap.mmap(fh.fileno(), 0, access=mmap.ACCESS_READ)
   
   
   with open('test.dat', 'wb') as fh:
       pass  # Note: if something is written, then I get no bus error.
   
   
   mm[2]
   
2. mhvk

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on May 21, 2020
3. mhvk commented on May 22, 2020

   

   mhvkmannequin

   on May 22, 2020

   MannequinAuthor

   More actions

   I should probably have added that the bus error happens on linux. On Windows, the opening of the file for writing leads to an error, as the file is still opened for reading inside the mmap.
4. graingert commented on Mar 9, 2021

   

   graingertmannequin

   on Mar 9, 2021

   Mannequin

   More actions

   I can confirm this happens on py3.5-3.10

   import mmap
   import pathlib
   import tempfile
   
   
   def main():
       with tempfile.TemporaryDirectory() as tmp:
           tmp_path = pathlib.Path(tmp)
           path = tmp_path / "eg"
   
           path.write_bytes(b"Hello, World!")
   
           with path.open("rb") as rf:
               mm = mmap.mmap(rf.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ)
               path.write_bytes(b"")
               bytes(mm)
   
   if __name__ == "__main__":
       main()
   
5. graingert

   added

   3.7 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   on Mar 9, 2021
6. ronaldoussoren commented on Mar 9, 2021

   

   ronaldoussoren

   on Mar 9, 2021

   Contributor

   More actions

   What happens here is that the file is truncated, which (more or less) truncates the memory mapping. Accessing a memory mapping beyond the length of the file results in a SIGBUS signal.

   I'm not sure if there is much Python can do about this other than shrinking the window for crashes like this by aggressively checking if the file size has changed (but even then a crash will happen if another proces truncates the file between the time the check is done and the memory is actually accessed).

   ---

   Variant of the script that explicitly truncates the file:

   def main():
       with tempfile.TemporaryDirectory() as tmp:
           tmp_path = pathlib.Path(tmp)
           path = tmp_path / "eg"
   
           path.write_bytes(b"Hello, World!")
   
           with path.open("r+b") as rf:
               mm = mmap.mmap(rf.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ)
               rf.truncate(0)
               bytes(mm)
   
   if __name__ == "__main__":
       main()
7. iritkatriel commented on Oct 18, 2021

   

   iritkatriel

   on Oct 18, 2021

   Member

   More actions

   Reproduced on 3.11.
8. iritkatriel

   added

   3.11only security fixesonly security fixes

   and removed

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   on Oct 18, 2021
9. ezio-melotti

   transferred this issue fromon Apr 10, 2022
10. iritkatriel commented on Mar 1, 2023

    

    iritkatriel

    on Mar 1, 2023

    Member

    More actions

    I think this is a duplicate of #60416.
    cc @terryjreedy.
11. ronaldoussoren commented on Mar 1, 2023

    

    ronaldoussoren

    on Mar 1, 2023

    Contributor

    More actions

    I agree.

    The other issue mentions that we cannot avoid this crash on our end due to limitations in the underlying APIs.
12. ronaldoussoren

    closed this as not plannedon Mar 1, 2023