Skip to content

When having on --with-system-expat, system still builds using bundled expat for tests #92875

Description

@mcepl

Bug report

When building 3.11.0b1 for openSUSE (openSUSE/Tumbleweed as of 2022-05-17) we have --with-system-expat on (of course) and just to be sure we also rm -rf Modules/expat (to be sure, we really don't use bundled expat). Unfortunately, in such situation test_freeze_simple_script in test.test_tools.test_freeze.TestFreeze fails:

[ 1020s] test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmpl92b_12m/app.py
[ 1020s] copying the source tree into /tmp/tmpl92b_12m/cpython...
[ 1020s] configuring python in /tmp/tmpl92b_12m/python-build...
[ 1020s] building python in /tmp/tmpl92b_12m/python-build...
[ 1020s] CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpl92b_12m/python-build', '-j8']' returned non-zero exit status 2.

[Plenty of lines of gcc spew skipped]

[ 1139s] make[1]: Entering directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] gcc -c -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I/tmp/tmpraer2tjx/cpython/Include/internal -IObjects -IInclude -IPython -I. -I/tmp/tmpraer2tjx/cpython/Include    -fPIC -I/tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec -DCONFIG_64=1 -DANSI=1 -DHAVE_UINT128_T=1 -o Modules/_decimal/libmpdec/mpdecimal.o /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/mpdecimal.c
[ 1139s] make[1]: Leaving directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] 
[ 1139s] --- STDERR ---
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c: In function ‘PySys_SetArgv’:
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3305:5: warning: ‘PySys_SetArgvEx’ is deprecated [-Wdeprecated-declarations]
[ 1139s]  3305 |     PySys_SetArgvEx(argc, argv, Py_IsolatedFlag == 0);
[ 1139s]       |     ^~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3259:1: note: declared here
[ 1139s]  3259 | PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath)
[ 1139s]       | ^~~~~~~~~~~~~~~
[ 1139s] make[1]: *** No rule to make target '/tmp/tmpraer2tjx/cpython/Modules/expat/xmlparse.c', needed by 'Modules/expat/xmlparse.o'.  Stop.
[ 1139s] make[1]: *** Waiting for unfinished jobs....
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:611:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:607:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:501:22:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:356:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   356 |     case 18: EXTRACT_DIGIT(s, x, 100000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] 
[ 1139s] ---- END ----
[ 1139s] ERROR
[ 1139s] 
[ 1139s] ======================================================================
[ 1139s] ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
[ 1139s] ----------------------------------------------------------------------
[ 1139s] Traceback (most recent call last):
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
[ 1139s]     outdir, scriptfile, python = helper.prepare(script, outdir)
[ 1139s]                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 159, in prepare
[ 1139s]     _run_quiet([MAKE, '-C', builddir, '-j8'])
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
[ 1139s]     return subprocess.run(
[ 1139s]            ^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/subprocess.py", line 558, in run
[ 1139s]     raise CalledProcessError(retcode, process.args,
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s] subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpraer2tjx/python-build', '-j8']' returned non-zero exit status 2.
[ 1139s] 

Complete build log with all details of building the package.

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    on May 17, 2022
  2. mcepl commented on May 17, 2022

    @mcepl
    ContributorAuthor

    OK, so it is enough just to set CONFIG_ARGS variable to something like "--enable-ipv6 --enable-shared --with-ensurepip=no --with-system-ffi --with-system-expat --with-lto --enable-optimizations --with-system-libmpdec --enable-loadable-sqlite-extensions".

  3. mcepl commented on May 17, 2022

    @mcepl
    ContributorAuthor

    Nope, it doesn't help. When I remove the bundled expat it fails just the same even with CONFIG_ARGS variable set.

  4. reopened this on May 17, 2022
  5. sharewax commented on Aug 12, 2022

    @sharewax

    One more report for it.
    Trying to build SCL for Centos 7, issue is the same:

    make: Leaving directory `/tmp/tmpzdbz0bvg/python-build'
    --- STDERR ---
    /tmp/tmpzdbz0bvg/cpython/Objects/obmalloc.c:1449:1: warning: always_inline function might not be inlinable [-Wattributes]
     1449 | arena_map_get(block *p, int create)
          | ^~~~~~~~~~~~~
    make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
    make: *** Waiting for unfinished jobs....
    ---- END ----
    ERROR
    ======================================================================
    ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
        outdir, scriptfile, python = helper.prepare(script, outdir)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 159, in prepare
        _run_quiet([MAKE, '-C', builddir, '-j8'])
      File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
        return subprocess.run(
               ^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/subprocess.py", line 569, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpzdbz0bvg/python-build', '-j8']' returned non-zero exit status 2.
    ----------------------------------------------------------------------
    Ran 1 test in 78.506s
    FAILED (errors=1)
    test test_tools failed
    1 test failed again:
        test_tools
    == Tests result: FAILURE then FAILURE ==
    409 tests OK.
    10 slowest tests:
    - test_concurrent_futures: 2 min 36 sec
    - test_multiprocessing_spawn: 2 min 28 sec
    - test_subprocess: 2 min 27 sec
    - test_asyncio: 1 min 36 sec
    - test_tools: 1 min 36 sec
    - test_multiprocessing_forkserver: 1 min 28 sec
    - test_multiprocessing_fork: 1 min 16 sec
    - test_signal: 49.0 sec
    - test_io: 35.8 sec
    - test_compileall: 33.4 sec
    1 test failed:
        test_tools
    24 tests skipped:
        test_curses test_devpoll test_idle test_ioctl test_kqueue
        test_launcher test_msilib test_ossaudiodev test_smtpnet
        test_socketserver test_startfile test_tcl test_tix test_tk
        test_ttk_guionly test_ttk_textonly test_turtle test_urllib2net
        test_urllibnet test_winconsoleio test_winreg test_winsound
        test_xmlrpc_net test_zipfile64
    1 re-run test:
        test_tools
    Total duration: 4 min 27 sec
    Tests result: FAILURE then FAILURE
    

    I removed the internal Modules/expat, but have no idea then it takes make target for xmlparse in this case.

    # grep expat build.log
    + rm -r Modules/expat
    checking for --with-system-expat... yes
    checking for stdlib extension module pyexpat... yes
    building 'pyexpat' extension
    ...
    0:00:33 load avg: 7.96 [264/434] test_pyexpat passed -- running: test_asyncio (32.8 sec)
    make: *** No rule to make target `/tmp/tmp02i4l09d/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
    make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
    

    Looks like #91752

    Removed rm -r Modules/expat from spec file - all compiled ok. Centos7.

  6. StanFromIreland commented on Aug 24, 2026

    @StanFromIreland
    Member

    Fixed by 1512d6c6ee2, please let me know if that's not the case and I can re-open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildtestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions