Skip to content

--with-system-libmpdec / --with-system-expat still uses the vendored headers #98707

Description

@hroncok

Bug report

Python 3.12.0a1 or the main branch. Run:

$ sudo dnf install mpdecimal-devel expat-devel  # or similar for your OS
...
$ git clean -fdx
$ rm -r Modules/_decimal/libmpdec
$ rm -r Modules/expat
$ ./configure --config-cache --enable-shared --with-system-libmpdec --with-system-expat
$ make
...
make: *** No rule to make target 'Modules/_decimal/libmpdec/basearith.h', needed by 'Modules/_decimal/_decimal.o'.  Stop.

$ git restore Modules/_decimal/libmpdec
$ make
...
make: *** No rule to make target 'Modules/expat/ascii.h', needed by 'Modules/pyexpat.o'.  Stop.

This works fine with Python 3.11.0. I suspect somewhere along #94474 or a similar change, this has changed behavior, but I have not tried bisecting yet.

Your environment

  • CPython versions tested on: 3.12.0a1 or the main branch
  • Operating system and architecture: Fedora Linux x86_64

Activity

  1. hroncok commented on Oct 26, 2022

    @hroncok
    ContributorAuthor

    Bisecting with only Modules/_decimal/libmpdec removed, because I got some other failures with removed Modules/expat on the way.

    EDIT: Apparently, that's what you get when you spell expat as expact.

  2. hroncok commented on Oct 26, 2022

    @hroncok
    ContributorAuthor

    Indeed, my gut feeling was correct:

    81dca70 is the first new commit
    commit 81dca70
    Author: Christian Heimes @tiran
    Date: Thu Jul 14 09:51:49 2022 +0200

    gh-93939: Build C extensions without setup.py (GH-94474)
    
  3. hroncok commented on Oct 26, 2022

    @hroncok
    ContributorAuthor

    I am really really bad at autotools, but my best guess is that this is caused by:

    MODULE_PYEXPAT_DEPS=$(LIBEXPAT_HEADERS) @LIBEXPAT_INTERNAL@

    As LIBEXPAT_HEADERS contains a list that starts with Modules/expat/ascii.h.

    MODULE__DECIMAL_DEPS=$(srcdir)/Modules/_decimal/docstrings.h $(LIBMPDEC_HEADERS) @LIBMPDEC_INTERNAL@

    As LIBMPDEC_HEADERS contains a list that starts with Modules/_decimal/libmpdec/basearith.h.

    In both cases, the dependency needs to be made conditional on the relevant --with-system-xxx option.

  4. hroncok commented on Oct 26, 2022

    @hroncok
    ContributorAuthor

    Testing a possible fix.

  5. changed the title [-]--with-system-libmpdec / --with-system-expact still uses the vendored headers[/-] [+]--with-system-libmpdec / --with-system-expat still uses the vendored headers[/+] on Oct 26, 2022
  6. added 2 commits that reference this issue on Oct 26, 2022
  7. hroncok commented on Oct 26, 2022

    @hroncok
    ContributorAuthor

    FTR this was a problem on older Pythons as well, but it was not exposed. #94474 did not introduce it, but it exposed it. Another problem this will most likely fix is:

    --- STDERR ---
    /tmp/tmpt5xmo3n_/cpython/Objects/obmalloc.c:1449:1: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
     1449 | arena_map_get(block *p, int create)
          | ^~~~~~~~~~~~~
    make: *** No rule to make target '/tmp/tmpt5xmo3n_/cpython/Modules/_decimal/libmpdec/basearith.c', needed by 'Modules/_decimal/libmpdec/basearith.o'.  Stop.
    make: *** Waiting for unfinished jobs....
    ---- END ----
    ERROR
    ======================================================================
    ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/builddir/build/BUILD/Python-3.11.0/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
        outdir, scriptfile, python = helper.prepare(script, outdir)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0/Tools/freeze/test/freeze.py", line 159, in prepare
        _run_quiet([MAKE, '-C', builddir, '-j8'])
      File "/builddir/build/BUILD/Python-3.11.0/Tools/freeze/test/freeze.py", line 25, in _run_quiet
        return subprocess.run(
               ^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0/Lib/subprocess.py", line 569, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpt5xmo3n_/python-build', '-j8']' returned non-zero exit status 2.
    ----------------------------------------------------------------------
    Ran 1 test in 70.677s
    FAILED (errors=1)
    

    EDIT: It does not fix it.

  8. added a commit that references this issue on Oct 27, 2022
  9. added 2 commits that reference this issue on Oct 27, 2022
    3f3d7c1
    b031b7b
  10. added 2 commits that reference this issue on Nov 11, 2022
  11. added a commit that references this issue on Nov 12, 2022
  12. added a commit that references this issue on Nov 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.11only security fixes3.12only security fixesbuildThe build process and cross-buildtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions