Repository navigation
Speeding up the Docs / Docs CI job #459
Description
The short story
I think the Docs CI job can be faster, which would improve the contribution and review experience.
Long version
In the documentation team meeting we've been talking about a few workflow improvements. The Docs / Docs CI job is quite slow, taking around 15-20 minutes. From some quick testing, it seems this is largely due to that we use Python from HEAD and build Python in the job1, rather than a build with optimisation etc. Tests of my speed-up patch reduce the core docs job to ~3m15s2.
As a reviewer, a green tick is a quick signal that the syntax of the documentation is probably correct, and I don't need to be as thorough, whereas if the build fails I can look in the logs and point out the error and a potential fix to the proponenet of the PR.
I would suggest moving to actions/setup-python for the Docs job, but I don't know if there's been prior discussion3 on this that I haven't been able to find, or if there are other reasons it makes sense to trade off speed for building Python from HEAD.
A
Footnotes
-
https://github.com/python/cpython/actions/workflows/doc.yml ↩
-
Find a way to avoid redundant CI workflow builds of cpython just to test Docs #435 discusses not building Python, but in the context of reusing another build from HEAD rather than an optimised version. ↩
Activity
AA-Turner commentedon Jun 7, 2022
cc: @JulienPalard @ezio-melotti
hugovk commentedon Jun 7, 2022
cc @ned-deily re: #435
actions/setup-pythonalso suggested at #429 (comment), with three 👍.The big question is: do we need to build docs against Python from HEAD?
If not, I think using
actions/setup-pythonis a great idea.And we have options of what Python to test with [with approx setup timings]:
3.103.11-devis the latest alpha/beta/RC release3.11-dev,3.12-dev(for example)Building Python from source takes about 141s, so the above options are quite a bit quicker.
But the patch saves much more time than ~141s in setup though.
actions/setup-python, and doctest on HEAD (as new syntax doesn't exist in the latest stable release).The real big saving:
actions/setup-pythontakes only ~2m compared to 6-15m with Python on HEAD!AA-Turner commentedon Jun 7, 2022
Thanks for finding the comment from #429!
I think this is a less compelling reason -- the Sphinx project tests with the latest development builds (currently deadsnakes' 3.11-dev nightly builds), so will catch deprecations there. Were there non-Sphinx deprecations you were referring to? (The link was to the collapsed summary page, so I'm not sure which deprecation warnings you're talking about).
A
methane commentedon Jun 7, 2022
doctest in CI job should be run with HEAD Python.
AA-Turner commentedon Jun 7, 2022
@methane -- in my patch the jobs are split so that doctest still runs on HEAD -- is this alright?
A
hugovk commentedon Jun 7, 2022
Agreed, if they're just from third-party libraries it's not important. Much better to speed up the builds.
AA-Turner commentedon Jun 15, 2022
python/cpython#93736 has been merged, thanks all.
A