Repository navigation
Slow functions calls for C-coded functions, accepting positional-or-kwarg argument vs positional-only #136681
Description
Bug report
Bug description:
I don't see measurable difference on pure-Python level, e.g. for
# a.py
import pyperf
def f1(x):
return 1
def f2(x, /):
return 1
runner = pyperf.Runner()
runner.bench_func("f1(1)", f1, 1)
runner.bench_func("f2(1)", f2, 1)I got on 3.13:
$ python a.py -q
f1(1): Mean +- std dev: 233 ns +- 2 ns
f2(1): Mean +- std dev: 232 ns +- 2 ns
But for functions, implemented in C, using the Argument Clinic - the difference is bigger:
$ python bench.py -q -o patch.json
isfinite(1.0): Mean +- std dev: 152 ns +- 1 ns
$ git checkout master && make -s
Checked 114 modules (35 built-in, 78 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)
$ python bench.py -q -o ref.json
isfinite(1.0): Mean +- std dev: 136 ns +- 1 ns
| Benchmark | ref | patch |
|---|---|---|
| isfinite(1.0) | 136 ns | 152 ns: 1.12x slower |
(In the second version - the math.isfinite() accept also keyword argument x)
a patch
(run ./python Tools/clinic/clinic.py Modules/mathmodule.c !)
diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c
index 7c2a421dd6..e33281ac8b 100644
--- a/Modules/mathmodule.c
+++ b/Modules/mathmodule.c
@@ -3156,14 +3156,13 @@ math_radians_impl(PyObject *module, double x)
math.isfinite
x: double
- /
Return True if x is neither an infinity nor a NaN, and False otherwise.
[clinic start generated code]*/
static PyObject *
math_isfinite_impl(PyObject *module, double x)
-/*[clinic end generated code: output=8ba1f396440c9901 input=46967d254812e54a]*/
+/*[clinic end generated code: output=8ba1f396440c9901 input=a47ad6e72998b81d]*/
{
return PyBool_FromLong((long)isfinite(x));
}This performance penalty affects API decisions (e.g. #131886) and that looks as a bug.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Activity
serhiy-storchaka commentedon Jul 16, 2025
Look at the generated code.
_PyArg_UnpackKeywordshas a fast path. I would check ifkwnamesis NULL here.skirpichev commentedon Jul 16, 2025
No. It's already checked. Twice. Both in _PyArg_UnpackKeywords() function and in the _PyArg_UnpackKeywords() macro. And this path - hits.
The problem is code for initialization of
_kwtuplein clinic's generated output:If I move that to
if (kwnames) { ... }block - speed loss is lost.serhiy-storchaka commentedon Jul 16, 2025
Static initializer should only be executed once.
skirpichev commentedon Jul 16, 2025
Hmm, with following in the clinic output:
I got same numbers as in the master (i.e. with positional-only isfinite) for a simple benchmark:
(CPython built with default configure flags.)
patch for the AC
Though, if I benchmark a loop with:
it's still worse!. I got
where
patch0- version withoutif (kwnames)condition.I suspect this might be related to specialization for c-methods calls in
Python/specialize.c. (Edit: indeed, it's because specialize_c_call().)@serhiy-storchaka, the AC patch alone does make sense for you?
serhiy-storchaka commentedon Jul 16, 2025
No, this change is not correct.
_PyArg_UnpackKeywordscannot be omitted ifkwnamesis NULL. For example, it checks that the correct number of positional arguments was passed.There are two implementations of
_PyArg_UnpackKeywords-- as a macro and as a C function. The macro version handles the fast path, it should add very little overhead. The function implementation is only used for non-fast path. So my suspect is that for some reasons the fast path may be not correctly detected, so the code goes the general slow path. Please check if the_PyArg_UnpackKeywordsfunction was called in that case.skirpichev commentedon Jul 16, 2025
That was just a draft:)
I already did this first. No - fast path was taken.
I think I've found a simpler solution: just add a static decorator for
argsbuf. Here benchmark for math.fmin (ref - current main, patch-v0 - support for kwargs, patch-v1 - make also argsbuf static). Up to a noise - the v1 patch match timings in the main branch:For one-arg functions (like isfinite()) - the difference holds also in the v1 version for loops. I hope this could be fixed too, using different specialized instructions.
pythongh-136681: make argsbuf static to speedup processing positional
pythongh-136681: make argsbuf static to speedup processing positional
pythongh-136681: make argsbuf static to speedup processing positional
pythongh-136681: make argsbuf static to speedup processing positional
kumaraditya303 commentedon Jul 17, 2025
There are redundant checks in the implementation of these functions:
Here the check for NULL check for parser and type check for kwnames can be turned into asserts, they should not be runtime checks, similarly for other functions as well.
skirpichev commentedon Jul 17, 2025
The function is not the reason for a slowdown, as the fast path is taken in the macro.
pythongh-136681: optimize builtin calls if kwargs allowed
pythongh-136681: optimize builtin calls if kwargs allowed
pythongh-136681: optimize builtin calls if kwargs allowed
skirpichev commentedon Jul 30, 2025
@serhiy-storchaka, @kumaraditya303 - #136732 implements a variation of the storage class specifier. So far, I kept this PR as a draft, as it's huge. Though, actual changes in AC are small. Let me know if it worth to make this review-ready.
This doesn't fix problem completely. Still, there are some differences, especially for 1-arg case (as noted in #136681 (comment)).
In the main:
With the patch:
Details
skirpichev commentedon Sep 10, 2025
PR #136732 is ready for review.
pythongh-136681: Improve _PyArg_UnpackKeywords fast-path macro
pythongh-136681: Pre-compute parser.pos at AC time; bltinmodule fully…
pythongh-136681: Pre-compute parser.pos at AC time; bltinmodule fully…
pythongh-136681: Delete dead parse_format/scan_keywords; skip parser_…
pythongh-136681: Split _PyArg_UnpackKeywords; pre-compute parser.pos …
pythongh-136681: Shrink _PyArg_Parser and skip its init for static kw…
gh-157407: Argument Clinic: parse positional-only `defining_class` me…