Repository navigation
Bug: test_invalid_command fails on Python 3.12 #1864
Description
Activity
I have discussed offline with @lostinth1cho
The behavior of the test in python 3.12 is interesting
Reacted by lostinth1choMy terminal output:
(commitizen) (base) ➜ commitizen git:(master) uv run --python 3.12 pytest tests/test_cli.py Using CPython 3.12.12 Removed virtual environment at: .venv Creating virtual environment at: .venv Built commitizen @ file:///Users/bearomorphism/workspace/commitizen Installed 98 packages in 508ms =================================================================================================== test session starts ==================================================================================================== platform darwin -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0 rootdir: /Users/bearomorphism/workspace/commitizen configfile: pyproject.toml plugins: pytest_freezer-0.4.9, mock-3.15.1, xdist-3.8.0, gitconfig-0.9.0, datadir-1.8.0, regressions-2.8.3, cov-7.0.0 collected 23 items tests/test_cli.py ....................... [100%] ==================================================================================================== 23 passed in 1.26s ==================================================================================================== (commitizen) (base) ➜ commitizen git:(master) uv run --python 3.12.3 pytest tests/test_cli.py Using CPython 3.12.3 Removed virtual environment at: .venv Creating virtual environment at: .venv Installed 98 packages in 401ms =================================================================================================== test session starts ==================================================================================================== platform darwin -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0 rootdir: /Users/bearomorphism/workspace/commitizen configfile: pyproject.toml plugins: pytest_freezer-0.4.9, mock-3.15.1, xdist-3.8.0, gitconfig-0.9.0, datadir-1.8.0, regressions-2.8.3, cov-7.0.0 collected 23 items tests/test_cli.py ..F.................... [100%] ========================================================================================================= FAILURES ========================================================================================================= _______________________________________________________________________________________ test_invalid_command[py_3.12-invalidCommand] _______________________________________________________________________________________ util = UtilFixture(mocker=<pytest_mock.plugin.MockerFixture object at 0x10cdbfdd0>, monkeypatch=<_pytest.monkeypatch.MonkeyPatch object at 0x10cdff530>, freezer=<freezegun.api.FrozenDateTimeFactory object at 0x10cdd2570>) capsys = <_pytest.capture.CaptureFixture object at 0x10cdd23c0>, file_regression = <pytest_regressions.file_regression.FileRegressionFixture object at 0x10cdd23f0>, arg = 'invalidCommand' @pytest.mark.parametrize( "arg", [ "--invalid-arg", "invalidCommand", ], ) @pytest.mark.usefixtures("python_version", "consistent_terminal_output") def test_invalid_command(util: UtilFixture, capsys, file_regression, arg): with pytest.raises(NoCommandFoundError): util.run_cli(arg) out, err = capsys.readouterr() assert out == "" > file_regression.check(err, extension=".txt") E AssertionError: FILES DIFFER: E /private/var/folders/mv/d2xpwq4s7cscbzk_v2ymnbq80000gn/T/pytest-of-bearomorphism/pytest-15/test_invalid_command_py_3_12_i0/test_invalid_command_py_3_12_invalidCommand_.txt E /private/var/folders/mv/d2xpwq4s7cscbzk_v2ymnbq80000gn/T/pytest-of-bearomorphism/pytest-15/test_invalid_command_py_3_12_i0/test_invalid_command_py_3_12_invalidCommand_.obtained.txt E HTML DIFF: /private/var/folders/mv/d2xpwq4s7cscbzk_v2ymnbq80000gn/T/pytest-of-bearomorphism/pytest-15/test_invalid_command_py_3_12_i0/test_invalid_command_py_3_12_invalidCommand_.obtained.diff.html E --- E +++ E @@ -1,4 +1,4 @@ E usage: cz [-h] [--config CONFIG] [--debug] [-n NAME] [-nr NO_RAISE] E {init,commit,c,ls,example,info,schema,bump,changelog,ch,check,version} E ... E -cz: error: argument {init,commit,c,ls,example,info,schema,bump,changelog,ch,check,version}: invalid choice: 'invalidCommand' (choose from init, commit, c, ls, example, info, schema, bump, changelog, ch, check, version) E +cz: error: argument {init,commit,c,ls,example,info,schema,bump,changelog,ch,check,version}: invalid choice: 'invalidCommand' (choose from 'init', 'commit', 'c', 'ls', 'example', 'info', 'schema', 'bump', 'changelog', 'ch', 'check', 'version') tests/test_cli.py:43: AssertionError ================================================================================================= short test summary info ================================================================================================== FAILED tests/test_cli.py::test_invalid_command[py_3.12-invalidCommand] - AssertionError: FILES DIFFER: =============================================================================================== 1 failed, 22 passed in 1.00s ===============================================================================================This is indeed an interesting finding.
I think the simplest migration is to upgrade python 3.12 to the latest version in your environment.
The test passes in python 3.12.10+ env and fails <= 3.12.6 in my environment (I haven't done a full binary search)@noirbizarre wdyt? Should we just skip the test in python 3.12?
- added a commit that references this issue
on May 12, 2026
Description
The test_invalid_command fails specifically on Python 3.12. Interestingly, the test passes as expected on Python 3.10, 3.11, 3.13, and 3.14.
On Python 3.12, argparse produces an error message with single quotes around choices, but our current regression reference for 3.12 expects the "no-quote" format (which is the standard in 3.13+). This inconsistency causes a mismatch only in the 3.12 environment.
Steps to reproduce
uv run pytest tests/test_cli.py::test_invalid_commandAssertionErrorfromfile_regression.check.Current behavior
The test fails because the obtained output contains single quotes around the valid choices, whereas the reference file does not.
Diff output:
Desired behavior
The regression files for Python 3.12 should be standardized. We need to decide whether Python 3.12 should strictly follow the "no quotes" format or if the reference files need an update to accommodate the specific minor version of Python 3.12 being used.
Screenshots
Environment
Commitizen Version: 4.13.6
Python Version:
Operating System: Linux