Skip to content

Preventing code quality regressions #11769

Description

@srittau

For some time now we've used pyright's stricter configuration to prevent code quality regressions. Unless a stubs package is mentioned in the config file, it's expected to have annotations for all its fields. While this has served us well for a while, it has become unsuitable since the introduction of _typeshed.Incomplete:

  • pyright doesn't understand that a (partial) Incomplete annotation is equivalent (for our purposes) to an unannotated item.
  • It's tedious to maintain the pyright stricter configuration in addition to the metadata in the stubs directory.
  • It's easy to cheat the regression check by using Incomplete.

To improve the situation, I propose to move to a custom solution:

  1. Add an incomplete marker to METADATA.toml (defaulting to false).
  2. Add a custom script to CI that checks that a stub package marked as complete can't have any incomplete (unannotated or using Incomplete) fields. Alternatively, we could add a disabled-by-default check to flake8-pyi and use that instead.
  3. At a later date, we could also add a PR CI script that add a PR comment warning a user if the amount of incomplete increases due to a PR.

(See also https://alexwaygood.github.io/typeshed-stats/.)

Activity

  1. Daverball commented on Apr 21, 2024

    @Daverball
    Contributor

    I think it might still be nice to have some support for stricter type checker configs on a per distribution basis on the CI side, but I agree that the current solution for detecting completeness is insufficient.

    It has kind of bothered me for a while now, that there's no way to e.g. enforce mypy --strict for a third party stub, even if you use those flags locally for initial development, regressions become inevitable since the stricter checks will not be enforced by the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    project: infrastructuretypeshed build, test, documentation, or distribution related

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions