Skip to content

Invalid mapping patterns give confusing SyntaxErrors #88534

Description

@brandtbucher
BPO 44368
Nosy @lysnikolaou, @pablogsal, @miss-islington, @brandtbucher
PRs
  • bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords #26630
  • [3.10] bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630) #26631
  • bpo-44368: Improve syntax errors with invalid as pattern targets #26632
  • [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) #26792
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2021-06-09.19:47:35.477>
    labels = ['interpreter-core', 'type-bug', '3.11']
    title = 'Invalid mapping patterns give confusing SyntaxErrors'
    updated_at = <Date 2021-06-18.21:16:02.181>
    user = 'https://github.com/brandtbucher'

    bugs.python.org fields:

    activity = <Date 2021-06-18.21:16:02.181>
    actor = 'pablogsal'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Parser']
    creation = <Date 2021-06-09.19:47:35.477>
    creator = 'brandtbucher'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44368
    keywords = ['patch']
    message_count = 20.0
    messages = ['395453', '395454', '395458', '395460', '395461', '395462', '395467', '395469', '395470', '395473', '395477', '395478', '395479', '395481', '395482', '395485', '395486', '395487', '395584', '396092']
    nosy_count = 4.0
    nosy_names = ['lys.nikolaou', 'pablogsal', 'miss-islington', 'brandtbucher']
    pr_nums = ['26630', '26631', '26632', '26792']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44368'
    versions = ['Python 3.11']

    Activity

    1. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      Here are a few that I found. Not sure when they were introduced:

      match ...:
      case {**rest, "key": value}:
      pass

      match ...:
      case {"first": first, **rest, "last": last}:
      pass

      match ...:
      case {**_}:
      pass

      These all give the following error while parsing the second line:

      File "<stdin>", line 1
      match ...:
      ^^^^^^^^^
      SyntaxError: invalid syntax. Perhaps you forgot a comma?

    2. added
      3.11only security fixes
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      type-bugAn unexpected behavior, bug, or error
      on Jun 9, 2021
    3. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      Perhaps we need something like invalid_mapping_pattern or invalid_mapping_pattern_double_star rules?

    4. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Probably, otherwise is going to hurt because the syntax errors that you describe trigger when two names are place together, which in general is a missing comma.

      The error message also doesn't say: "you are missing a comma" it says that the most typical reason is a missing comma, which is important to distinguish :)

    5. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      The backtracking with the soft keyword may make this very annoying, by the way.

    6. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Oh, wait, I think I misunderstood the problem. The problem is that the parser is backtracking and identifying match as a name.

      Indeed, the soft keyword is a pain :(

    7. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      I think it actually will be very useful to explain that these cases are invalid in the Syntax error, which will also solve this problem.

      This on the other hand shows a bigger problem: any generic syntax error that happens inside "match" will probably end identifying the keyword as a name, even if we don't have specific errors, the parser will point to it and complain about "match".

    8. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Oh, turns out I already added machinery to solved this but I was missing a piece!

    9. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      Wow, that was quite a roller-coaster ride. Thanks Pablo. :)

    10. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      I found a similar one, by the way (not related to mapping patterns):

      >>> match ...:
      ...     case 42 as _:
      ...         pass
      
        File "<stdin>", line 2
          case 42 as _:
                     ^
      SyntaxError: expected ':'

      Is this covered by your fix?

    11. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Is this covered by your fix?

      No, that is not a backtracking error (is not going all the way to the "match" expression).

    12. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      I will fix that one in another PR

    13. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      New changeset 457ce60 by Pablo Galindo in branch 'main':
      bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
      457ce60

    14. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Oh, this one is actually correct:

      match ...:
      case 42 as _:
      pass

      File "<stdin>", line 2
      case 42 as _:
      ^
      SyntaxError: expected ':'

      That is literally expecting a ":" and that's the error. It has identified "case 42" correctly and it now expects a ":". Is just that the error marker is wrong because it has reached the "as and _" as part of the parsing so the error is there.

      That is going to be a bit tricky to "fix"

    15. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.

    16. brandtbucher commented on Jun 9, 2021

      @brandtbucher
      MemberAuthor

      Like "SyntaxError: can't capture into a wildcard (consider removing)".

    17. miss-islington commented on Jun 9, 2021

      @miss-islington
      Contributor

      New changeset f807a4f by Miss Islington (bot) in branch '3.10':
      bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630)
      f807a4f

    18. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Could we just try parsing "as _" and raise if so? That wouldn't conflict with any existing rules, and that way we could actually have a helpful error message.

      No, the same happens with other targets such as:

      File "/home/pablogsal/github/python/master/lel.py", line 2
      case 42 as 1+1:
      ^
      SyntaxError: expected ':'

    19. pablogsal commented on Jun 9, 2021

      @pablogsal
      Member

      Checkout PR 26632, and see if this works for you

    20. pablogsal commented on Jun 10, 2021

      @pablogsal
      Member

      New changeset 0507303 by Pablo Galindo in branch 'main':
      bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632)
      0507303

    21. pablogsal commented on Jun 18, 2021

      @pablogsal
      Member

      New changeset a8c418d by Pablo Galindo in branch '3.10':
      [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) (GH-26792)
      a8c418d

    22. transferred this issue fromon Apr 10, 2022
    23. furkanonder commented on May 9, 2023

      @furkanonder
      Contributor

      @pablogsal @brandtbucher The issue seems to be solved. We can close the issue.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions