Python 3.13.3 runs __iter__ twice in a list comprehension

Bug report

Bug description:

I can't upload comprehensionTestbed2.py because I am told the .py type is not allowed so I here is the script I used to reproduce this issue:

===========================================

# TestClass: Class that interacts with the DVD data file.

class TestClass:
    # Initialize the file timestamp.
    def __init__(self):
        pass
    # The iterator method.
    def __iter__(self):
        print('S1')
        self._index = 0
        return self
    # The next method.
    def __next__(self):
        print('S5')
        if self._index == 5:
            raise StopIteration
        value = self._index
        self._index += 1
        return value


# Test code.

print('#1')
for n in TestClass():
    print(n)

print('#2')
numbers = [n for n in TestClass()]
print(numbers)

==============================================

Here is the output of the Python script above:

D:\Files\Testing> comprehensionTestbed2.py                
#1                                                        
S1                                                        
S5                                                        
0                                                        
S5                                                        
1                                                        
S5                                                        
2                                                        
S5                                                        
3                                                        
S5                                                        
4                                                        
S5                                                        
#2                                                        
S1                                                        
S1  <-----  The __iter__ method is invoked twice in the list comprehension when using 3.13.3 (and 3.13.2).                                            
S5                                                        
S5                                                        
S5                                                        
S5                                                        
S5                                                        
S5                                                        
[0, 1, 2, 3, 4]

This does not happen with python 3.12.9 and is a real problem when the iter method creates a mutex.

Edward

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Ah, could it be a duplicate of #127682?

Ah, could it be a duplicate of #127682?

I think so, this is a duplicate.

This was fixed in 3.13.4 but is back in 3.13.5: D:\Files\Testing> more comprehensionTestbed2.py # TestClass: Class that interacts with the DVD data file. class TestClass: # Initialize the file timestamp. def __init__(self): pass # The iterator method. def __iter__(self): print('S1') self._index = 0 return self # The next method. def __next__(self): print('S5') if self._index == 5: raise StopIteration value = self._index self._index += 1 return value # Test code. print('#1') for n in TestClass(): print(n) print('#2') numbers = [n for n in TestClass()] print(numbers) D:\Files\Testing> comprehensionTestbed2.py #1 S1 S5 0 S5 1 S5 2 S5 3 S5 4 S5 #2 S1 S1 S5 S5 S5 S5 S5 S5 [0, 1, 2, 3, 4] I was in the process of switching from 3.12.9 to 3.13.4 but then I noticed 3.13.5 had been released so I tested it. This is important to me because in production code the __iter__ method creates a lock and the script will hang when __iter__ is called the second time. Of course, I could get around that by using a counter but why is __iter__ being called more than once when I does not need to be. Edward

…

________________________________ From: Bénédikt Tran ***@***.***> Sent: Saturday, April 19, 2025 2:11 AM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***>; Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) Closed #132711<#132711> as duplicate. — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2AWY5MU4LAPD2A4LUL22IHKNAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJXGMZDQMJTGI3DONA>. You are receiving this because you authored the thread.

Unfortunately, we can't fix this bug without producing other bigger problems. So, fix in 3.13.4 was reverted.

You can read this topic for more information: https://discuss.python.org/t/python-3-13-5-is-now-available-yes-really/95211

Understood. For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Edward

…

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 12:58 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***>; Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://github.com/proxy/avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> Unfortunately, we can't fix this bug without producing other bigger problems. So, fix in 3.13.4 was reverted. You can read this topic for more information: https://discuss.python.org/t/python-3-13-5-is-now-available-yes-really/95211 — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2GBD5NGOSNBHBRNPNT3EMI4ZAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOBZGA2TCMJTGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension?

Only in comprehensions.

JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way:

def __iter__(self):
    return self

That works but it seems that the __iter__ method is not really supposed do anything but just return self. Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version? Edward

…

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 11:53 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***>; Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://github.com/proxy/avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Only in comprehensions. JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way: def __iter__(self): return self — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2DXTJJHYAPKHC34Y4L3EOVXTAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJQGAYDANJXGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

Sorry, I pushed the Send too soon. I wanted to thank you for your responses and all who were involved with this issue. Edward

…

________________________________ From: Edward Waugh ***@***.***> Sent: Friday, June 20, 2025 6:30 PM To: python/cpython ***@***.***>; python/cpython ***@***.***> Cc: Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) That works but it seems that the __iter__ method is not really supposed do anything but just return self. Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version? Edward

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 11:53 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***>; Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://github.com/proxy/avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Only in comprehensions. JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way: def __iter__(self): return self — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2DXTJJHYAPKHC34Y4L3EOVXTAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJQGAYDANJXGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

That works but it seems that the iter method is not really supposed do anything but just return self.

Yes, __iter__ method should be idempotent without side effects, so calling it twice gives the same result as calling it one time.

Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version?

It's already fixed on 3.14 branch, you can test on fresh beta: https://discuss.python.org/t/python-3-14-0-beta-3-is-here/95843

Navigation Menu

Repository navigation

Python 3.13.3 runs iter twice in a list comprehension #132711

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Python 3.13.3 runs __iter__ twice in a list comprehension #132711

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Activity

picnixz commented on Apr 18, 2025

picnixz commented on Apr 18, 2025

efimov-mikhail commented on Apr 19, 2025

fb2078 commented on Jun 19, 2025

efimov-mikhail commented on Jun 19, 2025

fb2078 commented on Jun 19, 2025

efimov-mikhail commented on Jun 20, 2025

fb2078 commented on Jun 21, 2025

fb2078 commented on Jun 21, 2025

efimov-mikhail commented on Jun 21, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Python 3.13.3 runs iter twice in a list comprehension #132711