[pathlib] Option so that OSError does not block glob in pathlib library

@matt32106

BPO	46050
Nosy	@matt32106

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-12-11.16:25:35.541>
labels = ['3.8', 'type-feature', 'library']
title = '[pathlib] Option so that OSError does not block glob in pathlib library'
updated_at = <Date 2021-12-11.16:25:35.541>
user = 'https://github.com/matt32106'

bugs.python.org fields:

activity = <Date 2021-12-11.16:25:35.541>
actor = 'matt32106'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-12-11.16:25:35.541>
creator = 'matt32106'
dependencies = []
files = []
hgrepos = []
issue_num = 46050
keywords = []
message_count = 1.0
messages = ['408314']
nosy_count = 1.0
nosy_names = ['matt32106']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46050'
versions = ['Python 3.8']

Linked PRs

GH-90208: Suppress OSError exceptions from pathlib.Path.glob() #104141

Hi there,
ISSUE DESCRIPTION
when I browse starting from the linux root ('/')
path = pathlib.Path('/')
_glob = '**/*'
for p in path.glob(_glob):

The program stops on my machine because of OSError.
File "/usr/lib/python3.8/pathlib.py", line 535, in _select_from
entries = list(scandir_it)

OSError: [Errno 22] Invalid argument: '/proc/5916/task/5916/net'

Entering post mortem debugging...

/usr/lib/python3.8/pathlib.py(535)_select_from()
533 try:
534 with scandir(parent_path) as scandir_it:
--> 535 entries = list(scandir_it)
536 for entry in entries:
537 if self.dironly:

I also another case is if a cloud drive is disconnected.

QUICK SOLUTION
I solved both issues for me by adding an adding an except OSError: return at the end of the two function below:

class _WildcardSelector(_Selector):
    def _select_from(self, parent_path, is_dir, exists, scandir):
        (...)
        except OSError:
            return

class _RecursiveWildcardSelector(_Selector):
    def _iterate_directories(self, parent_path, is_dir, scandir):
        (...)
        except OSError:
            return

FEATURE REQUEST
I don't know the consequences of those 2 modifications so perhaps they could follow an option when calling glob with an optional parameter ignoreOSerror = false by default?!

IMO it should ignore all OSError exceptions that are generated below the top-level path. PR incoming...

pythonGH-90208 - Suppress OSError exceptions from `pathlib.Path.glob()`

Merge branch 'main' into pythongh-90208-pathlib-glob-oserror-ignore

GH-90208: Suppress OSError exceptions from `pathlib.Path.glob()` (GH-…

Fixed in #104141 / 94f30c7

Merge branch 'main' into supermon

Merge branch 'main' into fixloadsuperspec

Navigation Menu

Repository navigation

[pathlib] Option so that OSError does not block glob in pathlib library #90208

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[pathlib] Option so that OSError does not block glob in pathlib library #90208

Description

Linked PRs

Activity

matt32106 commented on Dec 11, 2021

barneygale commented on May 3, 2023

barneygale commented on May 11, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions