fs.walk¶
Machinery for walking a filesystem.
Walking a filesystem means recursively visiting a directory and any sub-directories. It is a fairly common requirement for copying, searching etc. See Walking for details.
- class fs.walk.BoundWalker(fs: _F, walker_class: Type[Walker] = <class 'fs.walk.Walker'>)[source]¶
A class that binds a
Walker
instance to aFS
instance.You will typically not need to create instances of this class explicitly. Filesystems have a
walk
property which returns aBoundWalker
object.Example
>>> tmp_fs = fs.tempfs.TempFS() >>> tmp_fs.walk BoundWalker(TempFS())
A
BoundWalker
is callable. Calling it is an alias for thewalk
method.- __call__(path: Text = '/', namespaces: Optional[Collection[Text]] = None, **kwargs: Any) → Iterator[Step]¶
Walk the directory structure of a filesystem.
- Parameters
path (str) –
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
(defaults to['basic']
).
- Keyword Arguments
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable) – If
ignore_errors
isFalse
, then this callable will be invoked with a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
'breadth'
then the directory will be walked top down. Set to'depth'
to walk bottom up.filter (list) – If supplied, this parameter should be a list of file name patterns, e.g.
['*.py']
. Files will only be returned if the final component matches one of the patterns.exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
['~*', '.*']
. Files matching any of these patterns will be removed from the walk.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- Returns
an iterator of
(<path>, <dirs>, <files>)
named tuples, where<path>
is an absolute path to a directory, and<dirs>
and<files>
are a list ofInfo
objects for directories and files in<path>
.- Return type
Iterator
Example
>>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=['details']): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ...
This method invokes
Walker.walk
with boundFS
object.
- __init__(fs: _F, walker_class: Type[Walker] = <class 'fs.walk.Walker'>) → None[source]¶
Create a new walker bound to the given filesystem.
- dirs(path: Text = '/', **kwargs: Any) → Iterator[Text][source]¶
Walk a filesystem, yielding absolute paths to directories.
- Parameters
path (str) – A path to a directory.
- Keyword Arguments
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable) – If
ignore_errors
isFalse
, then this callable will be invoked with a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
'breadth'
then the directory will be walked top down. Set to'depth'
to walk bottom up.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- Returns
an iterator over directory paths (absolute from the filesystem root).
- Return type
Iterator
This method invokes
Walker.dirs
with the boundFS
object.
- files(path: Text = '/', **kwargs: Any) → Iterator[Text][source]¶
Walk a filesystem, yielding absolute paths to files.
- Parameters
path (str) – A path to a directory.
- Keyword Arguments
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable) – If
ignore_errors
isFalse
, then this callable will be invoked with a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
'breadth'
then the directory will be walked top down. Set to'depth'
to walk bottom up.filter (list) – If supplied, this parameter should be a list of file name patterns, e.g.
['*.py']
. Files will only be returned if the final component matches one of the patterns.exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
['~*', '.*']
. Files matching any of these patterns will be removed from the walk.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- Returns
An iterator over file paths (absolute from the filesystem root).
- Return type
Iterator
This method invokes
Walker.files
with the boundFS
object.
- info(path: Text = '/', namespaces: Optional[Collection[Text]] = None, **kwargs: Any) → Iterator[Tuple[Text, Info]][source]¶
Walk a filesystem, yielding path and
Info
of resources.- Parameters
path (str) – A path to a directory.
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
(defaults to['basic']
).
- Keyword Arguments
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable) – If
ignore_errors
isFalse
, then this callable will be invoked with a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
'breadth'
then the directory will be walked top down. Set to'depth'
to walk bottom up.filter (list) – If supplied, this parameter should be a list of file name patterns, e.g.
['*.py']
. Files will only be returned if the final component matches one of the patterns.exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
['~*', '.*']
. Files matching any of these patterns will be removed from the walk.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- Returns
an iterable yielding tuples of
(<absolute path>, <resource info>)
.- Return type
Iterable
This method invokes
Walker.info
with the boundFS
object.
- walk(path: Text = '/', namespaces: Optional[Collection[Text]] = None, **kwargs: Any) → Iterator[Step][source]¶
Walk the directory structure of a filesystem.
- Parameters
path (str) –
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
(defaults to['basic']
).
- Keyword Arguments
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable) – If
ignore_errors
isFalse
, then this callable will be invoked with a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
'breadth'
then the directory will be walked top down. Set to'depth'
to walk bottom up.filter (list) – If supplied, this parameter should be a list of file name patterns, e.g.
['*.py']
. Files will only be returned if the final component matches one of the patterns.exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
['~*', '.*']
. Files matching any of these patterns will be removed from the walk.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- Returns
an iterator of
(<path>, <dirs>, <files>)
named tuples, where<path>
is an absolute path to a directory, and<dirs>
and<files>
are a list ofInfo
objects for directories and files in<path>
.- Return type
Iterator
Example
>>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=['details']): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ...
This method invokes
Walker.walk
with boundFS
object.
- class fs.walk.Step(path, dirs, files)¶
type: a step in a directory walk.
- dirs¶
Alias for field number 1
- files¶
Alias for field number 2
- path¶
Alias for field number 0
- class fs.walk.Walker(ignore_errors: bool = False, on_error: Optional[OnError] = None, search: Text = 'breadth', filter: Optional[List[Text]] = None, exclude: Optional[List[Text]] = None, filter_dirs: Optional[List[Text]] = None, exclude_dirs: Optional[List[Text]] = None, max_depth: Optional[int] = None)[source]¶
A walker object recursively lists directories in a filesystem.
- __init__(ignore_errors: bool = False, on_error: Optional[OnError] = None, search: Text = 'breadth', filter: Optional[List[Text]] = None, exclude: Optional[List[Text]] = None, filter_dirs: Optional[List[Text]] = None, exclude_dirs: Optional[List[Text]] = None, max_depth: Optional[int] = None) → None[source]¶
Create a new
Walker
instance.- Parameters
ignore_errors (bool) – If
True
, any errors reading a directory will be ignored, otherwise exceptions will be raised.on_error (callable, optional) – If
ignore_errors
isFalse
, then this callable will be invoked for a path and the exception object. It should returnTrue
to ignore the error, orFalse
to re-raise it.search (str) – If
"breadth"
then the directory will be walked top down. Set to"depth"
to walk bottom up.filter (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
["*.py"]
. Files will only be returned if the final component matches one of the patterns.exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g.
["~*"]
. Files matching any of these patterns will be removed from the walk.filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns.
exclude_dirs (list, optional) – A list of patterns that will be used to filter out directories from the walk. e.g.
['*.svn', '*.git']
.max_depth (int, optional) – Maximum directory depth to walk.
- classmethod bind(fs: fs.walk._F) → fs.walk.BoundWalker[fs.walk._F][source]¶
Bind a
Walker
instance to a given filesystem.This binds in instance of the Walker to a given filesystem, so that you won’t need to explicitly provide the filesystem as a parameter.
- Parameters
fs (FS) – A filesystem object.
- Returns
a bound walker.
- Return type
Examples
Use this method to explicitly bind a filesystem instance:
>>> walker = Walker.bind(my_fs) >>> for path in walker.files(filter=['*.py']): ... print(path) /foo.py /bar.py
Unless you have written a customized walker class, you will be unlikely to need to call this explicitly, as filesystem objects already have a
walk
attribute which is a bound walker object:>>> for path in my_fs.walk.files(filter=['*.py']): ... print(path) /foo.py /bar.py
- check_file(fs: FS, info: Info) → bool[source]¶
Check if a filename should be included.
Override to exclude files from the walk.
- check_open_dir(fs: FS, path: Text, info: Info) → bool[source]¶
Check if a directory should be opened.
Override to exclude directories from the walk.
- check_scan_dir(fs: FS, path: Text, info: Info) → bool[source]¶
Check if a directory should be scanned.
Override to omit scanning of certain directories. If a directory is omitted, it will appear in the walk but its files and sub-directories will not.
- dirs(fs: FS, path: Text = '/') → Iterator[Text][source]¶
Walk a filesystem, yielding absolute paths to directories.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
- Yields
str – absolute path to directories on the filesystem found recursively within the given directory.
- files(fs: FS, path: Text = '/') → Iterator[Text][source]¶
Walk a filesystem, yielding absolute paths to files.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
- Yields
str – absolute path to files on the filesystem found recursively within the given directory.
- info(fs: FS, path: Text = '/', namespaces: Optional[Collection[Text]] = None) → Iterator[Tuple[Text, Info]][source]¶
Walk a filesystem, yielding tuples of
(<path>, <info>)
.- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of additional namespaces to add to the
Info
objects.
- Yields
(str, Info) – a tuple of
(<absolute path>, <resource info>)
.
- walk(fs: FS, path: Text = '/', namespaces: Optional[Collection[Text]] = None) → Iterator[Step][source]¶
Walk the directory structure of a filesystem.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of additional namespaces to add to the
Info
objects.
- Returns
an iterator of
Step
instances.- Return type
collections.Iterator
The return value is an iterator of
(<path>, <dirs>, <files>)
named tuples, where<path>
is an absolute path to a directory, and<dirs>
and<files>
are a list ofInfo
objects for directories and files in<path>
.Example
>>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=["details"]): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ...
- fs.walk.walk(fs: FS, path: Text = '/', namespaces: Optional[Collection[Text]] = None) → Iterator[Step]¶
Walk the directory structure of a filesystem.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of additional namespaces to add to the
Info
objects.
- Returns
an iterator of
Step
instances.- Return type
collections.Iterator
The return value is an iterator of
(<path>, <dirs>, <files>)
named tuples, where<path>
is an absolute path to a directory, and<dirs>
and<files>
are a list ofInfo
objects for directories and files in<path>
.Example
>>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=["details"]): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ...
- fs.walk.walk_dirs()¶
Walk a filesystem, yielding absolute paths to directories.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
- Yields
str – absolute path to directories on the filesystem found recursively within the given directory.
- fs.walk.walk_files()¶
Walk a filesystem, yielding absolute paths to files.
- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
- Yields
str – absolute path to files on the filesystem found recursively within the given directory.
- fs.walk.walk_info(fs: FS, path: Text = '/', namespaces: Optional[Collection[Text]] = None) → Iterator[Tuple[Text, Info]]¶
Walk a filesystem, yielding tuples of
(<path>, <info>)
.- Parameters
fs (FS) – A filesystem instance.
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of additional namespaces to add to the
Info
objects.
- Yields
(str, Info) – a tuple of
(<absolute path>, <resource info>)
.