fs.wrapfs¶
Base class for filesystem wrappers.
- class fs.wrapfs.WrapFS(wrap_fs: fs.wrapfs._F)[source]¶
A proxy for a filesystem object.
This class exposes an filesystem interface, where the data is stored on another filesystem(s), and is the basis for
SubFS
and other virtual filesystems.- __init__(wrap_fs: fs.wrapfs._F) → None[source]¶
Create a filesystem. See help(type(self)) for accurate signature.
- appendbytes(path: Text, data: bytes) → None[source]¶
Append bytes to the end of a file, creating it if needed.
- Parameters
path (str) – Path to a file.
data (bytes) – Bytes to append.
- Raises
TypeError – If
data
is not abytes
instance.fs.errors.ResourceNotFound – If a parent directory of
path
does not exist.
- appendtext(path: Text, text: Text, encoding: Text = 'utf-8', errors: Optional[Text] = None, newline: Text = '') → None[source]¶
Append text to the end of a file, creating it if needed.
- Parameters
path (str) – Path to a file.
text (str) – Text to append.
encoding (str) – Encoding for text files (defaults to
utf-8
).errors (str, optional) – What to do with unicode decode errors (see
codecs
module for more information).newline (str) – Newline parameter.
- Raises
TypeError – if
text
is not an unicode string.fs.errors.ResourceNotFound – if a parent directory of
path
does not exist.
- copy(src_path: Text, dst_path: Text, overwrite: bool = False, preserve_time: bool = False) → None[source]¶
Copy file contents from
src_path
todst_path
.- Parameters
src_path (str) – Path of source file.
dst_path (str) – Path to destination file.
overwrite (bool) – If
True
, overwrite the destination file if it exists (defaults toFalse
).preserve_time (bool) – If
True
, try to preserve mtime of the resource (defaults toFalse
).
- Raises
fs.errors.DestinationExists – If
dst_path
exists, andoverwrite
isFalse
.fs.errors.ResourceNotFound – If a parent directory of
dst_path
does not exist.fs.errors.FileExpected – If
src_path
is not a file.
- copydir(src_path: Text, dst_path: Text, create: bool = False, preserve_time: bool = False) → None[source]¶
Copy the contents of
src_path
todst_path
.- Parameters
src_path (str) – Path of source directory.
dst_path (str) – Path to destination directory.
create (bool) – If
True
, thendst_path
will be created if it doesn’t exist already (defaults toFalse
).preserve_time (bool) – If
True
, try to preserve mtime of the resource (defaults toFalse
).
- Raises
fs.errors.ResourceNotFound – If the
dst_path
does not exist, andcreate
is notTrue
.fs.errors.DirectoryExpected – If
src_path
is not a directory.
- create(path: Text, wipe: bool = False) → bool[source]¶
Create an empty file.
The default behavior is to create a new file if one doesn’t already exist. If
wipe
isTrue
, any existing file will be truncated.- Parameters
path (str) – Path to a new file in the filesystem.
wipe (bool) – If
True
, truncate any existing file to 0 bytes (defaults toFalse
).
- Returns
True
if a new file had to be created.- Return type
bool
- delegate_fs() → fs.wrapfs._F[source]¶
Get the proxied filesystem.
This method should return a filesystem for methods not associated with a path, e.g.
getmeta
.
- delegate_path(path: Text) → Tuple[_F, Text][source]¶
Encode a path for proxied filesystem.
- Parameters
path (str) – A path on the filesystem.
- Returns
a tuple of
(<filesystem>, <new_path>)
- Return type
(FS, str)
- desc(path: Text) → Text[source]¶
Return a short descriptive text regarding a path.
- Parameters
path (str) – A path to a resource on the filesystem.
- Returns
a short description of the path.
- Return type
str
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.
- download(path: Text, file: BinaryIO, chunk_size: Optional[int] = None, **options: Any) → None[source]¶
Copy a file from the filesystem to a file-like object.
This may be more efficient that opening and copying files manually if the filesystem supplies an optimized method.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).- Parameters
path (str) – Path to a resource.
file (file-like) – A file-like object open for writing in binary mode.
chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or
None
to use sensible default.**options – Implementation specific options required to open the source file.
Example
>>> with open('starwars.mov', 'wb') as write_file: ... my_fs.download('/Videos/starwars.mov', write_file)
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
- exists(path: Text) → bool[source]¶
Check if a path maps to a resource.
- Parameters
path (str) – Path to a resource.
- Returns
True
if a resource exists at the given path.- Return type
bool
- filterdir(path: Text, files: Optional[Iterable[Text]] = None, dirs: Optional[Iterable[Text]] = None, exclude_dirs: Optional[Iterable[Text]] = None, exclude_files: Optional[Iterable[Text]] = None, namespaces: Optional[Collection[Text]] = None, page: Optional[Tuple[int, int]] = None) → Iterator[Info][source]¶
Get an iterator of resource info, filtered by patterns.
This method enhances
scandir
with additional filtering functionality.- Parameters
path (str) – A path to a directory on the filesystem.
files (list, optional) – A list of UNIX shell-style patterns to filter file names, e.g.
['*.py']
.dirs (list, optional) – A list of UNIX shell-style patterns to filter directory names.
exclude_dirs (list, optional) – A list of patterns used to exclude directories.
exclude_files (list, optional) – A list of patterns used to exclude files.
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
.page (tuple, optional) – May be a tuple of
(<start>, <end>)
indexes to return an iterator of a subset of the resource info, orNone
to iterate over the entire directory. Paging a directory scan may be necessary for very large directories.
- Returns
an iterator of
Info
objects.- Return type
Iterator
- getinfo(path: Text, namespaces: Optional[Collection[Text]] = None) → Info[source]¶
Get information about a resource on a filesystem.
- Parameters
path (str) – A path to a resource on the filesystem.
namespaces (list, optional) – Info namespaces to query. The
"basic"
namespace is alway included in the returned info, whatever the value ofnamespaces
may be.
- Returns
resource information object.
- Return type
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.
For more information regarding resource information, see Resource Info.
- getmeta(namespace: Text = 'standard') → Mapping[Text, object][source]¶
Get meta information regarding a filesystem.
- Parameters
namespace (str) – The meta namespace (defaults to
"standard"
).- Returns
the meta information.
- Return type
dict
Meta information is associated with a namespace which may be specified with the
namespace
parameter. The default namespace,"standard"
, contains common information regarding the filesystem’s capabilities. Some filesystems may provide other namespaces which expose less common or implementation specific information. If a requested namespace is not supported by a filesystem, then an empty dictionary will be returned.The
"standard"
namespace supports the following keys:key
Description
case_insensitive
True
if this filesystem is case insensitive.invalid_path_chars
A string containing the characters that may not be used on this filesystem.
max_path_length
Maximum number of characters permitted in a path, or
None
for no limit.max_sys_path_length
Maximum number of characters permitted in a sys path, or
None
for no limit.network
True
if this filesystem requires a network.read_only
True
if this filesystem is read only.supports_rename
True
if this filesystem supports anos.rename
operation.Most builtin filesystems will provide all these keys, and third- party filesystems should do so whenever possible, but a key may not be present if there is no way to know the value.
Note
Meta information is constant for the lifetime of the filesystem, and may be cached.
- getsize(path: Text) → int[source]¶
Get the size (in bytes) of a resource.
- Parameters
path (str) – A path to a resource.
- Returns
the size of the resource.
- Return type
int
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
The size of a file is the total number of readable bytes, which may not reflect the exact number of bytes of reserved disk space (or other storage medium).
The size of a directory is the number of bytes of overhead use to store the directory entry.
- getsyspath(path: Text) → Text[source]¶
Get the system path of a resource.
- Parameters
path (str) – A path on the filesystem.
- Returns
the system path of the resource, if any.
- Return type
str
- Raises
fs.errors.NoSysPath – If there is no corresponding system path.
A system path is one recognized by the OS, that may be used outside of PyFilesystem (in an application or a shell for example). This method will get the corresponding system path that would be referenced by
path
.Not all filesystems have associated system paths. Network and memory based filesystems, for example, may not physically store data anywhere the OS knows about. It is also possible for some paths to have a system path, whereas others don’t.
This method will always return a str on Py3.* and unicode on Py2.7. See
getospath
if you need to encode the path as bytes.If
path
doesn’t have a system path, aNoSysPath
exception will be thrown.Note
A filesystem may return a system path even if no resource is referenced by that path – as long as it can be certain what that system path would be.
- gettype(path: Text) → ResourceType[source]¶
Get the type of a resource.
- Parameters
path (str) – A path on the filesystem.
- Returns
the type of the resource.
- Return type
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
A type of a resource is an integer that identifies the what the resource references. The standard type integers may be one of the values in the
ResourceType
enumerations.The most common resource types, supported by virtually all filesystems are
directory
(1) andfile
(2), but the following types are also possible:ResourceType
value
unknown
0
directory
1
file
2
character
3
block_special_file
4
fifo
5
socket
6
symlink
7
Standard resource types are positive integers, negative values are reserved for implementation specific resource types.
- geturl(path: Text, purpose: Text = 'download') → Text[source]¶
Get the URL to a given resource.
- Parameters
path (str) – A path on the filesystem
purpose (str) – A short string that indicates which URL to retrieve for the given path (if there is more than one). The default is
'download'
, which should return a URL that serves the file. Other filesystems may support other values forpurpose
: for instance,OSFS
supports'fs'
, which returns a FS URL (see FS URLs).
- Returns
a URL.
- Return type
str
- Raises
fs.errors.NoURL – If the path does not map to a URL.
- hash(path: Text, name: Text) → Text[source]¶
Get the hash of a file’s contents.
- Parameters
path (str) – A path on the filesystem.
name (str) – One of the algorithms supported by the
hashlib
module, e.g."md5"
or"sha256"
.
- Returns
The hex digest of the hash.
- Return type
str
- Raises
fs.errors.UnsupportedHash – If the requested hash is not supported.
fs.errors.ResourceNotFound – If
path
does not exist.fs.errors.FileExpected – If
path
exists but is not a file.
- hassyspath(path: Text) → bool[source]¶
Check if a path maps to a system path.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
if the resource atpath
has a syspath.- Return type
bool
- hasurl(path: Text, purpose: Text = 'download') → bool[source]¶
Check if a path has a corresponding URL.
- Parameters
path (str) – A path on the filesystem.
purpose (str) – A purpose parameter, as given in
geturl
.
- Returns
True
if an URL for the given purpose exists.- Return type
bool
- isdir(path: Text) → bool[source]¶
Check if a path maps to an existing directory.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
ifpath
maps to a directory.- Return type
bool
- isfile(path: Text) → bool[source]¶
Check if a path maps to an existing file.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
ifpath
maps to a file.- Return type
bool
- islink(path: Text) → bool[source]¶
Check if a path maps to a symlink.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
ifpath
maps to a symlink.- Return type
bool
- listdir(path: Text) → List[Text][source]¶
Get a list of the resource names in a directory.
This method will return a list of the resources in a directory. A resource is a file, directory, or one of the other types defined in
ResourceType
.- Parameters
path (str) – A path to a directory on the filesystem
- Returns
list of names, relative to
path
.- Return type
list
- Raises
fs.errors.DirectoryExpected – If
path
is not a directory.fs.errors.ResourceNotFound – If
path
does not exist.
- lock() → RLock[source]¶
Get a context manager that locks the filesystem.
Locking a filesystem gives a thread exclusive access to it. Other threads will block until the threads with the lock has left the context manager.
- Returns
a lock specific to the filesystem instance.
- Return type
threading.RLock
Example
>>> with my_fs.lock(): # May block ... # code here has exclusive access to the filesystem ... pass
It is a good idea to put a lock around any operations that you would like to be atomic. For instance if you are copying files, and you don’t want another thread to delete or modify anything while the copy is in progress.
Locking with this method is only required for code that calls multiple filesystem methods. Individual methods are thread safe already, and don’t need to be locked.
Note
This only locks at the Python level. There is nothing to prevent other processes from modifying the filesystem outside of the filesystem instance.
- makedir(path: Text, permissions: Optional[Permissions] = None, recreate: bool = False) → SubFS[FS][source]¶
Make a directory.
- Parameters
path (str) – Path to directory from root.
permissions (Permissions, optional) – a
Permissions
instance, orNone
to use default.recreate (bool) – Set to
True
to avoid raising an error if the directory already exists (defaults toFalse
).
- Returns
a filesystem whose root is the new directory.
- Return type
- Raises
fs.errors.DirectoryExists – If the path already exists.
fs.errors.ResourceNotFound – If the path is not found.
- makedirs(path: Text, permissions: Optional[Permissions] = None, recreate: bool = False) → SubFS[FS][source]¶
Make a directory, and any missing intermediate directories.
- Parameters
path (str) – Path to directory from root.
permissions (Permissions, optional) – Initial permissions, or
None
to use defaults.recreate (bool) – If
False
(the default), attempting to create an existing directory will raise an error. Set toTrue
to ignore existing directories.
- Returns
A sub-directory filesystem.
- Return type
- Raises
fs.errors.DirectoryExists – if the path is already a directory, and
recreate
isFalse
.fs.errors.DirectoryExpected – if one of the ancestors in the path is not a directory.
- move(src_path: Text, dst_path: Text, overwrite: bool = False, preserve_time: bool = False) → None[source]¶
Move a file from
src_path
todst_path
.- Parameters
src_path (str) – A path on the filesystem to move.
dst_path (str) – A path on the filesystem where the source file will be written to.
overwrite (bool) – If
True
, destination path will be overwritten if it exists.preserve_time (bool) – If
True
, try to preserve mtime of the resources (defaults toFalse
).
- Raises
fs.errors.FileExpected – If
src_path
maps to a directory instead of a file.fs.errors.DestinationExists – If
dst_path
exists, andoverwrite
isFalse
.fs.errors.ResourceNotFound – If a parent directory of
dst_path
does not exist.
- movedir(src_path: Text, dst_path: Text, create: bool = False, preserve_time: bool = False) → None[source]¶
Move directory
src_path
todst_path
.- Parameters
src_path (str) – Path of source directory on the filesystem.
dst_path (str) – Path to destination directory.
create (bool) – If
True
, thendst_path
will be created if it doesn’t exist already (defaults toFalse
).preserve_time (bool) – If
True
, try to preserve mtime of the resources (defaults toFalse
).
- Raises
fs.errors.ResourceNotFound – if
dst_path
does not exist, andcreate
isFalse
.fs.errors.DirectoryExpected – if
src_path
or one of its ancestors is not a directory.
- open(path: Text, mode: Text = 'r', buffering: int = - 1, encoding: Optional[Text] = None, errors: Optional[Text] = None, newline: Text = '', line_buffering: bool = False, **options: Any) → IO[AnyStr][source]¶
Open a file.
- Parameters
path (str) – A path to a file on the filesystem.
mode (str) – Mode to open the file object with (defaults to r).
buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size).
encoding (str) – Encoding for text files (defaults to
utf-8
)errors (str, optional) – What to do with unicode decode errors (see
codecs
module for more information).newline (str) – Newline parameter.
**options – keyword arguments for any additional information required by the filesystem (if any).
- Returns
a file-like object.
- Return type
io.IOBase
- Raises
fs.errors.FileExpected – If the path is not a file.
fs.errors.FileExists – If the file exists, and exclusive mode is specified (
x
in the mode).fs.errors.ResourceNotFound – If the path does not exist.
- openbin(path: Text, mode: Text = 'r', buffering: int = - 1, **options: Any) → BinaryIO[source]¶
Open a binary file-like object.
- Parameters
path (str) – A path on the filesystem.
mode (str) – Mode to open file (must be a valid non-text mode, defaults to r). Since this method only opens binary files, the
b
in the mode string is implied.buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, or any positive integer to indicate a buffer size).
**options – keyword arguments for any additional information required by the filesystem (if any).
- Returns
a file-like object.
- Return type
io.IOBase
- Raises
fs.errors.FileExpected – If
path
exists and is not a file.fs.errors.FileExists – If the
path
exists, and exclusive mode is specified (x
in the mode).fs.errors.ResourceNotFound – If
path
does not exist andmode
does not imply creating the file, or if any ancestor ofpath
does not exist.
- opendir(path: Text, factory: Optional[_OpendirFactory] = None) → SubFS[_W][source]¶
Get a filesystem object for a sub-directory.
- Parameters
path (str) – Path to a directory on the filesystem.
factory (callable, optional) – A callable that when invoked with an FS instance and
path
will return a new FS object representing the sub-directory contents. If nofactory
is supplied thensubfs_class
will be used.
- Returns
A filesystem representing a sub-directory.
- Return type
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.fs.errors.DirectoryExpected – If
path
is not a directory.
- readbytes(path: Text) → bytes[source]¶
Get the contents of a file as bytes.
- Parameters
path (str) – A path to a readable file on the filesystem.
- Returns
the file contents.
- Return type
bytes
- Raises
fs.errors.FileExpected – if
path
exists but is not a file.fs.errors.ResourceNotFound – if
path
does not exist.
- readtext(path: Text, encoding: Optional[Text] = None, errors: Optional[Text] = None, newline: Text = '') → Text[source]¶
Get the contents of a file as a string.
- Parameters
path (str) – A path to a readable file on the filesystem.
encoding (str, optional) – Encoding to use when reading contents in text mode (defaults to
None
, reading in binary mode).errors (str, optional) – Unicode errors parameter.
newline (str) – Newlines parameter.
- Returns
file contents.
- Return type
str
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.
- remove(path: Text) → None[source]¶
Remove a file from the filesystem.
- Parameters
path (str) – Path of the file to remove.
- Raises
fs.errors.FileExpected – If the path is a directory.
fs.errors.ResourceNotFound – If the path does not exist.
- removedir(path: Text) → None[source]¶
Remove a directory from the filesystem.
- Parameters
path (str) – Path of the directory to remove.
- Raises
fs.errors.DirectoryNotEmpty – If the directory is not empty ( see
removetree
for a way to remove the directory contents).fs.errors.DirectoryExpected – If the path does not refer to a directory.
fs.errors.ResourceNotFound – If no resource exists at the given path.
fs.errors.RemoveRootError – If an attempt is made to remove the root directory (i.e.
'/'
)
- removetree(dir_path: Text) → None[source]¶
Recursively remove a directory and all its contents.
This method is similar to
removedir
, but will remove the contents of the directory if it is not empty.- Parameters
dir_path (str) – Path to a directory on the filesystem.
- Raises
fs.errors.ResourceNotFound – If
dir_path
does not exist.fs.errors.DirectoryExpected – If
dir_path
is not a directory.
Caution
A filesystem should never delete its root folder, so
FS.removetree("/")
has different semantics: the contents of the root folder will be deleted, but the root will be untouched:>>> home_fs = fs.open_fs("~") >>> home_fs.removetree("/") >>> home_fs.exists("/") True >>> home_fs.isempty("/") True
Combined with
opendir
, this can be used to clear a directory without removing the directory itself:>>> home_fs = fs.open_fs("~") >>> home_fs.opendir("/Videos").removetree("/") >>> home_fs.exists("/Videos") True >>> home_fs.isempty("/Videos") True
- scandir(path: Text, namespaces: Optional[Collection[Text]] = None, page: Optional[Tuple[int, int]] = None) → Iterator[Info][source]¶
Get an iterator of resource info.
- Parameters
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
.page (tuple, optional) – May be a tuple of
(<start>, <end>)
indexes to return an iterator of a subset of the resource info, orNone
to iterate over the entire directory. Paging a directory scan may be necessary for very large directories.
- Returns
an iterator of
Info
objects.- Return type
Iterator
- Raises
fs.errors.DirectoryExpected – If
path
is not a directory.fs.errors.ResourceNotFound – If
path
does not exist.
- setinfo(path: Text, info: RawInfo) → None[source]¶
Set info on a resource.
This method is the complement to
getinfo
and is used to set info values on a resource.- Parameters
path (str) – Path to a resource on the filesystem.
info (dict) – Dictionary of resource info.
- Raises
fs.errors.ResourceNotFound – If
path
does not exist on the filesystem
The
info
dict should be in the same format as the raw info returned bygetinfo(file).raw
.Example
>>> details_info = {"details": { ... "modified": time.time() ... }} >>> my_fs.setinfo('file.txt', details_info)
- settimes(path: Text, accessed: Optional[datetime] = None, modified: Optional[datetime] = None) → None[source]¶
Set the accessed and modified time on a resource.
- Parameters
path – A path to a resource on the filesystem.
accessed (datetime, optional) – The accessed time, or
None
(the default) to use the current time.modified (datetime, optional) – The modified time, or
None
(the default) to use the same time as theaccessed
parameter.
- touch(path: Text) → None[source]¶
Touch a file on the filesystem.
Touching a file means creating a new file if
path
doesn’t exist, or update accessed and modified times if the path does exist. This method is similar to the linux command of the same name.- Parameters
path (str) – A path to a file on the filesystem.
- upload(path: Text, file: BinaryIO, chunk_size: Optional[int] = None, **options: Any) → None[source]¶
Set a file to the contents of a binary file object.
This method copies bytes from an open binary file to a file on the filesystem. If the destination exists, it will first be truncated.
- Parameters
path (str) – A path on the filesystem.
file (io.IOBase) – a file object open for reading in binary mode.
chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or
None
to use sensible default.**options – Implementation specific options required to open the source file.
- Raises
fs.errors.ResourceNotFound – If a parent directory of
path
does not exist.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).Example
>>> with open('~/movies/starwars.mov', 'rb') as read_file: ... my_fs.upload('starwars.mov', read_file)
- validatepath(path: Text) → Text[source]¶
Validate a path, returning a normalized absolute path on sucess.
Many filesystems have restrictions on the format of paths they support. This method will check that
path
is valid on the underlaying storage mechanism and throw aInvalidPath
exception if it is not.- Parameters
path (str) – A path.
- Returns
A normalized, absolute path.
- Return type
str
- Raises
fs.errors.InvalidPath – If the path is invalid.
fs.errors.FilesystemClosed – if the filesystem is closed.
fs.errors.InvalidCharsInPath – If the path contains invalid characters.
- property walk¶
a walker bound to this filesystem.
- Type
- writebytes(path: Text, contents: bytes) → None[source]¶
Copy binary data to a file.
- Parameters
path (str) – Destination path on the filesystem.
contents (bytes) – Data to be written.
- Raises
TypeError – if contents is not bytes.
- writefile(path: Text, file: IO[AnyStr], encoding: Optional[Text] = None, errors: Optional[Text] = None, newline: Text = '') → None[source]¶
Set a file to the contents of a file object.
- Parameters
path (str) – A path on the filesystem.
file (io.IOBase) – A file object open for reading.
encoding (str, optional) – Encoding of destination file, defaults to
None
for binary.errors (str, optional) – How encoding errors should be treated (same as
io.open
).newline (str) – Newline parameter (same as
io.open
).
This method is similar to
upload
, in that it copies data from a file-like object to a resource on the filesystem, but unlikeupload
, this method also supports creating files in text-mode (if theencoding
argument is supplied).Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).Example
>>> with open('myfile.txt') as read_file: ... my_fs.writefile('myfile.txt', read_file)