FileClient¶
- class mmengine.fileio.FileClient(backend=None, prefix=None, **kwargs)[源代码]¶
A general file client to access files in different backends.
The client loads a file or text in a specified backend from its path and returns it as a binary or text file. There are two ways to choose a backend, the name of backend and the prefix of path. Although both of them can be used to choose a storage backend,
backend
has a higher priority that is if they are all set, the storage backend will be chosen by the backend argument. If they are all None, the disk backend will be chosen. Note that It can also register other backend accessor with a given name, prefixes, and backend class. In addition, We use the singleton pattern to avoid repeated object creation. If the arguments are the same, the same object will be returned.警告
FileClient will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
- 参数:
示例
>>> # only set backend >>> file_client = FileClient(backend='petrel') >>> # only set prefix >>> file_client = FileClient(prefix='s3') >>> # set both backend and prefix but use backend to choose client >>> file_client = FileClient(backend='petrel', prefix='s3') >>> # if the arguments are the same, the same object is returned >>> file_client1 = FileClient(backend='petrel') >>> file_client1 is file_client True
- client¶
The backend object.
- Type:
- get(filepath)[源代码]¶
Read data from a given
filepath
with ‘rb’ mode.备注
There are two types of return values for
get
, one isbytes
and the other ismemoryview
. The advantage of using memoryview is that you can avoid copying, and if you want to convert it tobytes
, you can use.tobytes()
.- 参数:
filepath (str or Path) – Path to read data.
- 返回:
Expected bytes object or a memory view of the bytes object.
- 返回类型:
- get_local_path(filepath)[源代码]¶
Download data from
filepath
and write the data to local path.get_local_path
is decorated bycontxtlib.contextmanager()
. It can be called withwith
statement, and when exists from thewith
statement, the temporary path will be released.备注
If the
filepath
is a local path, just return itself.警告
get_local_path
is an experimental interface that may change in the future.示例
>>> file_client = FileClient(prefix='s3') >>> with file_client.get_local_path('s3://bucket/abc.jpg') as path: ... # do something here
- classmethod infer_client(file_client_args=None, uri=None)[源代码]¶
Infer a suitable file client based on the URI and arguments.
- 参数:
- 返回类型:
示例
>>> uri = 's3://path/of/your/file' >>> file_client = FileClient.infer_client(uri=uri) >>> file_client_args = {'backend': 'petrel'} >>> file_client = FileClient.infer_client(file_client_args)
- 返回:
Instantiated FileClient object.
- 返回类型:
- 参数:
- join_path(filepath, *filepaths)[源代码]¶
Concatenate all file paths.
Join one or more filepath components intelligently. The return value is the concatenation of filepath and any members of *filepaths.
- list_dir_or_file(dir_path, list_dir=True, list_file=True, suffix=None, recursive=False)[源代码]¶
Scan a directory to find the interested directories or files in arbitrary order.
备注
list_dir_or_file()
returns the path relative todir_path
.- 参数:
dir_path (str | Path) – Path of the directory.
list_dir (bool) – List the directories. Defaults to True.
list_file (bool) – List the path of files. Defaults to True.
suffix (str or tuple[str], optional) – File suffix that we are interested in. Defaults to None.
recursive (bool) – If set to True, recursively scan the directory. Defaults to False.
- 生成器:
Iterable[str] – A relative path to
dir_path
.- 返回类型:
- static parse_uri_prefix(uri)[源代码]¶
Parse the prefix of a uri.
示例
>>> FileClient.parse_uri_prefix('s3://path/of/your/file') 's3'
- put(obj, filepath)[源代码]¶
Write data to a given
filepath
with ‘wb’ mode.备注
put
should create a directory if the directory offilepath
does not exist.
- put_text(obj, filepath)[源代码]¶
Write data to a given
filepath
with ‘w’ mode.备注
put_text
should create a directory if the directory offilepath
does not exist.
- classmethod register_backend(name, backend=None, force=False, prefixes=None)[源代码]¶
Register a backend to FileClient.
This method can be used as a normal class method or a decorator.
class NewBackend(BaseStorageBackend): def get(self, filepath): return filepath def get_text(self, filepath): return filepath FileClient.register_backend('new', NewBackend)
or
@FileClient.register_backend('new') class NewBackend(BaseStorageBackend): def get(self, filepath): return filepath def get_text(self, filepath): return filepath
- 参数:
name (str) – The name of the registered backend.
backend (class, optional) – The backend class to be registered, which must be a subclass of
BaseStorageBackend
. When this method is used as a decorator, backend is None. Defaults to None.force (bool, optional) – Whether to override the backend if the name has already been registered. Defaults to False.
prefixes (str or list[str] or tuple[str], optional) – The prefixes of the registered storage backend. Defaults to None. New in version 1.3.15.