Shortcuts

FileClient

class mmengine.fileio.FileClient(backend=None, prefix=None, **kwargs)[源代码]

A general file client to access files in different backends.

The client loads a file or text in a specified backend from its path and returns it as a binary or text file. There are two ways to choose a backend, the name of backend and the prefix of path. Although both of them can be used to choose a storage backend, backend has a higher priority that is if they are all set, the storage backend will be chosen by the backend argument. If they are all None, the disk backend will be chosen. Note that It can also register other backend accessor with a given name, prefixes, and backend class. In addition, We use the singleton pattern to avoid repeated object creation. If the arguments are the same, the same object will be returned.

警告

FileClient will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io

参数:
  • backend (str, optional) – The storage backend type. Options are “disk”, “memcached”, “lmdb”, “http” and “petrel”. Defaults to None.

  • prefix (str, optional) – The prefix of the registered storage backend. Options are “s3”, “http”, “https”. Defaults to None.

示例

>>> # only set backend
>>> file_client = FileClient(backend='petrel')
>>> # only set prefix
>>> file_client = FileClient(prefix='s3')
>>> # set both backend and prefix but use backend to choose client
>>> file_client = FileClient(backend='petrel', prefix='s3')
>>> # if the arguments are the same, the same object is returned
>>> file_client1 = FileClient(backend='petrel')
>>> file_client1 is file_client
True
client

The backend object.

Type:

BaseStorageBackend

exists(filepath)[源代码]

Check whether a file path exists.

参数:

filepath (str or Path) – Path to be checked whether exists.

返回:

Return True if filepath exists, False otherwise.

返回类型:

bool

get(filepath)[源代码]

Read data from a given filepath with ‘rb’ mode.

备注

There are two types of return values for get, one is bytes and the other is memoryview. The advantage of using memoryview is that you can avoid copying, and if you want to convert it to bytes, you can use .tobytes().

参数:

filepath (str or Path) – Path to read data.

返回:

Expected bytes object or a memory view of the bytes object.

返回类型:

bytes | memoryview

get_local_path(filepath)[源代码]

Download data from filepath and write the data to local path.

get_local_path is decorated by contxtlib.contextmanager(). It can be called with with statement, and when exists from the with statement, the temporary path will be released.

备注

If the filepath is a local path, just return itself.

警告

get_local_path is an experimental interface that may change in the future.

参数:

filepath (str or Path) – Path to be read data.

返回类型:

Generator[str | Path, None, None]

示例

>>> file_client = FileClient(prefix='s3')
>>> with file_client.get_local_path('s3://bucket/abc.jpg') as path:
...     # do something here
生成器:

Iterable[str] – Only yield one path.

参数:

filepath (str | Path) –

返回类型:

Generator[str | Path, None, None]

get_text(filepath, encoding='utf-8')[源代码]

Read data from a given filepath with ‘r’ mode.

参数:
  • filepath (str or Path) – Path to read data.

  • encoding (str) – The encoding format used to open the filepath. Defaults to ‘utf-8’.

返回:

Expected text reading from filepath.

返回类型:

str

classmethod infer_client(file_client_args=None, uri=None)[源代码]

Infer a suitable file client based on the URI and arguments.

参数:
  • file_client_args (dict, optional) – Arguments to instantiate a FileClient. Defaults to None.

  • uri (str | Path, optional) – Uri to be parsed that contains the file prefix. Defaults to None.

返回类型:

FileClient

示例

>>> uri = 's3://path/of/your/file'
>>> file_client = FileClient.infer_client(uri=uri)
>>> file_client_args = {'backend': 'petrel'}
>>> file_client = FileClient.infer_client(file_client_args)
返回:

Instantiated FileClient object.

返回类型:

FileClient

参数:
  • file_client_args (dict | None) –

  • uri (str | Path | None) –

isdir(filepath)[源代码]

Check whether a file path is a directory.

参数:

filepath (str or Path) – Path to be checked whether it is a directory.

返回:

Return True if filepath points to a directory, False otherwise.

返回类型:

bool

isfile(filepath)[源代码]

Check whether a file path is a file.

参数:

filepath (str or Path) – Path to be checked whether it is a file.

返回:

Return True if filepath points to a file, False otherwise.

返回类型:

bool

join_path(filepath, *filepaths)[源代码]

Concatenate all file paths.

Join one or more filepath components intelligently. The return value is the concatenation of filepath and any members of *filepaths.

参数:
  • filepath (str or Path) – Path to be concatenated.

  • filepaths (str | Path) –

返回:

The result of concatenation.

返回类型:

str

list_dir_or_file(dir_path, list_dir=True, list_file=True, suffix=None, recursive=False)[源代码]

Scan a directory to find the interested directories or files in arbitrary order.

备注

list_dir_or_file() returns the path relative to dir_path.

参数:
  • dir_path (str | Path) – Path of the directory.

  • list_dir (bool) – List the directories. Defaults to True.

  • list_file (bool) – List the path of files. Defaults to True.

  • suffix (str or tuple[str], optional) – File suffix that we are interested in. Defaults to None.

  • recursive (bool) – If set to True, recursively scan the directory. Defaults to False.

生成器:

Iterable[str] – A relative path to dir_path.

返回类型:

Iterator[str]

static parse_uri_prefix(uri)[源代码]

Parse the prefix of a uri.

参数:

uri (str | Path) – Uri to be parsed that contains the file prefix.

返回类型:

str | None

示例

>>> FileClient.parse_uri_prefix('s3://path/of/your/file')
's3'
返回:

Return the prefix of uri if the uri contains ‘://’ else None.

返回类型:

str | None

参数:

uri (str | Path) –

put(obj, filepath)[源代码]

Write data to a given filepath with ‘wb’ mode.

备注

put should create a directory if the directory of filepath does not exist.

参数:
  • obj (bytes) – Data to be written.

  • filepath (str or Path) – Path to write data.

返回类型:

None

put_text(obj, filepath)[源代码]

Write data to a given filepath with ‘w’ mode.

备注

put_text should create a directory if the directory of filepath does not exist.

参数:
  • obj (str) – Data to be written.

  • filepath (str or Path) – Path to write data.

  • encoding (str, optional) – The encoding format used to open the filepath. Defaults to ‘utf-8’.

返回类型:

None

classmethod register_backend(name, backend=None, force=False, prefixes=None)[源代码]

Register a backend to FileClient.

This method can be used as a normal class method or a decorator.

class NewBackend(BaseStorageBackend):

    def get(self, filepath):
        return filepath

    def get_text(self, filepath):
        return filepath

FileClient.register_backend('new', NewBackend)

or

@FileClient.register_backend('new')
class NewBackend(BaseStorageBackend):

    def get(self, filepath):
        return filepath

    def get_text(self, filepath):
        return filepath
参数:
  • name (str) – The name of the registered backend.

  • backend (class, optional) – The backend class to be registered, which must be a subclass of BaseStorageBackend. When this method is used as a decorator, backend is None. Defaults to None.

  • force (bool, optional) – Whether to override the backend if the name has already been registered. Defaults to False.

  • prefixes (str or list[str] or tuple[str], optional) – The prefixes of the registered storage backend. Defaults to None. New in version 1.3.15.

remove(filepath)[源代码]

Remove a file.

参数:

filepath (str, Path) – Path to be removed.

返回类型:

None

Read the Docs v: latest
Versions
latest
stable
v0.10.3
v0.10.2
v0.10.1
v0.10.0
v0.9.1
v0.9.0
v0.8.5
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.4
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.3.0
v0.2.0
Downloads
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.