pctasks.core.storage.Storage#

class pctasks.core.storage.Storage#

Abstraction over storage.

Represents storage that is scoped to a storage base, which may be a directory, a storage account and container, a storage account, container, and a prefix, etc.

Some terminology Storage uses:

  • path: A path relative to the storage base

  • url: The http(s) URL for a path (to be renamed to href)

  • uri: A resource ID that can use non-http schemes; e.g. a pctasks.core.storage.blob.BlobUri like "blob://storage_account/container/prefix/blob.txt"

__init__()#

Methods

__init__()

delete_file(file_path)

Deletes a file

delete_folder(folder_path)

Deletes a folder (directory or blob prefix)

download_file(file_path, output_path[, ...])

Downloads a file to a local file.

file_exists(file_path)

Returns True if a file exists.

fsspec_path(path)

Return the fsspec-style URL.

get_authenticated_url(file_path)

Gets a URL with authentication information.

get_file_info(file_path)

Returns information for a file at the given path

get_path(uri)

Gets the path of a file from a given URI.

get_path_from_url(url)

Gets the path of a file from given HTTP URL.

get_substorage(path)

Creates a new Storage that is scoped to the given path

get_uri([file_path])

Returns the URI for a file at the file path, or the storage URI if no file path is given.

get_url(file_path)

Gets the http url of a file path.

list_files([name_starts_with, since_date, ...])

List file names.

read_bytes(file_path)

Reads bytes from a file in storage

read_json(file_path)

Reads a dict from a JSON file in storage. Args: file_path: Path to the JSON file.

read_ndjson(file_path)

Reads a list of dict from an NDJSON file in storage. Args: file_path: Path to the NDJSON file.

read_text(file_path)

Reads text from a file in storage

sign(href)

Appends a SAS Token to the HREFs.

upload_bytes(data, target_path[, overwrite])

Upload bytes to a storage file.

upload_code(file_path)

Upload a Python module or package.

upload_file(input_path, target_path[, overwrite])

Upload a local file to a storage file.

walk([max_depth, min_depth, ...])

Recursively walk storage.

write_bytes(file_path, data[, overwrite])

Writes bytes to a file.

write_dict(file_path, d[, overwrite])

Writes a text file

write_text(file_path, text[, overwrite])

Writes a text file

Attributes

fsspec_storage_options

Return the fsspec storage options for this storage.