pctasks.core.storage.blob.BlobStorage

class pctasks.core.storage.blob.BlobStorage(storage_account_name: str, container_name: str, prefix: Optional[str] = None, sas_token: Optional[str] = None, client_secret_credentials: Optional[ClientSecretCredentials] = None, account_url: Optional[str] = None)

Utility class for blob storage access. Represents access to a single blob container, with an optional prefix.

If a prefix is applied, then paths will be relative to that prefix. E.g. if BlobStorage for a BlobUri ‘blob://somesa/some-container/some/folder’ is used, then doing an operation like open_file on the file_path ‘file.txt’ will open the blob at ‘blob://somesa/some-container/some/folder/file.txt’.

Args:

• storage_account_name: The storage account name,

  e.g. ‘modissa’

• container_name: The container name to access.

• prefix: Optional prefix to base all paths off of.

• sas_token: Optional SAS token.

  If not supplied, uses DefaultAzureCredentials, in which you need to make sure the environment variables AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID are set.

• account_url: Optional account URL. If not supplied, uses

  the default Azure blob URL for a storage account.

Note:

If a service principle is used, make sure an appropriate IAM role (e.g. Storage Blob Data Contributor) on the storage account is assigned.

See Storage for method docstrings.

Parameters:

• storage_account_name (str) –

• container_name (str) –

• prefix (Optional[str]) –

• sas_token (Optional[str]) –

• client_secret_credentials (Optional[ClientSecretCredentials]) –

• account_url (Optional[str]) –

__init__(storage_account_name: str, container_name: str, prefix: Optional[str] = None, sas_token: Optional[str] = None, client_secret_credentials: Optional[ClientSecretCredentials] = None, account_url: Optional[str] = None) → None

Parameters:

• storage_account_name (str) –

• container_name (str) –

• prefix (Optional[str]) –

• sas_token (Optional[str]) –

• client_secret_credentials (Optional[ClientSecretCredentials]) –

• account_url (Optional[str]) –

Return type:

None

Methods

__init__(storage_account_name, container_name)
delete_file(file_path)	Deletes a file
delete_folder([folder_path])	Deletes a folder (directory or blob prefix)
download_file(file_path, output_path[, ...])	Downloads a file to a local file.
file_exists(file_path)	Returns True if a file exists.
from_account_key(blob_uri, account_key[, ...])
from_connection_string(connection_string, ...)
from_uri(blob_uri[, sas_token, ...])
fsspec_path(path)	Return the fsspec-style path.
get_authenticated_url(file_path[, read, ...])	Gets a URL with authentication information.
get_file_info(file_path)	Returns information for a file at the given path
get_path(uri)	Gets the path of a file from a given URI.
get_path_from_url(url)	Gets the path of a file from given HTTP URL.
get_substorage(path)	Creates a new Storage that is scoped to the given path
get_uri([file_path])	Returns the URI for a file at the file path, or the storage URI if no file path is given.
get_url(file_path)	Gets the http url of a file path.
list_files([name_starts_with, since_date, ...])	List file names.
read_bytes(file_path)	Reads bytes from a file in storage
read_json(file_path)	Reads a dict from a JSON file in storage. Args: file_path: Path to the JSON file.
read_ndjson(file_path)	Reads a list of dict from an NDJSON file in storage. Args: file_path: Path to the NDJSON file.
read_text(file_path)	Reads text from a file in storage
sign(href)	Appends a SAS Token to the HREFs.
upload_bytes(data, target_path[, overwrite])	Upload bytes to a storage file.
upload_code(file_path)	Upload a Python module or package.
upload_file(input_path, target_path[, ...])	Upload a file to blob storage.
walk([max_depth, min_depth, ...])	Recursively walk storage.
write_bytes(file_path, data[, overwrite])	Writes bytes to a file.
write_dict(file_path, d[, overwrite])	Writes a text file
write_text(file_path, text[, overwrite])	Writes a text file

Attributes

fsspec_storage_options	Return the fsspec storage options for this storage.
root_uri