airfs.storage.azure_file

Microsoft Azure Storage File

New in version 1.3.0.

Mount

An Azure storage account can be mounted using the airfs mount function.

storage_parameters await arguments to pass to the azure.storage.file.fileservice.FileService class from azure-storage-file Python library.

This example shows the mount of Azure Storage File with the minimal configuration:

import airfs

# Mount Azure Storage File manually (Minimal configuration)
airfs.mount(storage='azure_file', storage_parameters=dict(
        account_name='my_account_name', account_key='my_account_key'
    )
)

# Call of airfs on an Azure Storage file.
with airfs.open(
        '//my_account.file.core.windows.net/my_share/my_file', 'rt') as file:
    text = file.read()

If using multiple Azure storage accounts simultaneously, the sas_token argument of the FileService class is required to allow blob and files copies across different accounts.

It is possible to mount Azure Storage Blob and Azure Storage File with a single airfs.mount call by using storage='azure' instead of storage='azure_file'.

Limitation

Only one configuration per Azure Storage account can be mounted simultaneously.

Pre-allocating files

When flushing a file out of its current size, airfs first resize the file to allow the flush of the new data.

In case of multiple flushes on a raw IO or when using a buffered IO, this is done with extra requests to the Azure server. If The size to write is known before opening the file, it is possible to avoid these extra requests by to preallocate the required size in only one initial request.

The content_length argument allow pre-allocating a file to a defined size when opening it in write mode:

# Open a new file and preallocate it with 1024 bytes.
with airfs.open(
        '//my_account.file.core.windows.net/my_share/my_file', 'wb',
        content_length=1024
        ) as file:
    file.write(b'1')

# Append on an existing file and pre-resize it to 2048 bytes.
with airfs.open(
        '//my_account.file.core.windows.net/my_share/my_file', 'ab',
        content_length=2048
        ) as file:
    file.write(b'1')

The pre-allocation is done with padding of null characters (b'\0').

Files objects classes

Microsoft Azure Files Storage.

class airfs.storage.azure_file.AzureFileBufferedIO(name, mode='r', buffer_size=None, max_buffers=0, max_workers=None, **kwargs)[source]

Buffered binary Azure Files Storage Object I/O.

Parameters:

  • name (path-like object) – URL or path to the file which will be opened.

  • mode (str) – The mode can be ‘r’, ‘w’ for reading (default) or writing

  • buffer_size (int) – The size of buffer.

  • max_buffers (int) – The maximum number of buffers to preload in read mode or awaiting flush in “write” mode. 0 for no limit.

  • max_workers (int) – The maximum number of threads that can be used to execute the given calls.

  • storage_parameters (dict) – Azure service keyword arguments. This is generally Azure credentials and configuration. See “azure.storage.file.fileservice.FileService” for more information.

  • unsecure (bool) – If True, disables TLS/SSL to improve transfer performance. But makes connection unsecure.

  • content_length (int) – Define the size to preallocate on new file creation. This is not mandatory, and file will be resized on needs, but this allows to improve performance when file size is known in advance.

MAXIMUM_BUFFER_SIZE = 4194304

Maximal buffer_size value in bytes (Maximum upload range size)

close()

Flush the write buffers of the stream if applicable and close the object.

detach()

Disconnect this buffer from its underlying raw stream and return it.

After the raw stream has been detached, the buffer is in an unusable state.

fileno()

Returns underlying file descriptor if one exists.

OSError is raised if the IO object does not use a file descriptor.

flush()

Flush the write buffers of the stream if applicable.

isatty()

Return whether this is an ‘interactive’ stream.

Return False if it can’t be determined.

property mode

The mode.

Returns:

Mode.

Return type:

str

property name

The file name.

Returns:

Name.

Return type:

str

peek(size=-1)

Return bytes from the stream without advancing the position.

Parameters:

size (int) – Number of bytes to read. -1 to read the full stream.

Returns:

bytes read

Return type:

bytes

property raw

The underlying raw stream.

Returns:

Raw stream.

Return type:

ObjectRawIOBase subclass

read(size=-1)

Read the object content.

Read and return up to size bytes, with at most one call to the underlying raw stream.

Use at most one call to the underlying raw stream’s read method.

Parameters:

size (int) – Number of bytes to read. -1 to read the stream until the end.

Returns:

Object content

Return type:

bytes

read1(size=-1)

Read the object content.

Read and return up to size bytes, with at most one call to the underlying raw stream.

Use at most one call to the underlying raw stream’s read method.

Parameters:

size (int) – Number of bytes to read. -1 to read the stream until the end.

Returns:

Object content

Return type:

bytes

readable()

Return True if the stream can be read from.

If False, read() will raise OSError.

Returns:

Supports reading.

Return type:

bool

readinto(b)

Read the object content into a buffer.

Read bytes into a pre-allocated, writable bytes-like object b, and return the number of bytes read.

Parameters:

b (bytes-like object) – buffer.

Returns:

number of bytes read

Return type:

int

readinto1(b)

Read the object content into a buffer.

Read bytes into a pre-allocated, writable bytes-like object b, and return the number of bytes read.

Use at most one call to the underlying raw stream’s readinto method.

Parameters:

b (bytes-like object) – buffer.

Returns:

number of bytes read

Return type:

int

readline(size=-1, /)

Read and return a line from the stream.

If size is specified, at most size bytes will be read.

The line terminator is always b’n’ for binary files; for text files, the newlines argument to open can be used to select the line terminator(s) recognized.

readlines(hint=-1, /)

Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

seek(offset, whence=0)

Change the stream position to the given byte offset.

Parameters:

  • offset – Offset is interpreted relative to the position indicated by whence.

  • whence – The default value for whence is SEEK_SET. Values are: SEEK_SET or 0 – Start of the stream (the default); offset should be zero or positive SEEK_CUR or 1 – Current stream position; offset may be negative SEEK_END or 2 – End of the stream; offset is usually negative

Returns:

The new absolute position.

Return type:

int

seekable()

Return True if the stream supports random access.

If False, seek(), tell() and truncate() will raise OSError.

Returns:

Supports random access.

Return type:

bool

tell()

Return the current stream position.

Returns:

Stream position.

Return type:

int

truncate()

Truncate file to size bytes.

File pointer is left unchanged. Size defaults to the current IO position as reported by tell(). Returns the new size.

writable()

Return True if the stream supports writing.

If False, write() and truncate() will raise OSError.

Returns:

Supports writing.

Return type:

bool

write(b)

Write into the object.

Write the given bytes-like object, b, to the underlying raw stream, and return the number of bytes written.

Parameters:

b (bytes-like object) – Bytes to write.

Returns:

The number of bytes written.

Return type:

int

writelines(lines, /)

Write a list of lines to stream.

Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.

class airfs.storage.azure_file.AzureFileRawIO(*args, **kwargs)[source]

Binary Azure Files Storage Object I/O.

Parameters:

  • name (path-like object) – URL or path to the file which will be opened.

  • mode (str) – The mode can be ‘r’, ‘w’, ‘a’ for reading (default), writing or appending.

  • storage_parameters (dict) – Azure service keyword arguments. This is generally Azure credentials and configuration. See “azure.storage.file.fileservice.FileService” for more information.

  • unsecure (bool) – If True, disables TLS/SSL to improve transfer performance. But makes connection unsecure.

  • content_length (int) – Define the size to preallocate on new file creation. This is not mandatory, and file will be resized on needs, but this allows to improve performance when file size is known in advance.

MAX_FLUSH_SIZE = 4194304

Maximum size of one flush operation

close()

Flush the write buffers of the stream if applicable and close the object.

fileno()

Returns underlying file descriptor if one exists.

OSError is raised if the IO object does not use a file descriptor.

flush()

Flush.

Flush the write buffers of the stream if applicable and save the object on the storage.

isatty()

Return whether this is an ‘interactive’ stream.

Return False if it can’t be determined.

property mode

The mode.

Returns:

Mode.

Return type:

str

property name

The file name.

Returns:

Name.

Return type:

str

readable()

Return True if the stream can be read from.

If False, read() will raise OSError.

Returns:

Supports reading.

Return type:

bool

readall()

Read and return all the bytes from the stream until EOF.

Returns:

Object content

Return type:

bytes

readinto(b)

Read the object content into a buffer.

Read bytes into a pre-allocated, writable bytes-like object b, and return the number of bytes read.

Parameters:

b (bytes-like object) – buffer.

Returns:

number of bytes read

Return type:

int

readline(size=-1, /)

Read and return a line from the stream.

If size is specified, at most size bytes will be read.

The line terminator is always b’n’ for binary files; for text files, the newlines argument to open can be used to select the line terminator(s) recognized.

readlines(hint=-1, /)

Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

seek(offset, whence=0)

Change the stream position to the given byte offset.

Parameters:

  • offset (int) – Offset is interpreted relative to the position indicated by whence.

  • whence (int) – The default value for whence is SEEK_SET. Values are: SEEK_SET or 0 – Start of the stream (the default); offset should be zero or positive SEEK_CUR or 1 – Current stream position; offset may be negative SEEK_END or 2 – End of the stream; offset is usually negative

Returns:

The new absolute position.

Return type:

int

seekable()

Return True if the stream supports random access.

If False, seek(), tell() and truncate() will raise OSError.

Returns:

Supports random access.

Return type:

bool

tell()

Return the current stream position.

Returns:

Stream position.

Return type:

int

truncate()

Truncate file to size bytes.

File pointer is left unchanged. Size defaults to the current IO position as reported by tell(). Returns the new size.

writable()

Return True if the stream supports writing.

If False, write() and truncate() will raise OSError.

Returns:

Supports writing.

Return type:

bool

write(b)

Write into the object.

Write the given bytes-like object, b, to the underlying raw stream, and return the number of bytes written.

Parameters:

b (bytes-like object) – Bytes to write.

Returns:

The number of bytes written.

Return type:

int

writelines(lines, /)

Write a list of lines to stream.

Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.