Memory Descriptors and Objects¶

Descriptors¶

Memory descriptor is an engine-agnostic logical description of data (number of dimensions, dimension sizes, and data type), and, optionally, the information about the physical format of data in memory. If this information is not known yet, a memory descriptor can be created with format tag set to dnnl::memory::format_tag::any. This allows compute-intensive primitives to chose the most appropriate format for the computations. The user is then responsible for reordering their data into the new format if the formats do not match. See Memory Format Propagation.

A memory descriptor can be initialized either by specifying dimensions, and memory format tag or strides for each of them.

User can query amount of memory required by a memory descriptor using the dnnl::memory::desc::get_size() function. The size of data in general cannot be computed as the product of dimensions multiplied by the size of the data type. So users are required to use this function for better code portability.

Two memory descriptors can be compared using the equality and inequality operators. The comparison is especially useful when checking whether it is necessary to reorder data from the user’s data format to a primitive’s format.

Along with ordinary memory descriptors with all dimensions being positive, oneDNN supports zero-volume memory descriptors with one or more dimensions set to zero. This is used to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive typically does not perform any computations with this memory. For example:

The concatenation primitive would ignore all memory object with zeroes in the concatenation dimension / axis.
A forward convolution with a source memory object with zero in the minibatch dimension would always produce a destination memory object with a zero in the minibatch dimension and perform no computations.
However, a forward convolution with a zero in one of the weights dimensions is ill-defined and is considered to be an error by the library because there is no clear definition on what the output values should be.

Data handle of a zero-volume memory is never accessed.

API

struct dnnl::memory::desc¶

A memory descriptor.

Public Functions

desc()¶: Constructs a zero (empty) memory descriptor. Such a memory descriptor can be used to indicate absence of an argument.

desc(const memory::dims &adims, data_type adata_type, format_tag aformat_tag, bool allow_empty = false)¶

Constructs a memory descriptor.

Note

The logical order of dimensions corresponds to the abc... format tag, and the physical meaning of the dimensions depends both on the primitive that would operate on this memory and the operation context.

Parameters

adims: Tensor dimensions.
adata_type: Data precision/type.
aformat_tag: Memory format tag.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc(const dims &adims, data_type adata_type, const dims &strides, bool allow_empty = false)¶

Constructs a memory descriptor by strides.

Note

The logical order of dimensions corresponds to the abc... format tag, and the physical meaning of the dimensions depends both on the primitive that would operate on this memory and the operation context.

Parameters

adims: Tensor dimensions.
adata_type: Data precision/type.
strides: Strides for each dimension.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc submemory_desc(const dims &adims, const dims &offsets, bool allow_empty = false) const¶

Constructs a memory descriptor for a region inside an area described by this memory descriptor.

Return

A memory descriptor for the region.

Parameters

adims: Sizes of the region.
offsets: Offsets to the region from the encompassing memory object in each dimension.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

desc reshape(const dims &adims, bool allow_empty = false) const¶

Constructs a memory descriptor by reshaping an existing one. The new memory descriptor inherits the data type.

The operation ensures that the transformation of the physical memory format corresponds to the transformation of the logical dimensions. If such transformation is impossible, the function either throws an exception (default) or returns a zero memory descriptor depending on the allow_empty flag.

The reshape operation can be described as a combination of the following basic operations:

Add a dimension of size 1. This is always possible.
Remove a dimension of size 1.
Split a dimension into multiple ones. This is possible only if the product of all tensor dimensions stays constant.
Join multiple consecutive dimensions into a single one. This requires that the dimensions are dense in memory and have the same order as their logical counterparts.
- Here, ‘dense’ means: stride for dim[i] == (stride for dim[i + 1]) * dim[i + 1];
- And ‘same order’ means: i < j if and only if stride for dim[i] < stride for dim[j].

Note

Reshape may fail for optimized memory formats.

Return

A new memory descriptor with new dimensions.

Parameters

adims: New dimensions. The product of dimensions must remain constant.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

desc permute_axes(const std::vector<int> &permutation, bool allow_empty = false) const¶

Constructs a memory descriptor by permuting axes in an existing one.

The physical memory layout representation is adjusted accordingly to maintain the consistency between the logical and physical parts of the memory descriptor. The new memory descriptor inherits the data type.

The logical axes will be permuted in the following manner:

for (i = 0; i < ndims(); i++)
    new_desc.dims()[permutation[i]] = dims()[i];

Example:

std::vector<int> permutation = {1, 0}; // swap the first and
                                       // the second axes
dnnl::memory::desc in_md(
        {2, 3}, data_type, memory::format_tag::ab);
dnnl::memory::desc expect_out_md(
        {3, 2}, data_type, memory::format_tag::ba);

assert(in_md.permute_axes(permutation) == expect_out_md);

Return

A new memory descriptor with new dimensions.

Parameters

permutation: Axes permutation.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

memory::dims dims() const¶

Returns dimensions of the memory descriptor.

Potentially expensive due to the data copy involved.

Return: A copy of the dimensions vector.

memory::data_type data_type() const¶

Returns the data type of the memory descriptor.

Return: The data type.

size_t get_size() const¶

Returns size of the memory descriptor in bytes.

Return: The number of bytes required to allocate a memory buffer for the memory object described by this memory descriptor.

bool is_zero() const¶

Checks whether the memory descriptor is zero (empty).

Return: true if the memory descriptor describes an empty memory and false otherwise.

bool operator==(const desc &other) const¶

An equality operator.

Return

Whether this and the other memory descriptors have the same format tag, dimensions, strides, etc.

Parameters

other: Another memory descriptor.

bool operator!=(const desc &other) const¶

An inequality operator.

Return

Whether this and the other memory descriptors describe different memory.

Parameters

other: Another memory descriptor.

Objects¶

Memory objects combine memory descriptors with storage for data (a data handle). With USM, the data handle is simply a pointer to void. The data handle can be queried using dnnl::memory::get_data_handle() and set using dnnl::memory::set_data_handle(). The underlying SYCL buffer, when used, can be queried using dnnl::memory::get_sycl_buffer() and set using dnnl::memory::set_sycl_buffer(). A memory object can also be queried for the underlying memory descriptor and for its engine using dnnl::memory::get_desc() and dnnl::memory::get_engine().

API¶

struct dnnl::memory¶

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific engine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Public Functions

memory()¶

Default constructor.

Constructs an empty memory object, which can be used to indicate absence of a parameter.

memory(const desc &md, const engine &aengine, void *handle)¶

Constructs a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl::memory::set_data_handle() had been called.

See

memory::set_data_handle()

Parameters

md: Memory descriptor.
aengine: Engine to store the data on.
handle: Handle of the memory buffer to use.
- A pointer to the user-allocated buffer. In this case the library doesn’t own the buffer.
- The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer.
- DNNL_MEMORY_NONE to create dnnl_memory without an underlying buffer.

template<typename T, int ndims = 1> memory(const desc &md, const engine &aengine, cl::sycl::buffer<T, ndims> &buf)¶

Constructs a memory object from a SYCL buffer.

Parameters

md: Memory descriptor.
aengine: Engine to store the data on.
buf: A SYCL buffer.

memory(const desc &md, const engine &aengine)¶

Constructs a memory object.

The underlying buffer for the memory will be allocated by the library.

Parameters

md: Memory descriptor.
aengine: Engine to store the data on.

desc get_desc() const¶: Returns the associated memory descriptor.

engine get_engine() const¶: Returns the associated engine.

void *get_data_handle() const¶

Returns the underlying memory buffer.

On the CPU engine, or when using USM, this is a pointer to the allocated memory.

void set_data_handle(void *handle, const stream &astream) const¶

Sets the underlying memory buffer.

This function may write zero values to the memory specified by the handle if the memory object has a zero padding area. This may be time consuming and happens each time this function is called. The operation is always blocking and the stream parameter is a hint.

Note

Even when the memory object is used to hold values that stay constant during the execution of the program (pre-packed weights during inference, for example), the function will still write zeroes to the padding area if it exists. Hence, the handle parameter cannot and does not have a const qualifier.

Parameters

handle: Memory buffer to use. On the CPU engine or when USM is used, the data handle is a pointer to the actual data. It must have at least dnnl::memory::desc::get_size() bytes allocated.
astream: Stream to use to execute padding in.

void set_data_handle(void *handle) const¶

Sets the underlying memory buffer.

See documentation for dnnl::memory::set_data_handle(void *, const stream &) const for more information.

Parameters

handle: Memory buffer to use. For the CPU engine, the data handle is a pointer to the actual data. It must have at least dnnl::memory::desc::get_size() bytes allocated.

template<typename T = void> T *map_data() const¶

Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents.

Mapping enables read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until it is unmapped via dnnl::memory::unmap_data() call.

Note

Any primitives working with the memory should be completed before the memory is mapped. Use dnnl::stream::wait() to synchronize the corresponding execution stream.

Note

The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Return

Pointer to the mapped memory.

Template Parameters

T: Data type to return a pointer to.

void unmap_data(void *mapped_ptr) const¶

Unmaps a memory object and writes back any changes made to the previously mapped memory buffer.

Note

The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Parameters

mapped_ptr: A pointer previously returned by dnnl::memory::map_data().

template<typename T, int ndims = 1> cl::sycl::buffer<T, ndims> get_sycl_buffer(size_t *offset = nullptr) const¶

Returns the underlying SYCL buffer object.

Template Parameters

T: Type of the requested buffer.
ndims: Number of dimensions of the requested buffer.

Parameters

offset: Offset within the returned buffer at which the memory object’s data starts. Only meaningful for 1D buffers.

template<typename T, int ndims> void set_sycl_buffer(cl::sycl::buffer<T, ndims> &buf)¶

Sets the underlying buffer to the given SYCL buffer.

Template Parameters

T: Type of the buffer.
ndims: Number of dimensions of the buffer.

Parameters

buf: SYCL buffer.

DNNL_MEMORY_NONE¶: Special pointer value that indicates that a memory object should not have an underlying buffer.

DNNL_MEMORY_ALLOCATE¶: Special pointer value that indicates that the library needs to allocate an underlying buffer for a memory object.