# Vulkan® 1.4.304 - A Specification (with all registered extensions)

The Khronos® Vulkan Working Group
version 1.4.304, 2025-01-13 20:32:51Z
from git branch: Git branch not available commit: Git commit not available

Loading… please wait.

Search:

## 43. Optical Flow

### 43.1. Optical Flow Queues

`VK_NV_optical_flow` adds an optical flow queue type bit `VK_QUEUE_OPTICAL_FLOW_BIT_NV` to VkQueueFlagBits. Optical flow operations are supported by queues with an advertised queue capability of `VK_QUEUE_OPTICAL_FLOW_BIT_NV`. As in the case of other queue types, an application must use vkGetPhysicalDeviceQueueFamilyProperties to query whether the physical device has support for the Optical Flow Queue. When the implementation reports the `VK_QUEUE_OPTICAL_FLOW_BIT_NV` bit for a queue family, it advertises general support for Vulkan queue operations described in Devices and Queues.

### 43.2. Optical Flow Image Formats

To enumerate the supported image formats for a specific optical flow usage, call:

    // Provided by VK_NV_optical_flow
    VkResult vkGetPhysicalDeviceOpticalFlowImageFormatsNV(
        VkPhysicalDevice                            physicalDevice,
        const VkOpticalFlowImageFormatInfoNV*       pOpticalFlowImageFormatInfo,
        uint32_t*                                   pFormatCount,
        VkOpticalFlowImageFormatPropertiesNV*       pImageFormatProperties);

- `physicalDevice` is the physical device being queried.

- `pOpticalFlowImageFormatInfo` is a pointer to a VkOpticalFlowImageFormatInfoNV structure specifying the optical flow usage for which information is returned.

- `pFormatCount` is a pointer to an integer related to the number of optical flow properties available or queried, as described below.

- `pImageFormatProperties` is a pointer to an array of VkOpticalFlowImageFormatPropertiesNV structures in which supported formats and image parameters are returned.

If `pImageFormatProperties` is `NULL`, then the number of optical flow properties supported for the given `physicalDevice` is returned in `pFormatCount`. Otherwise, `pFormatCount` must point to a variable set by the application to the number of elements in the `pImageFormatProperties` array, and on return the variable is overwritten with the number of values actually written to `pImageFormatProperties`. If the value of `pFormatCount` is less than the number of optical flow properties supported, at most `pFormatCount` values will be written to `pImageFormatProperties`, and `VK_INCOMPLETE` will be returned instead of `VK_SUCCESS`, to indicate that not all the available values were returned.

Before creating an image to be used as an optical flow frame, obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 and vkGetPhysicalDeviceImageFormatProperties2 using one of the reported formats and adding VkOpticalFlowImageFormatInfoNV to the `pNext` chain of VkPhysicalDeviceImageFormatInfo2.

When querying the parameters with vkGetPhysicalDeviceImageFormatProperties2 for images used for optical flow operations, the VkOpticalFlowImageFormatInfoNV::`usage` field must contain one or more of the bits defined in VkOpticalFlowUsageFlagBitsNV.

Valid Usage (Implicit)

- VUID-vkGetPhysicalDeviceOpticalFlowImageFormatsNV-physicalDevice-parameter
  `physicalDevice` must be a valid VkPhysicalDevice handle

- VUID-vkGetPhysicalDeviceOpticalFlowImageFormatsNV-pOpticalFlowImageFormatInfo-parameter
  `pOpticalFlowImageFormatInfo` must be a valid pointer to a valid VkOpticalFlowImageFormatInfoNV structure

- VUID-vkGetPhysicalDeviceOpticalFlowImageFormatsNV-pFormatCount-parameter
  `pFormatCount` must be a valid pointer to a `uint32_t` value

- VUID-vkGetPhysicalDeviceOpticalFlowImageFormatsNV-pImageFormatProperties-parameter
  If the value referenced by `pFormatCount` is not `0`, and `pImageFormatProperties` is not `NULL`, `pImageFormatProperties` must be a valid pointer to an array of `pFormatCount` VkOpticalFlowImageFormatPropertiesNV structures

Return Codes

Success

  - `VK_SUCCESS`

  - `VK_INCOMPLETE`

Failure

  - `VK_ERROR_EXTENSION_NOT_PRESENT`

  - `VK_ERROR_INITIALIZATION_FAILED`

  - `VK_ERROR_FORMAT_NOT_SUPPORTED`

Note | `VK_FORMAT_B8G8R8A8_UNORM`, `VK_FORMAT_R8_UNORM` and `VK_FORMAT_G8_B8R8_2PLANE_420_UNORM` are initially supported for images with optical usage `VK_OPTICAL_FLOW_USAGE_INPUT_BIT_NV`. `VK_FORMAT_R16G16_SFIXED5_NV` is initially supported for images with optical flow usage `VK_OPTICAL_FLOW_USAGE_OUTPUT_BIT_NV`, `VK_OPTICAL_FLOW_USAGE_HINT_BIT_NV` and `VK_OPTICAL_FLOW_USAGE_GLOBAL_FLOW_BIT_NV`. `VK_FORMAT_R8_UINT` and `VK_FORMAT_R32_UINT` are initially supported for images with optical flow usage `VK_OPTICAL_FLOW_USAGE_COST_BIT_NV`. It is recommended to use `VK_FORMAT_R8_UINT` because of the lower bandwidth.

The VkOpticalFlowImageFormatInfoNV structure is defined as:

    // Provided by VK_NV_optical_flow
    typedef struct VkOpticalFlowImageFormatInfoNV {
        VkStructureType              sType;
        const void*                  pNext;
        VkOpticalFlowUsageFlagsNV    usage;
    } VkOpticalFlowImageFormatInfoNV;

- `sType` is a VkStructureType value identifying this structure.

- `pNext` is `NULL` or a pointer to a structure extending this structure.

- `usage` is a bitmask of VkOpticalFlowUsageFlagBitsNV describing the intended optical flow usage of the image.

Valid Usage (Implicit)

- VUID-VkOpticalFlowImageFormatInfoNV-sType-sType
  `sType` must be `VK_STRUCTURE_TYPE_OPTICAL_FLOW_IMAGE_FORMAT_INFO_NV`

- VUID-VkOpticalFlowImageFormatInfoNV-usage-parameter
  `usage` must be a valid combination of VkOpticalFlowUsageFlagBitsNV values

- VUID-VkOpticalFlowImageFormatInfoNV-usage-requiredbitmask
  `usage` must not be `0`

Bits which can be set in VkOpticalFlowImageFormatInfoNV::`usage`, controlling optical flow usage, are:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowUsageFlagBitsNV {
        VK_OPTICAL_FLOW_USAGE_UNKNOWN_NV = 0,
        VK_OPTICAL_FLOW_USAGE_INPUT_BIT_NV = 0x00000001,
        VK_OPTICAL_FLOW_USAGE_OUTPUT_BIT_NV = 0x00000002,
        VK_OPTICAL_FLOW_USAGE_HINT_BIT_NV = 0x00000004,
        VK_OPTICAL_FLOW_USAGE_COST_BIT_NV = 0x00000008,
        VK_OPTICAL_FLOW_USAGE_GLOBAL_FLOW_BIT_NV = 0x00000010,
    } VkOpticalFlowUsageFlagBitsNV;

- `VK_OPTICAL_FLOW_USAGE_INPUT_BIT_NV` specifies that the image can be used as input or reference frame for an optical flow operation.

- `VK_OPTICAL_FLOW_USAGE_OUTPUT_BIT_NV` specifies that the image can be used as output flow vector map for an optical flow operation.

- `VK_OPTICAL_FLOW_USAGE_HINT_BIT_NV` specifies that the image can be used as hint flow vector map for an optical flow operation.

- `VK_OPTICAL_FLOW_USAGE_COST_BIT_NV` specifies that the image can be used as output cost map for an optical flow operation.

- `VK_OPTICAL_FLOW_USAGE_GLOBAL_FLOW_BIT_NV` specifies that the image can be used as global flow vector for an optical flow operation.

    // Provided by VK_NV_optical_flow
    typedef VkFlags   VkOpticalFlowUsageFlagsNV;

`VkOpticalFlowUsageFlagsNV` is a bitmask type for setting a mask of zero or more VkOpticalFlowUsageFlagBitsNV.

The VkOpticalFlowImageFormatPropertiesNV structure is defined as:

    // Provided by VK_NV_optical_flow
    typedef struct VkOpticalFlowImageFormatPropertiesNV {
        VkStructureType    sType;
        const void*        pNext;
        VkFormat           format;
    } VkOpticalFlowImageFormatPropertiesNV;

- `sType` is a VkStructureType value identifying this structure.

- `pNext` is `NULL` or a pointer to a structure extending this structure.

- `format` is a VkFormat that specifies the format that can be used with the specified optical flow image usages.

Valid Usage (Implicit)

- VUID-VkOpticalFlowImageFormatPropertiesNV-sType-sType
  `sType` must be `VK_STRUCTURE_TYPE_OPTICAL_FLOW_IMAGE_FORMAT_PROPERTIES_NV`

- VUID-VkOpticalFlowImageFormatPropertiesNV-pNext-pNext
  `pNext` must be `NULL`

### 43.3. Optical Flow Session

#### 43.3.1. Optical Flow Session Object

Optical flow session objects are abstracted and represented by VkOpticalFlowSessionNV handles:

    // Provided by VK_NV_optical_flow
    VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkOpticalFlowSessionNV)

##### Creating an Optical Flow Session

To create an optical flow session object, call:

    // Provided by VK_NV_optical_flow
    VkResult vkCreateOpticalFlowSessionNV(
        VkDevice                                    device,
        const VkOpticalFlowSessionCreateInfoNV*     pCreateInfo,
        const VkAllocationCallbacks*                pAllocator,
        VkOpticalFlowSessionNV*                     pSession);

- `device` is the logical device that creates the optical flow session object.

- `pCreateInfo` is a pointer to a VkOpticalFlowSessionCreateInfoNV structure containing parameters specifying the creation of the optical flow session.

- `pAllocator` controls host memory allocation as described in the Memory Allocation chapter.

- `pSession` is a pointer to a VkOpticalFlowSessionNV handle specifying the optical flow session object which will be created by this function when it returns `VK_SUCCESS`

Valid Usage (Implicit)

- VUID-vkCreateOpticalFlowSessionNV-device-parameter
  `device` must be a valid VkDevice handle

- VUID-vkCreateOpticalFlowSessionNV-pCreateInfo-parameter
  `pCreateInfo` must be a valid pointer to a valid VkOpticalFlowSessionCreateInfoNV structure

- VUID-vkCreateOpticalFlowSessionNV-pAllocator-parameter
  If `pAllocator` is not `NULL`, `pAllocator` must be a valid pointer to a valid VkAllocationCallbacks structure

- VUID-vkCreateOpticalFlowSessionNV-pSession-parameter
  `pSession` must be a valid pointer to a VkOpticalFlowSessionNV handle

Return Codes

Success

  - `VK_SUCCESS`

Failure

  - `VK_ERROR_OUT_OF_HOST_MEMORY`

  - `VK_ERROR_INITIALIZATION_FAILED`

The VkOpticalFlowSessionCreateInfoNV structure is defined as:

    // Provided by VK_NV_optical_flow
    typedef struct VkOpticalFlowSessionCreateInfoNV {
        VkStructureType                      sType;
        void*                                pNext;
        uint32_t                             width;
        uint32_t                             height;
        VkFormat                             imageFormat;
        VkFormat                             flowVectorFormat;
        VkFormat                             costFormat;
        VkOpticalFlowGridSizeFlagsNV         outputGridSize;
        VkOpticalFlowGridSizeFlagsNV         hintGridSize;
        VkOpticalFlowPerformanceLevelNV      performanceLevel;
        VkOpticalFlowSessionCreateFlagsNV    flags;
    } VkOpticalFlowSessionCreateInfoNV;

- `sType` is a VkStructureType value identifying this structure.

- `pNext` is `NULL` or a pointer to a structure extending this structure.

- `width` is the width in pixels of the input or reference frame to be bound to this optical flow session.

- `height` is the height in pixels of the input or reference frame to be bound to this optical flow session.

- `imageFormat` is the VkFormat of the input and reference frame to be bound to this optical flow session.

- `flowVectorFormat` is the VkFormat of the flow vector maps (output or hint) to be bound to this optical flow session.

- `costFormat` is the VkFormat of the cost maps to be bound to this optical flow session.

- `outputGridSize` is exactly one bit of VkOpticalFlowGridSizeFlagsNV specifying the grid size of the output flow and cost maps to be bound to this optical flow session. The size of the output flow and cost maps is determined by `VkOpticalFlowSessionCreateInfoNV`::`width` and `VkOpticalFlowSessionCreateInfoNV`::`height` divided by `VkOpticalFlowSessionCreateInfoNV`::`outputGridSize`.

- `hintGridSize` is one exactly bit of VkOpticalFlowGridSizeFlagsNV specifying the grid size of the hint flow vector maps to be bound to this optical flow session. The size of the hint maps is determined by `VkOpticalFlowSessionCreateInfoNV`::`width` and `VkOpticalFlowSessionCreateInfoNV`::`height` divided by `VkOpticalFlowSessionCreateInfoNV`::`hintGridSize`.

- `performanceLevel` is the VkOpticalFlowPerformanceLevelNV used for this optical flow session.

- `flags` are the VkOpticalFlowSessionCreateFlagsNV used for this optical flow session.

Valid Usage

- VUID-VkOpticalFlowSessionCreateInfoNV-width-07581
  `width` must be greater than or equal to `VkPhysicalDeviceOpticalFlowPropertiesNV`::`minWidth` and less than or equal to `VkPhysicalDeviceOpticalFlowPropertiesNV`::`maxWidth`

- VUID-VkOpticalFlowSessionCreateInfoNV-height-07582
  `height` must be greater than or equal to `VkPhysicalDeviceOpticalFlowPropertiesNV`::`minHeight` and less than or equal to `VkPhysicalDeviceOpticalFlowPropertiesNV`::`maxHeight`

- VUID-VkOpticalFlowSessionCreateInfoNV-imageFormat-07583
  `imageFormat` must be one of the formats returned by vkGetPhysicalDeviceOpticalFlowImageFormatsNV for `VK_OPTICAL_FLOW_USAGE_INPUT_BIT_NV`

- VUID-VkOpticalFlowSessionCreateInfoNV-flowVectorFormat-07584
  `flowVectorFormat` must be one of the formats returned by vkGetPhysicalDeviceOpticalFlowImageFormatsNV for `VK_OPTICAL_FLOW_USAGE_OUTPUT_BIT_NV`

- VUID-VkOpticalFlowSessionCreateInfoNV-costFormat-07585
  `costFormat` must be one of the formats returned by vkGetPhysicalDeviceOpticalFlowImageFormatsNV for `VK_OPTICAL_FLOW_USAGE_COST_BIT_NV` if `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_COST_BIT_NV` is set in `flags`

- VUID-VkOpticalFlowSessionCreateInfoNV-outputGridSize-07586
  `outputGridSize` must be exactly one of the bits reported in `VkPhysicalDeviceOpticalFlowPropertiesNV`::`supportedOutputGridSizes`

- VUID-VkOpticalFlowSessionCreateInfoNV-hintGridSize-07587
  `hintGridSize` must be exactly one of the bits reported in `VkPhysicalDeviceOpticalFlowPropertiesNV`::`supportedHintGridSizes` if `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_HINT_BIT_NV` is set in `flags`

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-07588
  `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_HINT_BIT_NV` must not be set in `flags` if `VkPhysicalDeviceOpticalFlowPropertiesNV`::`hintSupported` is `VK_FALSE`

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-07589
  `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_COST_BIT_NV` must not be set in `flags` if `VkPhysicalDeviceOpticalFlowPropertiesNV`::`costSupported` is `VK_FALSE`

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-07590
  `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_GLOBAL_FLOW_BIT_NV` must not be set in `flags` if `VkPhysicalDeviceOpticalFlowPropertiesNV`::`globalFlowSupported` is `VK_FALSE`

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-07591
  `VK_OPTICAL_FLOW_SESSION_CREATE_ALLOW_REGIONS_BIT_NV` must not be set in `flags` if `VkPhysicalDeviceOpticalFlowPropertiesNV`::`maxNumRegionsOfInterest` is 0

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-07592
  `VK_OPTICAL_FLOW_SESSION_CREATE_BOTH_DIRECTIONS_BIT_NV` must not be set in `flags` if `VkPhysicalDeviceOpticalFlowPropertiesNV`::`bidirectionalFlowSupported` is `VK_FALSE`

Valid Usage (Implicit)

- VUID-VkOpticalFlowSessionCreateInfoNV-sType-sType
  `sType` must be `VK_STRUCTURE_TYPE_OPTICAL_FLOW_SESSION_CREATE_INFO_NV`

- VUID-VkOpticalFlowSessionCreateInfoNV-pNext-pNext
  `pNext` must be `NULL` or a pointer to a valid instance of VkOpticalFlowSessionCreatePrivateDataInfoNV

- VUID-VkOpticalFlowSessionCreateInfoNV-sType-unique
  The `sType` value of each struct in the `pNext` chain must be unique

- VUID-VkOpticalFlowSessionCreateInfoNV-imageFormat-parameter
  `imageFormat` must be a valid VkFormat value

- VUID-VkOpticalFlowSessionCreateInfoNV-flowVectorFormat-parameter
  `flowVectorFormat` must be a valid VkFormat value

- VUID-VkOpticalFlowSessionCreateInfoNV-costFormat-parameter
  If `costFormat` is not `0`, `costFormat` must be a valid VkFormat value

- VUID-VkOpticalFlowSessionCreateInfoNV-outputGridSize-parameter
  `outputGridSize` must be a valid combination of VkOpticalFlowGridSizeFlagBitsNV values

- VUID-VkOpticalFlowSessionCreateInfoNV-outputGridSize-requiredbitmask
  `outputGridSize` must not be `0`

- VUID-VkOpticalFlowSessionCreateInfoNV-hintGridSize-parameter
  `hintGridSize` must be a valid combination of VkOpticalFlowGridSizeFlagBitsNV values

- VUID-VkOpticalFlowSessionCreateInfoNV-performanceLevel-parameter
  If `performanceLevel` is not `0`, `performanceLevel` must be a valid VkOpticalFlowPerformanceLevelNV value

- VUID-VkOpticalFlowSessionCreateInfoNV-flags-parameter
  `flags` must be a valid combination of VkOpticalFlowSessionCreateFlagBitsNV values

The VkOpticalFlowSessionCreatePrivateDataInfoNV structure is for NV internal use only and is defined as:

    // Provided by VK_NV_optical_flow
    typedef struct VkOpticalFlowSessionCreatePrivateDataInfoNV {
        VkStructureType    sType;
        void*              pNext;
        uint32_t           id;
        uint32_t           size;
        const void*        pPrivateData;
    } VkOpticalFlowSessionCreatePrivateDataInfoNV;

- `sType` is a VkStructureType value identifying this structure.

- `pNext` is `NULL` or a pointer to a structure extending this structure.

- `id` is an identifier for data which is passed at a memory location specified in `VkOpticalFlowSessionCreatePrivateDataInfoNV`::`pPrivateData`.

- `size` is the size of data in bytes which is passed at a memory location specified in `VkOpticalFlowSessionCreatePrivateDataInfoNV`::`pPrivateData`.

- `pPrivateData` is a pointer to NV internal data.

Valid Usage (Implicit)

- VUID-VkOpticalFlowSessionCreatePrivateDataInfoNV-sType-sType
  `sType` must be `VK_STRUCTURE_TYPE_OPTICAL_FLOW_SESSION_CREATE_PRIVATE_DATA_INFO_NV`

- VUID-VkOpticalFlowSessionCreatePrivateDataInfoNV-pPrivateData-parameter
  `pPrivateData` must be a pointer value

Optical flow vectors are generated block-wise, one vector for each block of NxN pixels (referred to as grid).

Bits which can be set in VkOpticalFlowSessionCreateInfoNV::`outputGridSize` and VkOpticalFlowSessionCreateInfoNV::`hintGridSize`, or which are returned in VkPhysicalDeviceOpticalFlowPropertiesNV::`supportedOutputGridSizes` and VkPhysicalDeviceOpticalFlowPropertiesNV::`supportedHintGridSizes` controlling optical flow grid sizes, are:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowGridSizeFlagBitsNV {
        VK_OPTICAL_FLOW_GRID_SIZE_UNKNOWN_NV = 0,
        VK_OPTICAL_FLOW_GRID_SIZE_1X1_BIT_NV = 0x00000001,
        VK_OPTICAL_FLOW_GRID_SIZE_2X2_BIT_NV = 0x00000002,
        VK_OPTICAL_FLOW_GRID_SIZE_4X4_BIT_NV = 0x00000004,
        VK_OPTICAL_FLOW_GRID_SIZE_8X8_BIT_NV = 0x00000008,
    } VkOpticalFlowGridSizeFlagBitsNV;

- `VK_OPTICAL_FLOW_GRID_SIZE_1X1_BIT_NV` specifies that grid is 1x1 pixel.

- `VK_OPTICAL_FLOW_GRID_SIZE_2X2_BIT_NV` specifies that grid is 2x2 pixel.

- `VK_OPTICAL_FLOW_GRID_SIZE_4X4_BIT_NV` specifies that grid is 4x4 pixel.

- `VK_OPTICAL_FLOW_GRID_SIZE_8X8_BIT_NV` specifies that grid is 8x8 pixel.

    // Provided by VK_NV_optical_flow
    typedef VkFlags   VkOpticalFlowGridSizeFlagsNV;

`VkOpticalFlowGridSizeFlagsNV` is a bitmask type for setting a mask of zero or more VkOpticalFlowGridSizeFlagBitsNV.

Optical flow exposes performance levels which the application can choose based on the desired performance and quality requirement.

The optical flow performance level types are defined with the following:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowPerformanceLevelNV {
        VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_UNKNOWN_NV = 0,
        VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_SLOW_NV = 1,
        VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_MEDIUM_NV = 2,
        VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_FAST_NV = 3,
    } VkOpticalFlowPerformanceLevelNV;

- `VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_SLOW_NV` is a level with slower performance but higher quality.

- `VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_MEDIUM_NV` is a level with medium performance and medium quality.

- `VK_OPTICAL_FLOW_PERFORMANCE_LEVEL_FAST_NV` is a preset with higher performance but lower quality.

Bits which can be set in VkOpticalFlowSessionCreateInfoNV::`flags`, controlling optical flow session operations, are:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowSessionCreateFlagBitsNV {
        VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_HINT_BIT_NV = 0x00000001,
        VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_COST_BIT_NV = 0x00000002,
        VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_GLOBAL_FLOW_BIT_NV = 0x00000004,
        VK_OPTICAL_FLOW_SESSION_CREATE_ALLOW_REGIONS_BIT_NV = 0x00000008,
        VK_OPTICAL_FLOW_SESSION_CREATE_BOTH_DIRECTIONS_BIT_NV = 0x00000010,
    } VkOpticalFlowSessionCreateFlagBitsNV;

- `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_HINT_BIT_NV` specifies that a VkImageView with external flow vectors will be used as hints in performing the motion search and must be bound to `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_HINT_NV`.

- `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_COST_BIT_NV` specifies that the cost for the forward flow is generated in a VkImageView which must be bound to `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_COST_NV`. Additionally, if `VK_OPTICAL_FLOW_SESSION_CREATE_BOTH_DIRECTIONS_BIT_NV` is also set, the cost for backward flow is generated in a VkImageView which must be bound to `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_BACKWARD_COST_NV`. The cost is the confidence level of the flow vector for each grid in the frame. The Cost implies how (in)accurate the flow vector is. Higher cost value implies the flow vector to be less accurate and vice-versa.

- `VK_OPTICAL_FLOW_SESSION_CREATE_ENABLE_GLOBAL_FLOW_BIT_NV` specifies that a global flow vector is estimated from forward flow in a single pixel VkImageView which must be bound to `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_GLOBAL_FLOW_NV`.

- `VK_OPTICAL_FLOW_SESSION_CREATE_ALLOW_REGIONS_BIT_NV` specifies that regions of interest can be specified in VkOpticalFlowExecuteInfoNV.

- `VK_OPTICAL_FLOW_SESSION_CREATE_BOTH_DIRECTIONS_BIT_NV` specifies that backward flow is generated in addition to forward flow which is always generated.

`VkOpticalFlowSessionCreateFlagsNV` is a bitmask type for setting a mask of zero or more VkOpticalFlowSessionCreateFlagBitsNV.

    // Provided by VK_NV_optical_flow
    typedef VkFlags   VkOpticalFlowSessionCreateFlagsNV;

##### Destroying an Optical Flow Session

To destroy an optical flow session object, call:

    // Provided by VK_NV_optical_flow
    void vkDestroyOpticalFlowSessionNV(
        VkDevice                                    device,
        VkOpticalFlowSessionNV                      session,
        const VkAllocationCallbacks*                pAllocator);

- `device` is the device that was used for the creation of the optical flow session.

- `session` is the optical flow session to be destroyed.

- `pAllocator` controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage (Implicit)

- VUID-vkDestroyOpticalFlowSessionNV-device-parameter
  `device` must be a valid VkDevice handle

- VUID-vkDestroyOpticalFlowSessionNV-session-parameter
  `session` must be a valid VkOpticalFlowSessionNV handle

- VUID-vkDestroyOpticalFlowSessionNV-pAllocator-parameter
  If `pAllocator` is not `NULL`, `pAllocator` must be a valid pointer to a valid VkAllocationCallbacks structure

- VUID-vkDestroyOpticalFlowSessionNV-session-parent
  `session` must have been created, allocated, or retrieved from `device`

#### 43.3.2. Binding Vulkan Image Views to an Optical Flow Session

To bind a vulkan image to an optical flow session object, call:

    // Provided by VK_NV_optical_flow
    VkResult vkBindOpticalFlowSessionImageNV(
        VkDevice                                    device,
        VkOpticalFlowSessionNV                      session,
        VkOpticalFlowSessionBindingPointNV          bindingPoint,
        VkImageView                                 view,
        VkImageLayout                               layout);

- `device` is the device which owns the optical flow session object `session`.

- `session` is the optical flow session object to which the image view is to be bound.

- `bindingPoint` specifies the binding point VkOpticalFlowSessionBindingPointNV to which the image view is bound.

- `view` is a VkImageView to be bound.

- layout must specify the layout that the image subresources accessible from `view` will be in at the time the optical flow vectors are calculated with vkCmdOpticalFlowExecuteNV on a `VkDevice`.

Valid Usage (Implicit)

- VUID-vkBindOpticalFlowSessionImageNV-device-parameter
  `device` must be a valid VkDevice handle

- VUID-vkBindOpticalFlowSessionImageNV-session-parameter
  `session` must be a valid VkOpticalFlowSessionNV handle

- VUID-vkBindOpticalFlowSessionImageNV-bindingPoint-parameter
  `bindingPoint` must be a valid VkOpticalFlowSessionBindingPointNV value

- VUID-vkBindOpticalFlowSessionImageNV-view-parameter
  If `view` is not VK_NULL_HANDLE, `view` must be a valid VkImageView handle

- VUID-vkBindOpticalFlowSessionImageNV-layout-parameter
  `layout` must be a valid VkImageLayout value

- VUID-vkBindOpticalFlowSessionImageNV-session-parent
  `session` must have been created, allocated, or retrieved from `device`

- VUID-vkBindOpticalFlowSessionImageNV-view-parent
  If `view` is a valid handle, it must have been created, allocated, or retrieved from `device`

Return Codes

Success

  - `VK_SUCCESS`

Failure

  - `VK_ERROR_OUT_OF_HOST_MEMORY`

  - `VK_ERROR_INITIALIZATION_FAILED`

The optical flow session binding points are defined with the following:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowSessionBindingPointNV {
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_UNKNOWN_NV = 0,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_INPUT_NV = 1,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_REFERENCE_NV = 2,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_HINT_NV = 3,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_FLOW_VECTOR_NV = 4,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_BACKWARD_FLOW_VECTOR_NV = 5,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_COST_NV = 6,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_BACKWARD_COST_NV = 7,
        VK_OPTICAL_FLOW_SESSION_BINDING_POINT_GLOBAL_FLOW_NV = 8,
    } VkOpticalFlowSessionBindingPointNV;

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_INPUT_NV` specifies the binding point for the input frame.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_REFERENCE_NV` specifies the binding point for the input reference frame.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_HINT_NV` specifies the binding point for the optional external hint flow vectors.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_FLOW_VECTOR_NV` specifies the binding point for output flow vectors of default forward flow calculation.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_BACKWARD_FLOW_VECTOR_NV` specifies the binding point for the optional output flow vector map of optional backward flow calculation.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_COST_NV` specifies the binding point for the optional output cost map of default forward flow calculation.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_BACKWARD_COST_NV` specifies the binding point for the optional output cost map of optional backward flow calculation.

- `VK_OPTICAL_FLOW_SESSION_BINDING_POINT_GLOBAL_FLOW_NV` specifies the binding point for the optional global flow value of default forward flow calculation.

#### 43.3.3. Optical Flow Execution

Default direction of flow estimation is forward which calculates the optical flow from input frame to reference frame. Optionally backward flow estimation can be additionally calculated. An output flow vector (Vx, Vy) means that current pixel (x, y) of input frame can be found at location (x+Vx, y+Vy) in reference frame. A backward flow vector (Vx, Vy) means that current pixel (x, y) of reference frame can be found at location (x+Vx, y+Vy) in input frame.

To calculate optical flow vectors from two input frames, call:

    // Provided by VK_NV_optical_flow
    void vkCmdOpticalFlowExecuteNV(
        VkCommandBuffer                             commandBuffer,
        VkOpticalFlowSessionNV                      session,
        const VkOpticalFlowExecuteInfoNV*           pExecuteInfo);

- `commandBuffer` is the command buffer into which the command will be recorded.

- `session` is the optical flow session object on which this command is operating.

- `pExecuteInfo` Info is a pointer to a VkOpticalFlowExecuteInfoNV.

Valid Usage (Implicit)

- VUID-vkCmdOpticalFlowExecuteNV-commandBuffer-parameter
  `commandBuffer` must be a valid VkCommandBuffer handle

- VUID-vkCmdOpticalFlowExecuteNV-session-parameter
  `session` must be a valid VkOpticalFlowSessionNV handle

- VUID-vkCmdOpticalFlowExecuteNV-pExecuteInfo-parameter
  `pExecuteInfo` must be a valid pointer to a valid VkOpticalFlowExecuteInfoNV structure

- VUID-vkCmdOpticalFlowExecuteNV-commandBuffer-recording
  `commandBuffer` must be in the recording state

- VUID-vkCmdOpticalFlowExecuteNV-commandBuffer-cmdpool
  The `VkCommandPool` that `commandBuffer` was allocated from must support optical flow operations

- VUID-vkCmdOpticalFlowExecuteNV-renderpass
  This command must only be called outside of a render pass instance

- VUID-vkCmdOpticalFlowExecuteNV-videocoding
  This command must only be called outside of a video coding scope

- VUID-vkCmdOpticalFlowExecuteNV-commonparent
  Both of `commandBuffer`, and `session` must have been created, allocated, or retrieved from the same VkDevice

Host Synchronization

- Host access to the `VkCommandPool` that `commandBuffer` was allocated from must be externally synchronized

Command Properties

Command Buffer Levels | Render Pass Scope | Video Coding Scope | Supported Queue Types | Command Type
--- | --- | --- | --- | ---
Primary Secondary | Outside | Outside | Opticalflow | Action

The VkOpticalFlowExecuteInfoNV structure is defined as:

    // Provided by VK_NV_optical_flow
    typedef struct VkOpticalFlowExecuteInfoNV {
        VkStructureType                sType;
        void*                          pNext;
        VkOpticalFlowExecuteFlagsNV    flags;
        uint32_t                       regionCount;
        const VkRect2D*                pRegions;
    } VkOpticalFlowExecuteInfoNV;

- `sType` is a VkStructureType value identifying this structure.

- `pNext` is `NULL` or a pointer to a structure extending this structure.

- `flags` are the VkOpticalFlowExecuteFlagsNV used for this command.

- `regionCount` is the number of regions of interest specified in `pRegions`.

- `pRegions` is a pointer to `regionCount` `VkRect2D` regions of interest.

Valid Usage

- VUID-VkOpticalFlowExecuteInfoNV-regionCount-07593
  `regionCount` must be 0 if `VK_OPTICAL_FLOW_SESSION_CREATE_ALLOW_REGIONS_BIT_NV` was not set for `VkOpticalFlowSessionNV` on which this command is operating

Valid Usage (Implicit)

- VUID-VkOpticalFlowExecuteInfoNV-sType-sType
  `sType` must be `VK_STRUCTURE_TYPE_OPTICAL_FLOW_EXECUTE_INFO_NV`

- VUID-VkOpticalFlowExecuteInfoNV-pNext-pNext
  `pNext` must be `NULL`

- VUID-VkOpticalFlowExecuteInfoNV-flags-parameter
  `flags` must be a valid combination of VkOpticalFlowExecuteFlagBitsNV values

- VUID-VkOpticalFlowExecuteInfoNV-pRegions-parameter
  If `regionCount` is not `0`, `pRegions` must be a valid pointer to an array of `regionCount` VkRect2D structures

Bits which can be set in VkOpticalFlowExecuteInfoNV::`flags`, controlling optical flow execution, are:

    // Provided by VK_NV_optical_flow
    typedef enum VkOpticalFlowExecuteFlagBitsNV {
        VK_OPTICAL_FLOW_EXECUTE_DISABLE_TEMPORAL_HINTS_BIT_NV = 0x00000001,
    } VkOpticalFlowExecuteFlagBitsNV;

- `VK_OPTICAL_FLOW_EXECUTE_DISABLE_TEMPORAL_HINTS_BIT_NV` specifies that temporal hints from previously generated flow vectors are not used. If temporal hints are enabled, optical flow vectors from previous vkCmdOpticalFlowExecuteNV call are automatically used as hints for the current vkCmdOpticalFlowExecuteNV call, to take advantage of temporal correlation in a video sequence. Temporal hints should be disabled if there is a-priori knowledge of no temporal correlation (e.g. a scene change, independent successive frame pairs).

`VkOpticalFlowExecuteFlagsNV` is a bitmask type for setting a mask of zero or more VkOpticalFlowExecuteFlagBitsNV.

    // Provided by VK_NV_optical_flow
    typedef VkFlags   VkOpticalFlowExecuteFlagsNV;