Architecture of the Vulkan Loader Interfaces

Table of Contents

• Overview
  • Who Should Read This Document
  • The Loader
    • Goals of the Loader
  • Layers
  • Drivers
    • Installable Client Drivers
  • VkConfig
• Important Vulkan Concepts
  • Instance Versus Device
    • Instance-Specific
      • Instance Objects
      • Instance Functions
      • Instance Extensions
    • Device-Specific
      • Device Objects
      • Device Functions
      • Device Extensions
  • Dispatch Tables and Call Chains
    • Instance Call Chain Example
    • Device Call Chain Example
• Elevated Privilege Caveats
• Application Interface to the Loader
• Layer Interface with the Loader
• Driver Interface With the Loader
• Debugging Issues
• Loader Policies
• Filter Environment Variable Behaviors
  • Comparison Strings
  • Comma-Delimited Lists
  • Globs
  • Case-Insensitive
  • Environment Variable Priority
• Table of Debug Environment Variables
  • Active Environment Variables
  • Deprecated Environment Variables
• Glossary of Terms

Overview

Vulkan is a layered architecture, made up of the following elements:

• The Vulkan Application
• The Vulkan Loader
• Vulkan Layers
• Drivers
• VkConfig

The general concepts in this document are applicable to the loaders available for Windows, Linux, Android, and macOS systems.

Who Should Read This Document

While this document is primarily targeted at developers of Vulkan applications, drivers and layers, the information contained in it could be useful to anyone wanting a better understanding of the Vulkan runtime.

The Loader

The application sits at the top and interfaces directly with the Vulkan loader. At the bottom of the stack sits the drivers. A driver can control one or more physical devices capable of rendering Vulkan, implement a conversion from Vulkan into a native graphics API (like MoltenVk, or implement a fully software path that can be executed on a CPU to simulate a Vulkan device (like SwiftShader or LavaPipe). Remember, Vulkan-capable hardware may be graphics-based, compute-based, or both. Between the application and the drivers, the loader can inject any number of optional layers that provide special functionality. The loader is critical to managing the proper dispatching of Vulkan functions to the appropriate set of layers and drivers. The Vulkan object model allows the loader to insert layers into a call-chain so that the layers can process Vulkan functions prior to the driver being called.

This document is intended to provide an overview of the necessary interfaces between each of these.

Goals of the Loader

The loader was designed with the following goals in mind:

1. Support one or more Vulkan-capable drivers on a user's system without them interfering with one another.
2. Support Vulkan Layers which are optional modules that can be enabled by an application, developer, or standard system settings.
3. Keep the overall overhead of the loader to the minimum possible.

Layers

Layers are optional components that augment the Vulkan development environment. They can intercept, evaluate, and modify existing Vulkan functions on their way from the application down to the drivers and back up. Layers are implemented as libraries that can be enabled in different ways and are loaded during CreateInstance. Each layer can choose to hook, or intercept, Vulkan functions which in turn can be ignored, inspected, or augmented. Any function a layer does not hook is simply skipped for that layer and the control flow will simply continue on to the next supporting layer or driver. Because of this, a layer can choose whether to intercept all known Vulkan functions or only a subset it is interested in.

Some examples of features that layers may expose include:

• Validating API usage
• Tracing API calls
• Debugging aids
• Profiling
• Overlay

Because layers are optional and dynamically loaded, they can be enabled and disabled as desired. For example, while developing and debugging an application, enabling certain layers can assist in making sure it properly uses the Vulkan API. But when releasing the application, those layers are unnecessary and thus won't be enabled, increasing the speed of the application.

Drivers

The library that implements Vulkan, either through supporting a physical hardware device directly, converting Vulkan commands into native graphics commands, or simulating Vulkan through software, is considered "a driver". The most common type of driver is still the Installable Client Driver (or ICD). The loader is responsible for discovering available Vulkan drivers on the system. Given a list of available drivers, the loader can enumerate all the available physical devices and provide this information for an application.

Installable Client Drivers

Vulkan allows multiple ICDs each supporting one or more devices. Each of these devices is represented by a Vulkan VkPhysicalDevice object. The loader is responsible for discovering available Vulkan ICDs via the standard driver search on the system.

VkConfig

VkConfig is a tool LunarG has developed to assist with modifying the Vulkan environment on the local system. It can be used to find layers, enable them, change layer settings, and other useful features. VkConfig can be found by either installing the Vulkan SDK or by building the source out of the LunarG VulkanTools GitHub Repo.

VkConfig generates three outputs, two of which work with the Vulkan loader and layers. These outputs are:

• The Vulkan Override Layer
• The Vulkan Layer Settings File
• VkConfig Configuration Settings

These files are found in different locations based on your platform:

The Override Meta-Layer is an important part of how VkConfig works. This layer, when found by the loader, forces the loading of the desired layers that were enabled inside of VkConfig as well as disables those layers that were intentionally disabled (including implicit layers).

The Vulkan Layer Settings file can be used to specify certain behaviors and actions each enabled layer is expected to perform. These settings can also be controlled by VkConfig, or they can be manually enabled. For details on what settings can be used, refer to the individual layers.

In the future, VkConfig may have additional interactions with the Vulkan loader.

More details on VkConfig can be found in its GitHub documentation.

Important Vulkan Concepts

Vulkan has a few concepts that provide a fundamental basis for its organization. These concepts should be understood by any one attempting to use Vulkan or develop any of its components.

Instance Versus Device

An important concept to understand, which is brought up repeatedly throughout this document, is how the Vulkan API is organized. Many objects, functions, extensions, and other behavior in Vulkan can be separated into two groups:

• Instance-specific
• Device-specific

Instance-Specific

A "Vulkan instance" (VkInstance) is a high-level construct used to provide Vulkan system-level information and functionality.

Instance Objects

A few Vulkan objects associated directly with an instance are:

• VkInstance
• VkPhysicalDevice
• VkPhysicalDeviceGroup

Instance Functions

An "instance function" is any Vulkan function where the first parameter is an instance object or no object at all.

Some Vulkan instance functions are:

• vkEnumerateInstanceExtensionProperties
• vkEnumeratePhysicalDevices
• vkCreateInstance
• vkDestroyInstance

An application can link directly to all core instance functions through the Vulkan loader's headers. Alternatively, an application can query function pointers using vkGetInstanceProcAddr. vkGetInstanceProcAddr can be used to query any instance or device entry-points in addition to all core entry-points.

If vkGetInstanceProcAddr is called using a VkInstance, then any function pointer returned is specific to that VkInstance and any additional objects that are created from it.

Instance Extensions

Extensions to Vulkan are similarly associated based on what type of functions they provide. Because of this, extensions are broken up into instance or device extensions where most, if not all of the functions, in the extension are of the corresponding type. For example, an "instance extension" is composed primarily of "instance functions" which primarily take instance objects. These will be discussed in more detail later.

Device-Specific

A Vulkan device (VkDevice), on the other-hand, is a logical identifier used to associate functions with a particular Vulkan physical device (VkPhysicalDevice) through a particular driver on a user's system.

Device Objects

A few of the Vulkan constructs associated directly with a device include:

• VkDevice
• VkQueue
• VkCommandBuffer

Device Functions

A "device function" is any Vulkan function which takes any device object as its first parameter or a child object of the device. The vast majority of Vulkan functions are device functions. Some Vulkan device functions are:

• vkQueueSubmit
• vkBeginCommandBuffer
• vkCreateEvent

Vulkan devices functions may be queried using either vkGetInstanceProcAddr or vkGetDeviceProcAddr. If an application chooses to use vkGetInstanceProcAddr, each call will have additional function calls built into the call chain, which will reduce performance slightly. If, instead, the application uses vkGetDeviceProcAddr, the call chain will be more optimized to the specific device, but the returned function pointers will only work for the device used when querying them. Unlike vkGetInstanceProcAddr, vkGetDeviceProcAddr can only be used on Vulkan device functions.

The best solution is to query instance extension functions using vkGetInstanceProcAddr, and to query device extension functions using vkGetDeviceProcAddr. See Best Application Performance Setup section in the LoaderApplicationInterface.html document for more information on this.

Device Extensions

As with instance extensions, a device extension is a set of Vulkan device functions extending the Vulkan language. More information about device extensions can be found later in this document.

Dispatch Tables and Call Chains

Vulkan uses an object model to control the scope of a particular action or operation. The object to be acted on is always the first parameter of a Vulkan call and is a dispatchable object (see Vulkan specification section 3.3 Object Model). Under the covers, the dispatchable object handle is a pointer to a structure, which in turn, contains a pointer to a dispatch table maintained by the loader. This dispatch table contains pointers to the Vulkan functions appropriate to that object.

There are two types of dispatch tables the loader maintains:

• Instance Dispatch Table
  • Created in the loader during the call to vkCreateInstance
• Device Dispatch Table
  • Created in the loader during the call to vkCreateDevice

At that time the application and the system can each specify optional layers to be included. The loader will initialize the specified layers to create a call chain for each Vulkan function and each entry of the dispatch table will point to the first element of that chain. Thus, the loader builds an instance call chain for each VkInstance that is created and a device call chain for each VkDevice that is created.

When an application calls a Vulkan function, this typically will first hit a trampoline function in the loader. These trampoline functions are small, simple functions that jump to the appropriate dispatch table entry for the object they are given. Additionally, for functions in the instance call chain, the loader has an additional function, called a terminator, which is called after all enabled layers to marshall the appropriate information to all available drivers.

Instance Call Chain Example

For example, the diagram below represents what happens in the call chain for vkCreateInstance. After initializing the chain, the loader calls into the first layer's vkCreateInstance, which will call the next layer's vkCreateInstance before finally terminating in the loader again where it will call every driver's vkCreateInstance. This allows every enabled layer in the chain to set up what it needs based on the VkInstanceCreateInfo structure from the application.

This also highlights some of the complexity the loader must manage when using instance call chains. As shown here, the loader's terminator must aggregate information to and from multiple drivers when they are present. This implies that the loader has to be aware of any instance-level extensions which work on a VkInstance to aggregate them correctly.

Device Call Chain Example

Device call chains are created in vkCreateDevice and are generally simpler because they deal with only a single device. This allows for the specific driver exposing this device to always be the terminator of the chain.

Elevated Privilege Caveats

To ensure that the system is safe from exploitation, Vulkan applications which are run with elevated privileges are restricted from certain operations, such as reading environment variables from unsecure locations or searching for files in user controlled paths. This is done to ensure that an application running with elevated privileges does not run using components that were not installed in the proper approved locations.

The loader uses platform-specific mechanisms (such as secure_getenv and its equivalents) for querying sensitive environment variables to avoid accidentally using untrusted results.

These behaviors also result in ignoring certain environment variables, such as:

• VK_DRIVER_FILES / VK_ICD_FILENAMES
• VK_ADD_DRIVER_FILES
• VK_LAYER_PATH
• VK_ADD_LAYER_PATH
• VK_IMPLICIT_LAYER_PATH
• VK_ADD_IMPLICIT_LAYER_PATH
• XDG_CONFIG_HOME (Linux/Mac-specific)
• XDG_DATA_HOME (Linux/Mac-specific)

For more information on the affected search paths, refer to Layer Discovery and Driver Discovery.

Application Interface to the Loader

The Application interface to the Vulkan loader is now detailed in the LoaderApplicationInterface.html document found in the same directory as this file.

Layer Interface with the Loader

The Layer interface to the Vulkan loader is detailed in the LoaderLayerInterface.html document found in the same directory as this file.

Driver Interface With the Loader

The Driver interface to the Vulkan loader is detailed in the LoaderDriverInterface.html document found in the same directory as this file.

Debugging Issues

If your application is crashing or behaving weirdly, the loader provides several mechanisms for you to debug the issues. These are detailed in the LoaderDebugging.html document found in the same directory as this file.

Loader Policies

Loader policies with regards to the loader interaction with drivers and layers are now documented in the appropriate sections. The intention of these sections is to clearly define expected behavior of the loader with regards to its interactions with those components. This could be especially useful in cases where a new or specialized loader may be required that conforms to the behavior of the existing loader. Because of this, the primary focus of those sections is on expected behaviors for all relevant components to create a consistent experience across platforms. In the long-run, this could also be used as validation requirements for any existing Vulkan loaders.

To review the particular policy sections, please refer to one or both of the sections listed below:

• Loader And Driver Policy
• Loader And Layer Policy

Filter Environment Variable Behaviors

The filter environment variables provided in certain areas have some common restrictions and behaviors that should be listed.

Comparison Strings

The filter variables will be compared against the appropriate strings for either drivers or layers. The appropriate string for layers is the layer name provided in the layer's manifest file. Since drivers don’t have a name like layers, this substring is used to compare against the driver manifest's filename.

Comma-Delimited Lists

All of the filter environment variables accept comma-delimited input. Therefore, you can chain multiple strings together and it will use the strings to individually enable or disable the appropriate item in the current list of available items.

Globs

To provide enough flexibility to limit name searches to only those wanted by the developer, the loader uses a limited glob format for strings. Acceptable globs are:

• Prefixes: "string*"
• Suffixes: "*string"
• Substrings: "*string*"
• Whole strings: "string"
  • In the case of whole strings, the string will be compared against each layer or driver file name in its entirety.
  • Because of this, it will only match the specific target such as: VK_LAYER_KHRONOS_validation will match the layer name VK_LAYER_KHRONOS_validation, but not a layer named VK_LAYER_KHRONOS_validation2 (not that there is such a layer).

This is especially useful because it is difficult sometimes to determine the full name of a driver manifest file or even some commonly used layers such as VK_LAYER_KHRONOS_validation.

Case-Insensitive

All of the filter environment variables assume the strings inside of the glob are not case-sensitive. Therefore, “Bob”, “bob”, and “BOB” all amount to the same thing.

Environment Variable Priority

The values from the disable environment variable will be considered before the enable or select environment variable. Because of this, it is possible to disable a layer/driver using the disable environment variable, only to have it be re-enabled by the enable/select environment variable. This is useful if you disable all layers/drivers with the intent of only enabling a smaller subset of specific layers/drivers for issue triaging.

Table of Debug Environment Variables

The following are all the Debug Environment Variables available for use with the Loader. These are referenced throughout the text, but collected here for ease of discovery.

Active Environment Variables

Deprecated Environment Variables

These environment variables are still active and supported, however support may be removed in a future loader release.

Glossary of Terms