Debugging The Vulkan Desktop Loader
Table of Contents
Debugging Issues
If your application is crashing or behaving weirdly, the loader provides several mechanisms for you to debug the issues.
NOTE: This functionality is all specific to the desktop Vulkan loader and does not work for the Android loader.
Loader Logging
The Vulkan desktop loader has added logging functionality that can be enabled by using the VK_LOADER_DEBUG environment variable. The results will be output to the standard output, but will also be passed to any VK_EXT_debug_utils messengers present as well. The variable can be set to a comma-delimited list of debug level options which include:
- error Report any errors encountered by the loader
- warn Report any warnings encountered by the loader
- info Report info-level messages generated by the loader
- debug Report debug-level messages generated by the loader
- layer Report all layer-specific messages generated by the loader
- driver Report all driver-specific messages generated by the loader
- all Report all messages generated by the loader (includes all of the above)
If you're not sure where the issue comes from, at least set it to output all messages through the "info" level:
set VK_LOADER_DEBUG=error,warn,info
Then, you can search the list for any errors or warnings that might provide a hint at why you're seeing issues.
For more info on enabling loader logging, refer to the Enable Loader Debug Layer Output and the Table of Debug Environment Variables below.
Debugging Possible Layer Issues
Enable Layer Logging
If you suspect a layer issue, set the loader logging to specifically output layer messages in addition to warnings and errors:
set VK_LOADER_DEBUG=error,warn,layer
Most important layer messages should go out with error or warning levels set, but this will provide more layer-specific info as well such as:
- What layers are found
- Where they were found
- If they are implicit, what environment variables can be used to disable them
- If there is any incompatibility with a given layer, this could include:
- The layer library file (.so/.dll) wasn't found
- The layer library is the wrong bit-depth for the executing application (i.e. 32-bit vs 64-bit)
- The layer itself doesn't support the application desired version of Vulkan
- If any environment variables are disabling any layers
For example, the output of the loader looking for implicit layers may look like the following:
LAYER: Searching for layer manifest files
LAYER: In following locations:
LAYER: /home/${USER}/.config/vulkan/implicit_layer.d
LAYER: /etc/xdg/vulkan/implicit_layer.d
LAYER: /usr/local/etc/vulkan/implicit_layer.d
LAYER: /etc/vulkan/implicit_layer.d
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d
LAYER: /home/${USER}/.local/share/flatpak/exports/share/vulkan/implicit_layer.d
LAYER: /var/lib/flatpak/exports/share/vulkan/implicit_layer.d
LAYER: /usr/local/share/vulkan/implicit_layer.d
LAYER: /usr/share/vulkan/implicit_layer.d
LAYER: Found the following files:
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d/renderdoc_capture.json
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d/steamfossilize_i386.json
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d/steamfossilize_x86_64.json
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d/steamoverlay_i386.json
LAYER: /home/${USER}/.local/share/vulkan/implicit_layer.d/steamoverlay_x86_64.json
LAYER: /usr/share/vulkan/implicit_layer.d/nvidia_layers.json
LAYER: /usr/share/vulkan/implicit_layer.d/VkLayer_MESA_device_select.json
Then, the loading of layer libraries is reported similar to this:
LAYER | DEBUG: Loading layer library libVkLayer_khronos_validation.so
LAYER | INFO: Insert instance layer VK_LAYER_KHRONOS_validation (libVkLayer_khronos_validation.so)
LAYER | DEBUG: Loading layer library libVkLayer_MESA_device_select.so
LAYER | INFO: Insert instance layer VK_LAYER_MESA_device_select (libVkLayer_MESA_device_select.so)
Finally, when the Vulkan instance is created, you can see the full instance call-chain from a functional standpoint with output like this:
LAYER: vkCreateInstance layer callstack setup to:
LAYER: <Application>
LAYER: ||
LAYER: <Loader>
LAYER: ||
LAYER: VK_LAYER_MESA_device_select
LAYER: Type: Implicit
LAYER: Disable Env Var: NODEVICE_SELECT
LAYER: Manifest: /usr/share/vulkan/implicit_layer.d/VkLayer_MESA_device_select.json
LAYER: Library: libVkLayer_MESA_device_select.so
LAYER: ||
LAYER: VK_LAYER_KHRONOS_validation
LAYER: Type: Explicit
LAYER: Manifest: /usr/share/vulkan/explicit_layer.d/VkLayer_khronos_validation.json
LAYER: Library: libVkLayer_khronos_validation.so
LAYER: ||
LAYER: <Drivers>
In this scenario, two layers were used (the same two that were loaded earlier):
VK_LAYER_MESA_device_selectVK_LAYER_KHRONOS_validation
This information now shows us that the VK_LAYER_MESA_device_select is loaded first, followed by VK_LAYER_KHRONOS_validation which will then continue into any available drivers. It also shows that VK_LAYER_MESA_device_select is an implicit layer which implies that it wasn't directly enabled by the application. On the other hand, VK_LAYER_KHRONOS_validation is shown as an explicit layer which indicates that it was likely enabled by the application.
Disable Layers
NOTE: This functionality is only available with Loaders built with version 1.3.234 of the Vulkan headers and later.
Sometimes, implicit layers can cause issues with an application. Because of this, the next step is to try to disable one or more of the listed implicit layers. You can use the filtering environment variables (VK_LOADER_LAYERS_ENABLE and VK_LOADER_LAYERS_DISABLE) to selectively enable or disable various layers. If you're not sure what to do, try disabling all implicit layers manually by setting VK_LOADER_LAYERS_DISABLE to 'implicit'.
set VK_LOADER_LAYERS_DISABLE=~implicit~
This will disable all implicit layers and the loader will report any disabled layers to the logging output when layer logging is enabled in the following way:
WARNING | LAYER: Implicit layer "VK_LAYER_MESA_device_select" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
WARNING | LAYER: Implicit layer "VK_LAYER_AMD_switchable_graphics_64" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
WARNING | LAYER: Implicit layer "VK_LAYER_Twitch_Overlay" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
Selectively Re-enable Layers
NOTE: This functionality is only available with Loaders built with version 1.3.234 of the Vulkan headers and later.
When trying to diagnose problems caused by layers, it is useful to first disable all layers and re-enable each layer individually. If the problem reappears, then it is immediately clear which layer is the source of the issue.
For example, from the above given list of disabled layers, let’s selectively re-enable one:
set VK_LOADER_LAYERS_DISABLE=~implicit~
set VK_LOADER_LAYERS_ENABLE=*AMD*
This would keep both the "VK_LAYER_MESA_device_select" and "VK_LAYER_Twitch_Overlay" layers disabled, while enabling the "VK_LAYER_AMD_switchable_graphics_64" layer. If everything continues to work, then the evidence seems to suggest the issue is likely not related to the AMD layer. This would lead to enabling one other layer and trying again:
set VK_LOADER_LAYERS_DISABLE=~implicit~
set VK_LOADER_LAYERS_ENABLE=*AMD*,*twitch*
And so forth.
For more info on how to use the filtering environment variables, refer to the Layer Filtering section of the LoaderLayerInterface document.
Debugging Possible Driver Issues
Enable Driver Logging
NOTE: This functionality is only available with Loaders built with version 1.3.234 of the Vulkan headers and later.
If you suspect a driver issue, set the loader logging to specifically output driver messages:
set VK_LOADER_DEBUG=error,warn,driver
Most important driver messages should go out with error or warning levels set, but this will provide more driver-specific info as well such as:
- What drivers are found
- Where they were found
- If there is any incompatibility with a given driver
- If any environment variables are disabling any of the drivers
For example, the output of the loader looking for drivers on a Linux system may look like the following (NOTE: additional spaces have been removed from the output for easier reading):
DRIVER: Searching for driver manifest files
DRIVER: In following folders:
DRIVER: /home/$(USER)/.config/vulkan/icd.d
DRIVER: /etc/xdg/vulkan/icd.d
DRIVER: /etc/vulkan/icd.d
DRIVER: /home/$(USER)/.local/share/vulkan/icd.d
DRIVER: /home/$(USER)/.local/share/flatpak/exports/share/vulkan/icd.d
DRIVER: /var/lib/flatpak/exports/share/vulkan/icd.d
DRIVER: /usr/local/share/vulkan/icd.d
DRIVER: /usr/share/vulkan/icd.d
DRIVER: Found the following files:
DRIVER: /usr/share/vulkan/icd.d/intel_icd.x86_64.json
DRIVER: /usr/share/vulkan/icd.d/lvp_icd.x86_64.json
DRIVER: /usr/share/vulkan/icd.d/radeon_icd.x86_64.json
DRIVER: /usr/share/vulkan/icd.d/lvp_icd.i686.json
DRIVER: /usr/share/vulkan/icd.d/radeon_icd.i686.json
DRIVER: /usr/share/vulkan/icd.d/intel_icd.i686.json
DRIVER: /usr/share/vulkan/icd.d/nvidia_icd.json
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/intel_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/lvp_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/radeon_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/lvp_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_lvp.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/radeon_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_radeon.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/intel_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_intel.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/nvidia_icd.json, version "1.0.0"
Then when the application selects the device to use, you will see the Vulkan device call chain reported in the following way (NOTE: additional spaces have been removed from the output for easier reading):
DRIVER: vkCreateDevice layer callstack setup to:
DRIVER: <Application>
DRIVER: ||
DRIVER: <Loader>
DRIVER: ||
DRIVER: <Device>
DRIVER: Using "Intel(R) UHD Graphics 630 (CFL GT2)" with driver: "/usr/lib64/libvulkan_intel.so"
Selectively Enable Specific Drivers
NOTE: This functionality is only available with Loaders built with version 1.3.234 of the Vulkan headers and later.
You can now use the filtering environment variables (VK_LOADER_DRIVERS_SELECT and VK_LOADER_DRIVERS_DISABLE) to control what drivers the loader will attempt to load. For drivers, the string globs passed into the above environment variables will be compared against the driver JSON file name since there is no driver name known to the loader until much later in the Vulkan initialization process.
For example, to disable all drivers except Nvidia you could do the following:
set VK_LOADER_DRIVERS_DISABLE=*
set VK_LOADER_DRIVERS_SELECT=*nvidia*
The loader outputs messages like the following when the environment variables are used:
WARNING | DRIVER: Driver "intel_icd.x86_64.json" ignored because not selected by env var 'VK_LOADER_DRIVERS_SELECT'
WARNING | DRIVER: Driver "radeon_icd.x86_64.json" ignored because it was disabled by env var 'VK_LOADER_DRIVERS_DISABLE'
For more info on how to use the filtering environment variables, refer to the Driver Filtering section of the LoaderDriverInterface document.