Game Developer & Computer Graphics Engineer
I'm working on a Vulkan render engine and I wanted to be able to click on objects in the scene to select them, as a programmer the first thing I did was a quick google search, and I was a bit disappointed with the amount of material I found, so after managing to implement it myself
I decided to make a blog post about it.
Quick disclaimer, this is by no means the best way to implement object picking, but I will show what worked for me.
Lets start with a quick theoretical explanation, what we're trying to do here is give every object an ID, you can either pass it through descriptors or vertex attributes (I chose the latter, for bigger projects I would recommend using descriptor sets), and then write that ID into an ssbo in the fragment shader.
Cool!, lets start implementing, we'll start by creating the color and depth attachment for the frame buffer, The ID is a uint32_t so we have a max of 4,294,967,295 objects. You'll notice that our color attachment is VK_FORMAT_R32_FLOAT, for some reason when the texture is UINT instead, I get garbage values, so I stuck with that, since it is for debug purposes anyway.
Now VkFramebuffer creation, notice that there is a seperate renderpass for object picking.
Object Picking VkRenderPass creation:
VkCommandBuffer creation:
And a seperate VKPipeline, now most of this VKPipeline is pretty standard , a few things are important to mention.
First is that we're going to have a VkDynamicState for the scissor. This is because every click we're going to set the scissor to a size of (1,1) and an offset of (mouse_position.x,mouse_position.y). This is an easy optimisation meaning we can write the ID's in the fragment, knowing it only runs once.
The second thing is that you need to also bind an SSBO (Shader Storage Buffer Object) to the desciptor set of every VKPipeline you want to access the ID from, for either read and/or write (meaning we definitely need it in the object picking VKPipeline).
Vertex/Fragment shaders:
To make sure I execute the Object picking VKRenderpass only when the mouse is clicked, I set an internal flag before rendering that gets reset when rendering ends
Command buffer recording:
Now that the ssbo has the value of the chosen object, I can either read it on the CPU if its host-visible(which is slower) or just read the ssbo in whatever shader stage I want.
Example of ssbo reading on cpu
Possible Improvements
So there are a few optimizations I didn't need but could be implemented.
-
The color attachment is only useful for debugging as shown below, and could be removed as soon as you see everything is working as intended.
-
You could also use multiple render targets, meaning the whole object picking could be done in one renderpass and one drawcall.
Example of color attatchment on one object when the scissor is set to the full screen, useful for debugging
ID:30.0