Tuesday, September 19, 2017

How to Process Mouse Events in a 3D Scene, Part 3

3. Copy the Object ID buffer into local program memory to work with.

First, your program must allocate an array to hold the ID of the object underneath each pixel:

GLuint *data;
data = new GLuint[windowDimensions.x * windowDimensions.y];

Then, after drawing the scene each frame, copy the values in the Object ID color buffer to this array.  While glBindFramebuffer selects which framebuffer we read from during reading operations, the glReadBuffer function selects which attachment within that framebuffer to read from.

glReadPixels(0, 0, windowDimensions.x, windowDimensions.y, GL_RED_INTEGER, GL_UNSIGNED_INT, data);

Here we set the read buffer to our ObjectID color buffer, copy the object IDs into our data array, and then set the read buffer back to the original color attachment so that we can blit to the default framebuffer.


Now, when a mouse event comes in we can easily discover what object was clicked on.

auto handleMouseClick = [windowDimensions, &data](int x, int y, int mouseButton) {
    unsigned int objectID = data[x + y * windowDimensions.x];

Until next time!

Part 1
Part 2

How to Process Mouse Events in a 3D Scene, Part 2

2. Modify the vertex and fragment shaders

The shaders must be modified to write the object ID to the new color buffer.  Given an existing vertex shader, it must be modified with the lines:

flat out unsigned int ObjectID;
uniform unsigned int inObjectID;
void main()

    ObjectID = inObjectID;
    // etc.

This declares a new uniform providing the object id to the vertex shader from the program, and passing it on to the fragment shader.  Similarly, the fragment shader must be modified like this:

flat in unsigned int ObjectID;
out unsigned int outObjectID;
void main()
    outObjectID = ObjectID;
    // etc.


Similarly, the first two lines here declare an input variable from the vertex shader, and an output variable that is bound to a framebuffer attachment.

The flat modifier simply means that the value is not interpolated across vertices, and is required to use an integer type in this way.

Before linking the shader program, you must bind the outObjectID out variable of the fragment shader to the proper framebuffer attachment.  The second argument to the following function specifies which color attachment to bind the out variable to.  Here, we want to bind to GL_COLOR_ATTACHMENT1.

glBindFragDataLocation(shaderProgram, 1, "outObjectID");

Additionally, when drawing the scene, before drawing each object you must send its unique ID to the graphics card, updating the uniform.

GLuint objectIDUniform = glGetUniformLocation(shaderProgram, "inObjectID");
glUniform1ui(objectIDUniform, object.id);

The first line gets a handle to the shader uniform, and the second line sets its value to the object's unique ID.  A good time to update the objectID uniform might be right before or after you update the model matrix with the object's position in world space.

Part 1
Part 3

How to Process Mouse Events in a 3D Scene, Part 1

1. Create the new color buffer

Unfortunately, OpenGL's default framebuffer does not support adding arbitrary color buffers.  The only way to write to multiple color buffers in your fragment shader, is to create an FBO to use instead of the default framebuffer.  Then, you must blit from the FBO to the default framebuffer.

Here's a block of code to do this.  Because we are creating a brand new FBO to draw the scene to, and do not have access to the default framebuffer, we must also create color and depth/stencil buffers for drawing the scene normally.  Here, windowDimensions is a struct with x and y properties giving, well, the window dimensions.

/* handles for the FBO and renderbuffers */
GLuint framebuffer;
GLuint rboColor;

GLuint rboObjectID;
GLuint rboDepthStencil;

/* create and bind the FBO */
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);

/* create a renderbuffer for drawing colors, and bind it to GL_COLOR_ATTACHMENT0 */
glGenRenderbuffers(1, &rboColor);
glBindRenderbuffer(GL_RENDERBUFFER, rboColor);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA32F, windowDimensions.x, windowDimensions.y);

/* create a renderbuffer to hold the Object IDs, and bind it to GL_COLOR_ATTACHMENT1 */
glGenRenderbuffers(1, &rboObjectID);
glBindRenderbuffer(GL_RENDERBUFFER, rboObjectID);
glRenderbufferStorage(GL_RENDERBUFFER, GL_R32UI, windowDimensions.x, windowDimensions.y);

/* create a depth/stencil buffer, and bind it to GL_DEPTH_STENCIL_ATTACHMENT */
glGenRenderbuffers(1, &rboDepthStencil);
glBindRenderbuffer(GL_RENDERBUFFER, rboDepthStencil);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, windowDimensions.x, windowDimensions.y);

Due to the glBindFramebuffer call, all drawing will go to our new FBO instead of to the default framebuffer.

Again, since FBOs are typically used for offscreen rendering, after drawing the scene we must blit from our FBO to the default framebuffer in order for the user to see the scene.

glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);

    0, 0, windowDimensions.x, windowDimensions.y,
    0, 0, windowDimensions.x, windowDimensions.y,
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, framebuffer);

glfwSwapBuffers(window); /* or whatever you use to swap buffers */

The first argument to the glBindFramebuffer can be GL_DRAW_FRAMEBUFFER, GL_READ_FRAMEBUFFER, or GL_FRAMEBUFFER which sets both.  Here, we set the draw framebuffer to 0, i.e. the default framebuffer, leaving the read framebuffer set to our FBO.  After blitting, we set the draw framebuffer back to our FBO so that drawing happens in the correct spot.

One more detail.  When clearing our FBO, we must make sure to clear both color attachments.  Here we use the glClearBuffer* function, the second argument of which tells which attachment we wish to clear.

float clearColor[] = {0.0f, 0.0f, 0.0f, 1.0f};
unsigned int clearObject[] = {0, 0, 0, 0};
glClearBufferfv(GL_COLOR, 0, clearColor);

glClearBufferuiv(GL_COLOR, 1, clearObject);
glClearBufferfi(GL_DEPTH_STENCIL, 0, 1, 0);

Part 2
Part 3

How to Process Mouse Events in a 3D Scene

Generally, to tell what object a user clicked on in a 3D scene, you have to cast a ray from the camera through the mouse cursor, and find the nearest object that it hits.  Incidentally, your GPU already does this each frame as it renders the scene.  This can be leveraged so that you don't need to actually cast a ray in your program, instead using the information that your GPU computes anyway each frame.

As your graphics card writes to the color buffer each frame, it manages a depth buffer tracking the distance of each fragment from the viewer.  Generally, it only updates the color buffer if it also updates the depth buffer.

Your GPU can easily be programmed to write to another buffer at the same time as it paints the scene - another "color buffer" in OpenGL terminology - but one that does not store a color, but rather an integer giving you the ID of the object beneath that pixel.

This way, when the user clicks or moves the mouse, you can find out what object they interacted with just by accessing this buffer.  There is no need to loop through all of the objects in the scene or do any messy ray casting operations at all.

Here are the basic steps:

1. Create a new color buffer with a 32-bit unsigned int as its internal format.  This is a suitable data type to hold the unique IDs of objects in a 3D scene.

2. Add a uniform to the vertex shader representing the object ID, and pass this value unchanged to the fragment shader.  From there, write this value to our new color buffer.

3. After drawing each frame, copy this new color buffer into a local array in your program.  When the user clicks or moves the mouse, you can tell what object was clicked on just by indexing into this array.

Part 1
Part 2
Part 3

Saturday, September 16, 2017

Friday, September 15, 2017


hey i'm into cryptocurrencies i'm mining ethereum