Tuesday, September 19, 2017

How to Process Mouse Events in a 3D Scene, Part 3

3. Copy the Object ID buffer into local program memory to work with.

First, your program must allocate an array to hold the ID of the object underneath each pixel:

GLuint *data;
data = new GLuint[windowDimensions.x * windowDimensions.y];

Then, after drawing the scene each frame, copy the values in the Object ID color buffer to this array.  While glBindFramebuffer selects which framebuffer we read from during reading operations, the glReadBuffer function selects which attachment within that framebuffer to read from.

glReadPixels(0, 0, windowDimensions.x, windowDimensions.y, GL_RED_INTEGER, GL_UNSIGNED_INT, data);

Here we set the read buffer to our ObjectID color buffer, copy the object IDs into our data array, and then set the read buffer back to the original color attachment so that we can blit to the default framebuffer.


Now, when a mouse event comes in we can easily discover what object was clicked on.

auto handleMouseClick = [windowDimensions, &data](int x, int y, int mouseButton) {
    unsigned int objectID = data[x + y * windowDimensions.x];

Until next time!

Part 1
Part 2

How to Process Mouse Events in a 3D Scene, Part 2

2. Modify the vertex and fragment shaders

The shaders must be modified to write the object ID to the new color buffer.  Given an existing vertex shader, it must be modified with the lines:

flat out unsigned int ObjectID;
uniform unsigned int inObjectID;
void main()

    ObjectID = inObjectID;
    // etc.

This declares a new uniform providing the object id to the vertex shader from the program, and passing it on to the fragment shader.  Similarly, the fragment shader must be modified like this:

flat in unsigned int ObjectID;
out unsigned int outObjectID;
void main()
    outObjectID = ObjectID;
    // etc.


Similarly, the first two lines here declare an input variable from the vertex shader, and an output variable that is bound to a framebuffer attachment.

The flat modifier simply means that the value is not interpolated across vertices, and is required to use an integer type in this way.

Before linking the shader program, you must bind the outObjectID out variable of the fragment shader to the proper framebuffer attachment.  The second argument to the following function specifies which color attachment to bind the out variable to.  Here, we want to bind to GL_COLOR_ATTACHMENT1.

glBindFragDataLocation(shaderProgram, 1, "outObjectID");

Additionally, when drawing the scene, before drawing each object you must send its unique ID to the graphics card, updating the uniform.

GLuint objectIDUniform = glGetUniformLocation(shaderProgram, "inObjectID");
glUniform1ui(objectIDUniform, object.id);

The first line gets a handle to the shader uniform, and the second line sets its value to the object's unique ID.  A good time to update the objectID uniform might be right before or after you update the model matrix with the object's position in world space.

Part 1
Part 3

How to Process Mouse Events in a 3D Scene, Part 1

1. Create the new color buffer

Unfortunately, OpenGL's default framebuffer does not support adding arbitrary color buffers.  The only way to write to multiple color buffers in your fragment shader, is to create an FBO to use instead of the default framebuffer.  Then, you must blit from the FBO to the default framebuffer.

Here's a block of code to do this.  Because we are creating a brand new FBO to draw the scene to, and do not have access to the default framebuffer, we must also create color and depth/stencil buffers for drawing the scene normally.  Here, windowDimensions is a struct with x and y properties giving, well, the window dimensions.

/* handles for the FBO and renderbuffers */
GLuint framebuffer;
GLuint rboColor;

GLuint rboObjectID;
GLuint rboDepthStencil;

/* create and bind the FBO */
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);

/* create a renderbuffer for drawing colors, and bind it to GL_COLOR_ATTACHMENT0 */
glGenRenderbuffers(1, &rboColor);
glBindRenderbuffer(GL_RENDERBUFFER, rboColor);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA32F, windowDimensions.x, windowDimensions.y);

/* create a renderbuffer to hold the Object IDs, and bind it to GL_COLOR_ATTACHMENT1 */
glGenRenderbuffers(1, &rboObjectID);
glBindRenderbuffer(GL_RENDERBUFFER, rboObjectID);
glRenderbufferStorage(GL_RENDERBUFFER, GL_R32UI, windowDimensions.x, windowDimensions.y);

/* create a depth/stencil buffer, and bind it to GL_DEPTH_STENCIL_ATTACHMENT */
glGenRenderbuffers(1, &rboDepthStencil);
glBindRenderbuffer(GL_RENDERBUFFER, rboDepthStencil);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, windowDimensions.x, windowDimensions.y);

Due to the glBindFramebuffer call, all drawing will go to our new FBO instead of to the default framebuffer.

Again, since FBOs are typically used for offscreen rendering, after drawing the scene we must blit from our FBO to the default framebuffer in order for the user to see the scene.

glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);

    0, 0, windowDimensions.x, windowDimensions.y,
    0, 0, windowDimensions.x, windowDimensions.y,
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, framebuffer);

glfwSwapBuffers(window); /* or whatever you use to swap buffers */

The first argument to the glBindFramebuffer can be GL_DRAW_FRAMEBUFFER, GL_READ_FRAMEBUFFER, or GL_FRAMEBUFFER which sets both.  Here, we set the draw framebuffer to 0, i.e. the default framebuffer, leaving the read framebuffer set to our FBO.  After blitting, we set the draw framebuffer back to our FBO so that drawing happens in the correct spot.

One more detail.  When clearing our FBO, we must make sure to clear both color attachments.  Here we use the glClearBuffer* function, the second argument of which tells which attachment we wish to clear.

float clearColor[] = {0.0f, 0.0f, 0.0f, 1.0f};
unsigned int clearObject[] = {0, 0, 0, 0};
glClearBufferfv(GL_COLOR, 0, clearColor);

glClearBufferuiv(GL_COLOR, 1, clearObject);
glClearBufferfi(GL_DEPTH_STENCIL, 0, 1, 0);

Part 2
Part 3

How to Process Mouse Events in a 3D Scene

Generally, to tell what object a user clicked on in a 3D scene, you have to cast a ray from the camera through the mouse cursor, and find the nearest object that it hits.  Incidentally, your GPU already does this each frame as it renders the scene.  This can be leveraged so that you don't need to actually cast a ray in your program, instead using the information that your GPU computes anyway each frame.

As your graphics card writes to the color buffer each frame, it manages a depth buffer tracking the distance of each fragment from the viewer.  Generally, it only updates the color buffer if it also updates the depth buffer.

Your GPU can easily be programmed to write to another buffer at the same time as it paints the scene - another "color buffer" in OpenGL terminology - but one that does not store a color, but rather an integer giving you the ID of the object beneath that pixel.

This way, when the user clicks or moves the mouse, you can find out what object they interacted with just by accessing this buffer.  There is no need to loop through all of the objects in the scene or do any messy ray casting operations at all.

Here are the basic steps:

1. Create a new color buffer with a 32-bit unsigned int as its internal format.  This is a suitable data type to hold the unique IDs of objects in a 3D scene.

2. Add a uniform to the vertex shader representing the object ID, and pass this value unchanged to the fragment shader.  From there, write this value to our new color buffer.

3. After drawing each frame, copy this new color buffer into a local array in your program.  When the user clicks or moves the mouse, you can tell what object was clicked on just by indexing into this array.

Part 1
Part 2
Part 3

Saturday, September 16, 2017

Friday, September 15, 2017


hey i'm into cryptocurrencies i'm mining ethereum

Monday, August 28, 2017

Shearing Vitali Sets


As everybody knows, the rational numbers are a countable subset of the reals.  The reals are uncountable; almost all real numbers are not rational numbers.  It just so happens that the difference between any two rational numbers is also a rational number.

If you add to each rational number a certain irrational number such as sqrt(2), you arrive at another countable subset of the real numbers, but a disjoint one from the original rational numbers.  Again, the difference between any two elements of the set Q + sqrt(2) is a rational number, because the sqrt(2)'s cancel out when they are subtracted.

For example, above the dotted line is the set Q of rational numbers (some points are omitted).  Below the dotted line is the set Q + sqrt(2) of rational numbers increased by sqrt(2).  The set below the line looks very much like the set above the line - except all of the points are shifted to the right.

Consider the following equivalence relation ~ on the real numbers.
We say x ~ y (x is equivalent to y) if there exists a rational number q such that x = y + q.

Under this relation, all of the elements of Q (the points above the dotted line) are equivalent to one another; and all of the elements of Q + sqrt(2) (the points below the dotted line) are equivalent to one another.

A vitali set is a set with exactly one representative from each equivalence class.

In other words a vitali set is a set V such that for each real number r, there is exactly one v in V such that r - v is rational.


Consider a vitali set V.

Let F by the set of bijections from V to [0, 1].  There are uncountably infinitely many of these bijections.

For each f in F, consider the set W(f) = {v + f(v) : v in V}.  (That is, take each element v of the vitali set V and add to it f(v), its index from [0, 1].)  For how many functions f in F is W(f) a vitali set?  Almost all?  Almost none?

I don't know the answer to this question; it seems rather interesting.

Monday, May 1, 2017

How to Use Vim (without developing (as much) RSI)

Instead of reaching over to Esc all the time, just type Ctrl+[

Tuesday, April 18, 2017

Best Games

These are the best games of all time.

The Witcher 1/2/3

Tactical fighting games.  Very deep, clearly made by people who have read much.

Morrowind, Oblivion

Story-based RPGs with great modding support.  Skyrim is not listed because the dragons are too small and it's kind of dull.  However these games are fantastic.

Master of Orion: Conquer the Stars

A worthy successor to moo, moo 2, and moo 3.

Red Alert 3

Fast-paced RTS.

The Binding of Isaac

Probably Turing complete, if I had to guess.

Sunday, April 2, 2017

Asians for President!

I think they would make a good president.

Tuesday, January 24, 2017

Saturday, January 14, 2017