Building a UI toolkit with GPU rendering

Hands On Mini Tutorial Series

Preamble

This is the start of a series of mini tutorials in the style of teaching what I’m learning.

It is my belief that the state of cross-platform application development is miserable. There’s no reliable usable toolkit that does the job properly.

So let’s figure out how to make our own.

Sounds fun?

This will be a hands-on, follow-along kind of series. If you would like to get the most out of it, you should be doing your own experiments, and learning from first-hand experience.

I’m not an expert myself. Don’t expect this material to be a kind of reference. Instead think of it as a guide. I’m trying to show you the way. Figure out which concepts are relevant to us, and then try to learn them and apply them.

We’ll start with rendering 2D shapes on the GPU. Then we’ll move on to widget layout, text layout, and more advanced stuff.

By the end, we’ll hopefully have some kind of a toolkit for rendering UI, and hopefully it’ll be cross-platform.

ShaderToy and GraphToy

GPUs are designed to render 3D objects. To render 2D shapes, we basically tell it to draw one big rectangle, then we tell it how to color each pixel on that rectangle. I won’t get into the details because I myself don’t know them very much, but we will start with shadertoy.com for they already have this setup!

You have a fragment shader, which is a function that recieves a point on the screen (fragCoord) and returns the desired color for that point (fragColor). The point is vector with two components: x and y, that basically represents the pixel coordinates. The color is a four component vector: r, g, b, a. The values are floating points and go from 0.0 to 1.0.

I don’t quite like the default names, so I change them to ‘color’ and ‘point’.

Here’s an example shader that just returns green for every single pixel:

Drawing 2D shapes on the GPU is very different from drawing them on the CPU.

When you draw on the CPU, you have a desired shape, and you have to figure out which pixels need to be set to which color. The challenging part is devising the algorithm that covers all these pixels in the shortest amount of time.

When you draw 2D shapes with the GPU, you always cover all the pixels. The challenge is then how to figure out the color of each pixel on the screen independently. There’s no “state” about the next or previous pixel.

We can do “clever” things to generate a gradient, for example, instead of just plain red, we can vary the red component based on the horizontal position of the point. The further it is to the left, the more red.

How can we use this capability to draw 2D shapes? That’s what we’ll start talking about in the first installment.

Now, to make sure you follow along and write your own code, I won’t be sharing any code sample. I’ll only be sharing screenshots from shadertoy.

Another great utility I’ll be using from time to time is graphtoy.com. It helps us visualize some of the function that we’ll be using.

See you next time!