HCF EP 001: Empty Project Setup

2024.12.06

This is the first article about the development of "HandCraftedForum"

Repository URL: https://github.com/hasenj/HandCraftedForum

HandCraftedForum is a web project that will use a mini framework I have developed over the course of the last couple of years.

As such, the base/skeleton project setup will reflect the framework.

The framework overall does not have a particular name, but it's composed of the following components:

VBolt: a storage layer built on top of BoltDB and VPack, a serialization library.

VBeam: the server side web framework. It mostly consists of an RPC system that lets the client side call functions on the server side without thinking about ReST or HTTP.

VLense: a client side framework, consist mostly of routing and some utilities to help create UIs (p)react style without callbacks or "hooks". (We provide our own version of what hooks are meant to do; more on that later).

The overall theme is straight forward programming with data and procedures. The server side code uses VBolt to store and retrieve data. VBeam provides an interface to the client to communicate with the server side code. The client code renders the UI using the data it obtained from the server + transformation applied via user interaction.

The skeleton of the project will let us run a web server locally and opens a mostly empty page with a welcome message rendered via client side code.

License

Before we start, we should choose a license. This is mostly a technicality but has implications.

I'm generally in favor of open source for libraries and code snippets, but not for final products. I believe software products should be sold for money.

I asked Claude if there's a license I can use that allows people to freely use the source code and study from it while prohibiting people from taking the product as-is and rebranding it as their own. I didn't think there was such an option, but to my surprise, turns out there is: the

commons clause

combined with any other license.

So I combined the commons clause with the MIT license.

The license does not comply with the OSI definition of "open source", but we don't care.

The license lets you use the code freely in all the way that matters. It just does not late you take the final product and make money from it.

Check the linked website for the FAQ regarding the license.

Server side code

First we create a module using go mod init forum

module forum

go 1.22.1

Then we create app.go:

go

package forum

import (
    "forum/cfg"

    "go.hasen.dev/vbeam"
    "go.hasen.dev/vbolt"
)

func MakeApplication() *vbeam.Application {
    vbeam.RunBackServer(cfg.Backport)
    db := vbolt.Open(cfg.DBPath)
    var app = vbeam.NewApplication("HandCraftedForum", db)
    return app
}

This creates a mostly empty application.

The MakeApplication function is mostly a convention I use. Note that this package is not the main package, so there will be no main function here. Instead, we will have two different main packages: one that we run locally, and one that we run in production.

The local and production version differ in the following ways:

• The local server also bundles the frontend code and performs type checking on it

• It generates the server side bindings for the client (more on that later)

• It uses different paths for database file and static folder

• The local server serves the frontend code from the local file system, while the production server embeds the generated frontend bundle and serves it from RAM. In fact, the code is written to expect the frontend bundle to have been already generated in a prior step.

Let's look at this function line by line:

go

vbeam.RunBackServer(cfg.Backport)

This innocent looking line does something very important: it tells any previously running instance of this server to shutdown.

A "back server" is a private backdoor that lets us control the server. The way we do that is by using a specific port number that no other program uses. If you have multiple servers running and they all use this framework, you need to pick these port numbers such that they do not conflict.

The first thing a back server does is send a "shutdown" command to the given port number. If a server is already listening there, it will shutdown.

This allows nearly instant deployment with zero shut down time.

The next line create a db instance. For now we have nothing on the db, so it doesn't matter what we do with it. It's just that the db is required for the next line: the vbeam function that creates a new application and sets up the RPC system.

Again this is the empty skeleton application so we will just return it as-is.

We also create a cfg package to store some code level configuration variables that vary between local and production. In this case, we see cfg.DBPath.

There's also a variable that does not vary between local and production: the backport number. I just chose this number arbitrarily.

cfg/cfg.go

package cfg

const Backport = 12832

To distinguish local from production, we use build tags.

cfg/local.go

//go:build !release

package cfg

const IsRelease = false
const DBPath = ".serve/data/db.bolt"
const StaticDir = ".serve/static/"

cfg/release.go

//go:build release

package cfg

const IsRelease = true
const DBPath = "data/db.bolt"
const StaticDir = "static/"

Next we create the command that runs the local development server. This will mostly follow a template.

First we create a directory local and create local.go and name the package main so it can be executed:

local/local.go

package main

func main() {

}

Now, before writing out the full content, I want to first show the function that launches the server:

import (
    "fmt"
    "forum"
    "net/http"
    "os"

    core_server "go.hasen.dev/core_server/lib"

    "go.hasen.dev/vbeam"

    "forum/cfg"
)

const Port = 5212
const Domain = "forum.localhost"
const FEDist = ".serve/frontend"

func StartLocalServer() {
    defer vbeam.NiceStackTraceOnPanic()

    app := forum.MakeApplication()
    app.Frontend = os.DirFS(FEDist)
    app.StaticData = os.DirFS(cfg.StaticDir)
    vbeam.GenerateTSBindings(app, "frontend/server.ts")

    var addr = fmt.Sprintf(":%d", Port)
    var appServer = &http.Server{Addr: addr, Handler: app}

    core_server.AnnounceForwardTarget(Domain, Port)
    appServer.ListenAndServe()
}

Notice that after we call MakeApplication that we created above, we set the Frontend and StaticData fields on it.

In the production main, we will set those variables differently. We will see when the time comes.

Next we generate the client side bindings. We don't have "RPC" yet, but when we make one, a binding module will be generated automatically.

This assumes that we will put all the frontend code in the frontend directory.

Next we prepare an http server from the Go standard library and set the handler to the app we just created, because the vbeam.Application object implements the http server interface.

The next line is important for allowing us to open the website using a domain on port 80 instead of localhost:5212

core_server.AnnounceForwardTarget(Domain, Port)

This assumes we have core_server running.

Core server is a very small reverse proxy that is configured only via UDP messages. This is the most important message: it maps a domain to a port. Meaning, when a request comes for the given domain, it reverse proxies to the given port on localhost.

The core server can be installed this way:

go install go.hasen.dev/core_server@latest

Once installed, run it this way:

nohup core_server & disown

You don't need to worry about running it multiple times; it's idempotent. If you run it again, it will shutdown the previous instance before starting.

Keep in mind: this is entirely optional. If you don't have core_server, you can always access the server using the localhost:port combination.

Next we show the main function for the local server:

var FEOpts = esbuilder.FEBuildOptions{
    FERoot: "frontend",
    EntryTS: []string{
        "main.tsx",
    },
    EntryHTML: []string{"index.html"},
    CopyItems: []string{
        "images",
    },
    Outdir: FEDist,
    Define: map[string]string{
        "BROWSER": "true",
        "DEBUG":   "true",
        "VERBOSE": "false",
    },
}

var FEWatchDirs = []string{
    "frontend",
    "frontend/images",
}

func main() {
    os.Mkdir(".serve", 0644)
    os.Mkdir(".serve/static", 0644)
    os.Mkdir(".serve/frontend", 0644)

    var args local_ui.LocalServerArgs
    args.Domain = Domain
    args.Port = Port
    args.FEOpts = FEOpts
    args.FEWatchDirs = FEWatchDirs
    args.StartServer = StartLocalServer

    local_ui.LaunchUI(args)
}

This setups the frontend builder configuration as well as the list of directories to watch for changes in order to typecheck and rebuild.

This won't make sense without talking about the frontend first.

The frontend

The frontend consists of an index.html file that loads the entry typescript file, which will start a client side router, and then render the current page depending on the URL.

frontend/index.html

<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>HandCraftedForum</title>
</head>

<body>
    <script src="/main.tsx" type="module"></script>
</body>

</html>

Nothing special here, except the fact that we are referring to /main.tsx. The frontend bundler will replace this with the path that resulted from the bundling operation.

We need to first setup the dependencies

npm install --save preact vlens
npm install --save-dev typescript

We also setup a basic tsconfig.json file and configure it to use preact for jsx.

json

{
    "compilerOptions": {
        "module": "es2020",
        "target": "es2020",
        "lib": ["es2020", "dom"],
        "moduleResolution": "node",
        "esModuleInterop": true,
        "skipLibCheck": true,
        "noEmit": true,
        "strict": true,
        "jsx": "react",
        "jsxFactory": "preact.h",
        "jsxFragmentFactory": "preact.Fragment",
        "baseUrl": "frontend",
        "noImplicitAny": true,
        "paths": {
            "@app/*": ["./*"]
        }
    },
}

Notice I've set it up so that we can use the prefix @app to refer to our own codebase.

Now we can setup a skeleton main.tsx:

import * as vlens from "vlens";

async function main() {
    vlens.initRoutes([
        vlens.routeHandler("/", () => import("@app/home")),
    ]);
}

main();

This initialize the app and sets up a single route mapping.

Client side routing works by matching a route with a fetching function and view function.

The fetch function must return a promise for some data. This data will be stored and passed to the view function. The view function always takes this data as input. The reference to this piece of data will be stable across rendering cycles.

This is a crucial feature of this mini framework: there's no state management. There's just data. Data can be used to reference other data. The "entry point" to all the data you need is the object returned from the fetch function above.

You can associate "side" data ("hooked" data) by relying on the stability of the reference. We will show how later in the project.

The routeHandler is a helper function that sets the route by dynamically importing a module and looking for "magic" names fetch and view.

Actually the second argument does not need to be a module. It can be any function that returns a promise for an object satisfying the interface:

export type RouteHandler<Data = any> = {
    fetch: (route: string, prefix: string) => Promise<rpc.Response<Data>>;
    view: (route: string, prefix: string, data: Data) => preact.ComponentChild;
}

You can just as easily do something more complicated than just loading a module, but for now we will simplify our life by just creating a module for each route with a fetch and view functions.

For the home route, we will again just make the bare minimum empty fetch and view functions:

import * as preact from "preact"
import * as rpc from "vlens/rpc";

type Data = {}

export async function fetch(route: string, prefix: string) {
    return rpc.ok<Data>({})
}

export function view(route: string, prefix: string, data: Data): preact.ComponentChild {
    return <div>
        <h2>Hand Creafted Forum</h2>
        <img src="/images/framework.png" />
    </div>
}

The fetch function fetches no data per se; it just returns an empty object as if it was an "ok" response from the server. While the empty object is generally useless, it's not entirely useless: it has a reference id, and that reference id is stable.

We will not use that yet, but it's worth keeping in mind.

Our view function just returns basically static html (as jsx).

In the jsx, we refer to an image. We create an images directory inside frontend and place the image file there.

We are now ready to run our local server. Here's a video of me doing that:

Here's the resulting page:

We have successfully created an empty skeleton project using our framework.

Note: in the video you can see me edit the jsx code and then reload the page to see the results. This is not "hot module reload". You still have to reload manually. We are just saving you the bundling step by doing it automatically when you save the code file.

Code base for Episode 001:

Web view: HandCraftedForum/tree/EP001

Zip download: EP001.zip

Follow along

Download the code base, see if you can run it locally.

If you face any problem, report it to me, either by opening issues on github or replying here on substack.