bulb

Overview ¶

Package bulb implements a basic network load balancer for educational purposes.

Goal ¶

BULB implements functionality for an HTTP(S) server that proxies the requests it receives to a pool of backend HTTP(S) servers, spreading the load between them. It provides various different strategies for selecting which backend server should handle each request. BULB's primary purpose is to demonstrate basics of the topic and how they can be implemented in Go. Performance, security, and efficiency are secondary goals.

Design ¶

The design is quite simple, consisting of backends, backend pools, strategies, and load balancers. Each is described below in the order in which they are constructed.

Backends ¶

Each backend server is represented by an instance of Backend. This contains the URL describing the path from the load balancer to the backend host. It also contains the backend-specific configuration for health checks. This includes generating the HTTP(S) requests that should be sent to this backend during health checks, and the logic for determining whether a response demonstrates good health. Backend-agnostic health check behaviour is configured in the load balancer, documented below.

The most common case will be a backend that takes health checks with the GET method on the same port, using the path `/healthz`. Such a backend can be constructed using SimpleBackend as follows:

backend, err := bulb.SimpleBackend(addr, "/healthz", "GET")

Backend pools ¶

Backends are grouped together in pools. This allows careful management of the set of backends, so that new backends can be added at runtime without causing a harmful data race. Idiomatically, the pool is created from an existing set of backends with NewPool, with new backends added at runtime with Pool.AddBackends. Alternatively, the zero value of a pool can be created with `var pool bulb.Pool` and the backends added later with `pool.AddBackends`.

Pools are used by load balancing strategies to select backends. The list of backends can be accessed either by looping over the iterator returned by Pool.All, Pool.Healthy, or Pool.Unhealthy, or by indexing into it using Pool.Get/Pool.GetHealthy. Both approaches are safely guarded by a mutex to ensure that adding more backends does not produce a data race. Pool.Get ensures that the given index is reduced modulo the number of backends if necessary, so that the result is always in bounds.

Strategies ¶

The load balancing process uses a strategy to select which backend should handle each request. Different strategies have different performance and behavioural characteristics. This package implements the following two strategies, which are documented below:

• HashIP
• RoundRobin

The HashIP strategy hashes the IP address of the client making the request and uses the result as an index into the pool of backends. This has reasonable performance and ensures that successive requests by the same client will be handled by the same backend. However, this property may not hold if new backends are added, the backend becomes unhealthy, or the client's IP address changes. This may lead to load being balanced unevenly across backends.

The RoundRobin strategy sends successive requests to sequential backends. This is simple and quick, but does not consider the current load of each backend.

Users can also implement a custom Strategy by implementing the interface.

Load balancers ¶

Having prepared a pool of backends and a load balancing strategy, a LoadBalancer can now be prepared using New. Additional configuration for a load balancer can be passed using one or more Option values. For example, the error logger can be configured using WithLogger and health checks can be enabled using WithHealthChecks. See below for documentation on health checks. The load balancer is a net/http.Handler, so it is typically used as the handler in a *net/http.Server or a *net/http/httptest.Server.

Health checks ¶

Health checks are an important part of load balancing. They are used to identify backends that have become unavailable or suffered some kind of error. Once a backend has become unhealthy, subsequent requests are not sent to the backend until it becomes healthy again. This maximises the health of the overall service. When backends are added to a backend pool, they are marked healthy by default. Once a load balancer's health checks are started by calling LoadBalancer.StartHealthChecks, the load balancer will send regular health checks to check whether the backend is still healthy. The health checks are configured using the HealthCheckConfig passed to the load balancer using the WithHealthChecks option.

Example ¶

An example load balancer, with health checks, is as follows:

// Construct the set of backends.
backends := []*bulb.Backend{
	bulb.SimpleBackend("http://10.0.0.1:8080/", "/healthz", http.MethodHead),
	bulb.SimpleBackend("http://10.0.0.2:8080/", "/healthz", http.MethodHead),
	bulb.SimpleBackend("http://10.0.0.3:8080/", "/healthz", http.MethodHead),
}

// Gather the backends into a pool.
pool, err := bulb.NewPool(backends)
if err != nil {
	// Handle the error.
}

strategy := bulb.RoundRobin()
config := &bulb.HealthCheckConfig{
	Interval:         30 * time.Second,
	FailureThreshold: 3,
}

balancer, err := bulb.New(pool, strategy, bulb.WithHealthChecks(config))
if err != nil {
	// Handle the error.
}

// Start the health checks in a background
// goroutine.
cancel, err := balancer.StartHealthChecks()
if err != nil {
	// Handle the error.
}

defer cancel() // Stop the health checks once we're done.

// Start the HTTP server for the load balancer.
err = http.ListenAndServe(addr, balancer)
if err != nil {
	// Handle the error.
}