Building devserver: An Ultra-Tiny Rust Server

For my WebXR work I needed a development-only server to host a static website over HTTPS. I'd only be accessing the files on my local computer and from an Oculus Quest on the same network.

I wanted a server with the following properties:

Surprisingly it's difficult to find an easily installed webserver that fits the bill!

So I made a tiny development-only server called devserver.

devserver is a great tool for local development, but to be completely clear up-front no effort has been made to make it secure for production purposes.

You can check it out on crates.io and github.

In this post I'll describe the process of building devserver in pursuit of the above goals.

Easy and Fast to Install

If you have Rust you have cargo.

By hosting devserver on crates.io installation becomes as easy as:

cargo install devserver

Wonderful.

I love tools that install almost instantly. It's a luxurious desire, but it's a rare feeling to use software that takes up little space and installs quickly. I did not want the installation of devserver to interrupt someone's workflow.

Tools installed with cargo install build the Rust code, and Rust unfortunately has a reputation for slow compile times. Perhaps it'd be possible to distribute a prebuilt binary, but I didn't want to go that route.

Other similar Rust development servers takes 3 minutes to install. devserver takes just 25 seconds. 25 seconds is still too long for my tastes, but it's fine for now.

In order to keep install times low I was very careful choosing crates. devserver only has direct dependencies on the following 4 crates:

The Rust ecosystem is full of many excellent crates, but most web related crates are tailored towards the more complex use case of production web servers and as such take a while to build.

devserver implements a tiny version of HTTP and WebSockets to accomplish just enough to cover its use cases.

Minimalist HTTP

devserver contains a tiny HTTP implementation that isn't feature rich, but in practice covers most use cases.

A small function reads the HTTP header into a byte array:

pub fn read_header<T: Read + Write>(stream: &mut T) -> Vec<u8> {
    let mut buffer = Vec::new();
    let mut reader = std::io::BufReader::new(stream);
    loop {
        reader.read_until(b'\n', &mut buffer).unwrap();
        // Read until end of header.
        if &buffer[buffer.len() - 4..] == b"\r\n\r\n" {
            break;
        }
    }
    buffer
}

It is assumed that all requests sent to the server are GET requests, and devserver treats them as such.

Some string wrangling is done to get the correct path and file extension:

let request_string = str::from_utf8(&buffer).unwrap();

if request_string.is_empty() {
    return;
}
// Split the request into different parts.
let mut parts = request_string.split(' ');

let _method = parts.next().unwrap().trim();
let path = parts.next().unwrap().trim();
let _http_version = parts.next().unwrap().trim();

// Replace white space characters with proper whitespace.
let path = path.replace("%20", " ");
let path = if path.ends_with("/") {
    Path::new(root_path).join(Path::new(&format!(
        "{}{}",
        path.trim_start_matches('/'),
        "index.html"
    )))
} else {
    Path::new(root_path).join(path.trim_matches('/'))
};

let extension = path.extension().and_then(OsStr::to_str);

// If no extension is specified assume html
let path = if extension == None {
    path.with_extension("html")
} else {
    path.to_owned()
};
let extension = extension.unwrap_or("html");

devserver then finds the file on the disk, the file extension is associated with a MIME type (Also known as a 'Media Type'), and the HTTP response is written to the return stream:

let content_type = extension_to_mime_impl(Some(extension));
let response = format!(
    "HTTP/1.1 200 OK\r\nContent-type: {}\r\nContent-Length: {}\r\n\r\n",
    content_type, content_length
);

let mut bytes = response.as_bytes().to_vec();
bytes.append(&mut file_contents);
stream.write_all(&bytes).unwrap();

It's all shockingly simple for how well it works. This handles most cases I've thrown at it and yet it's only a few lines of code!

Automatic Reload

I use devserver to develop interactive game-like content, like the Rust game I made for Ludum Dare, and quick iteration times are critical for similar creative work.

It's a great experience to make changes to a file and have it automatically update in the browser by the time you alt-tab to it.

There are two parts to this problem:

Unfortunately watching for file and folder changes across platforms is finicky.

Fortunately the Notify crate already put in the hard work to figure it all out and it is a small enough dependency it doesn't hurt devserver's minimalist build goals too much.

Notifying the browser that it needs to refresh requires an open connection to the browser, or some sort of continuous polling. Continous polling felt too heavy handed, so that was ruled out.

The approach I settled on was using a WebSocket to notify the browser that the a file had changed. Initially I planned on sending the new file to the browser so it could reload just that file, but I later decided to just settle for the "Good enough" solution of refreshing the browser.

I evaluated various WebSocket libraries to use. Tungstenite seemed like a good solution, but sadly when I tried it out it nearly doubled devserver's clean build times, so I decided to consider other alternatives.

I wasn't too fond of the idea, but perhaps if I could implement a spartan HTTP response I could also create a spartan WebSocket implementation? All devserver needed was just enough to signal the server somehow. So I started reading resources online about WebSockets and consulting the spec.

The WebSocket standard requires a seemingly arbitrarily formatted response to declare "Yes I really am a WebSocket" which is handled by the following code:

// Perform a ceremony of getting the SHA1 hash of the sec_websocket_key joined with
// an arbitrary string and then take the base 64 encoding of that.
let sec_websocket_accept = format!(
    "{}{}",
    sec_websocket_key, "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
);
let mut hasher = Sha1::new();
hasher.input(sec_websocket_accept.as_bytes());
let result = hasher.result();
let bytes = base64::encode(&result);

format!("HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: {}\r\n\r\n",bytes)

The above code introduces two direct dependencies: the sha-1 crate and the `base64 crate, both required for the string formatting ceremony.

Once the connection is opened a blank message is sent along the WebSocket whenever a file changes. The web page refreshes whenever it receives any message from the server's WebSocket:

Ok(event) => {
    let (_path, refresh) = match event {
        /* Various events from Notify about file and folder changes */
    };

    if refresh {
        // A blank message is sent triggering a refresh on any change.
        // If this message fails to send, then likely the socket has been closed.
        if send_websocket_message(&stream, "").is_err() {
            break;
        };
    }
}

devserver injects the following snippet of Javascript into each .html file served to establish the WebSocket connection:

// This code is inserted by devserver to enable reloading.
const socket = new WebSocket("ws://" + window.location.hostname + ":8129");
socket.addEventListener('open', function (event) { console.log("Reloading enabled!"); });
socket.addEventListener('message', function (event) { location.reload(); });

The code is appended in a <script> tag to the end of the document, and even though that's not the correct spot for a <script> tag the browsers handle it totally fine without error! Hooray for lenient browsers!

So with the combination of the Notify crate and a MacGyver-ed WebSocket implementation automatic page reloading works with only a small hit to devserver's build times.

I could never remember (even though it's my own code!) if the flag was "--reload" or "--refresh" so devserver accepts both so I can never make that error again. Eventually I just made the flag on by default because I was using it every time I ran devserver.

HTTPS

Many development servers require configuring a certificate for HTTPS support, which can be a pain if all you want to do is locally develop for web APIs that require HTTPS (as WebXR does).

devserver is very much not an acceptable production server. This lets devserver cut corners to make the experience of local development simpler.

Instead of using a valid certificate devserver just uses an invalid hardcoded security certificate. Web browsers warn that the certificate is invalid, but it is easy to ignore the warnings. You should not do this normally, but for local development it can be OK.

This is absolutely my least favorite part of devserver, and it's something I'd like to change in the future if possible.

It's not an elegant solution, but it allows devserver to run as a single command and support both https and http with zero configuration.

The native-tls crate is used to handle opening the secure HTTPS socket. native-tls was a great choice for devserver because it handles the complex requirements of HTTPS but avoids lengthy build times by using the native platform TLS implementations.

devserver handles both HTTPS and HTTP connections at the same time without a setting by checking which type an incoming connection is.

HTTP requests always begin with a verb like GET, but HTTPS requests always begin with a number. By peeking the first few bytes devserver can decide which path to take:

let mut buf = [0; 2];
stream.peek(&mut buf).expect("peek failed");

let is_https =
    !((buf[0] as char).is_alphabetic() && (buf[1] as char).is_alphabetic());

if is_https {
    // acceptor.accept will block indefinitely if called with an HTTP stream.
    if let Ok(stream) = acceptor.accept(stream) {
        handle_client(stream, &path, reload);
    }
} else {
    handle_client(stream, &path, reload);
}

Splitting Frontend and Backend

The last notable thing about devserver is its organization.

The devserver crate itself is a very thin wrapper around the devserver_lib crate where most of the above described code lives. By splitting the crates this way it becomes trivially easy to embed devserver_lib within another tool. The devserver crate itself just handles the command line arguments to setup and run the server.

So if someone wants to use devserver_lib's functionality they can just add it as a dependency.

In Summary

In the end devserver involved far more wheel reinventing than I initially planned for, but the end result is actually quite robust. I've been using it reliably on and off for months now and have only run into a few small issues (that have been fixed).

I'm super happy that I now have a tiny cross-platform dev web server I can install in seconds. It's the sort of handy tool I know I'll reach for for years to come.

If you need a small development server install devserver with

cargo install devserver.

If you find any issues with devserver (or you're horribly offended by some of its spartan implementations) please open an issue or consider contributing: https://github.com/kettle11/devserver.