See Python, See Python Go, Go Python Go

Andrey Petrov is the author of urllib3, the creator of Briefmetrics and ssh-chat, and a former Googler and YCombinator alum. He’s here to tell us of a dangerous expedition his requests undertook, which sent them from Python, through the land of C, to a place called Go (and back again).

Today we're going to make a Python library that is actually the Go webserver, for which we can write handlers in Python. It makes Python servers really fast, and—more importantly—it’s a bit fun and experimental. This post is a more detailed overview of my PyCon 2016 talk of the same title. If you'd like to play along at home, this code was written in Go 1.6 and Python 3.5 and the entire complete working thing is open source (MIT license) and and it's available to clone and fork here.

First, a refresher:

Running a Webserver in Go

package main

import (
    "fmt"
    "net/http"
)

func index(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "Hello, world.\n")
}

func main() {
    http.HandleFunc("/", index)
    http.ListenAndServe("127.0.0.1:5000", nil)
}

Running a Webserver in Python

from flask import Flask
app = Flask(__name__)

@app.route('/')
def index():
    return 'Hello, world!\n'

if __name__ == '__main__':
    app.run(host='127.0.0.1', port=5000)

Running a Go Webserver in Python??

from gohttp import route, run

@route('/')
def index(w, req):
    w.write("Hello, world.\n")

if __name__ == '__main__':
    run(host='127.0.0.1', port=5000)

Yo--whaa???

That's right.

Want to give it a try right now? Hit this shiny button:

How?

At first, Go was created to be run as a single statically linked binary process. Later, more execution modes were added to let us compile Go as dynamically linked binaries. In Go 1.5, additional modes were added to allow us to build Go code into shared library that is runnable from other runtimes.

Just like how all kinds of Python modules like lxml use C to run super-optimized code, you can now run Go code just the same. More or less.

Considerations

Before throwing everything away and starting fresh using this newfound power, there are some challenges that need to be considered.

Runtime Overhead

When a Go library is used from another runtime, it spins up the Go runtime in parallel with the caller's runtime (if any). That is, it gets the goroutine threads and the garbage collector and all that other nice stuff that would normally be initialized up when running a Go program on its own.

This is different than calling vanilla C code because technically there is no innate C runtime. There is no default worker pools, no default garbage collector. You might call a C library which has its own equivalents of this, then all bets are off, but in most simple cases you get very little overhead when calling C functions. This is something to consider when calling out to code that requires its own runtime.

Runtime Boundaries

Moving memory (or objects) between runtimes can be tricky and dangerous, especially when garbage collectors are involved. Both Go and Python have their own garbage collector. If you share the same memory pointer between the two runtimes, one garbage collector might decide that it's no longer used and reclaim the memory while the other runtime would be all "WHY'D YOU DO THAT??" and crash. Or worse, it could try to move it around, or change the memory's layout in a way that the runtimes disagree, then we'd get weird hard-to-diagnose heisenbugs.

The safest thing to do is to copy data across boundaries when possible, or treat it as immutable read-only data when it's too big to be copied.

Runtime Demilitarized Zone

When mediating calls between two runtimes like Go and Python, we use C in between them as a kind of demilitarized zone, because C has no runtime and we can trust it to not mess with our data all willy nilly.

The Plan

In this proof-of-concept, we use the Go webserver, but we want to provide a Python handler that will get called when our route gets hit.

1. Make a Go webserver, easy peasy.

2. Make it into a module that is exported into a C shared library.

3. Add a handler registry bridge in C.

4. Add some helpers for calling Go interface functions from C.

5. Add headers for importing the shared library in Python.

6. Write our handler to use the C registry bridge in Python, and the helpers to interact with the data to create a response.

World of Go

Let's explore how to call C from Go and Go from C.

Calling C from Go

Calling C from Go is about as easy as it gets. In fact, we can embed C code in a comment right above an import "C" statement and the Go compiler will build and link it for us.

package main

/*
int the_answer() {
    return 42;
}
*/
import "C"
import "fmt"

func main() {
    r := C.the_answer()
    fmt.Println(r)
}

We use the magic C.* namespace to access anything from the world of C, even if it's not inlined directly above it. We can #include things as we normally would, too, to import code from whichever other C files we would like to use.

Additional reading material:

• https://go.dev/blog/cgo
• https://pkg.go.dev/cmd/cgo
• https://go.dev/wiki/

Calling Go from C

To call Go from C, we'll need to compile our Go code as a shared object and import it. To identify which Go API we want to expose in C, we export it explicitly with an //export ... comment directive.

package main

import "C"

//export TheAnswer
func TheAnswer() C.int {
    return C.int(42)
}

func main() {}

Three things about this code snippet:

1. We need to make sure that the interface for the exported function is properly laden with C types. That means inputs and outputs all need to be C types, and our Go code will cast in and out of them as needed.

2. Our shared object needs to be package main and have an empty main() function. Part of the process for CGO building into a shared library is creating an injection point for spawning the Go runtime.

3. There are many nuances regarding passing memory from Go to C which are not expressed in this basic example. You can learn more on that in the links at the end of the section.

$ go build -buildmode=c-shared -o libanswer.so

This will create a libanswer.so shared object and a corresponding libanswer.h header file that we can reference from our C code.

Now we'll make our C code in a different directory and bring in the libanswer.so and libanswer.h files there.

#include <stdio.h>
#include "libanswer.h"

int main() {
    int r = TheAnswer();
    printf("%d\n", r);
    return 0;
}

$ gcc -o answer main.c -L. -lanswer
$ ./answer
42

Success! We called Go code from C.

More specific reading in the aforementioned links:

• https://pkg.go.dev/cmd/cgo#hdr-C_references_to_Go
• https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers

World of Python

Now onto the Python side of this business. Same idea, so let's look at how to call Python from C and C from Python.

Calling C from Python

There are two approaches to calling C from Python.

One method is using the C Stable ABI which lets us dive in with no additional dependencies. This works by explicitly defining all of the necessary headers and stubs that Python needs to figure out how to call the C code.

The other method is using CFFI, which automatically generates all of the headers and stubs for us. We'll explore the CFFI method for the sake of convenience.

# answer_build.py:
from cffi import FFI
ffi = FFI()

ffi.cdef("int the_answer();")

ffi.set_source("_answer",
"""
    int the_answer() {
        return 42;
    }
""")

if __name__ == "__main__":
    ffi.compile()

Calling the CFFI file will generate the necessary boilerplate for calling this corresponding C code from Python.

$ python answer_build.py
$ ls
_answer.c       _answer.o       _answer.so      answer_build.py

Now to call it from Python, we'll need two more files: answer.py and an empty __init__.py (because Python).

# answer.py:
from _answer import lib

r = lib.the_answer()
print(r)

Here we go:

$ python answer.py
42

Success: we called C from Python!

This was cheating a bit, in the same way we cheated in the Go version because the C was embedded inside of Python code. But, it's not that far from a real world scenario: We could just as easily #include our way into all kinds of external C logic, even if that part is embedded.

There are other ways of doing this too, have a look at the CFFI documentation, but this will do for now.

Putting it together: gohttplib

Alternate title: The Go, The Bad, and the Ugly

The full source code with Python and C examples of gohttplib is available on Github: https://github.com/shazow/gohttplib

We're going to fly through the important bits really fast to get the idea of how it works and how to run it on Heroku.

The Go and C

package main

/*
typedef struct Request_
{
    const char *Method;
    const char *Host;
    const char *URL;
} Request;

typedef unsigned int ResponseWriterPtr;

typedef void FuncPtr(ResponseWriterPtr w, Request *r);

extern void Call_HandleFunc(ResponseWriterPtr w, Request *r, FuncPtr *fn);
*/
import "C"
import (
    "net/http"
    "unsafe"
)

var cpointers = PtrProxy()

//export ListenAndServe
func ListenAndServe(caddr *C.char) {
    addr := C.GoString(caddr)
    http.ListenAndServe(addr, nil)
}

//export HandleFunc
func HandleFunc(cpattern *C.char, cfn *C.FuncPtr) {
    pattern := C.GoString(cpattern)
    http.HandleFunc(pattern, func(w http.ResponseWriter, req *http.Request) {
        // Wrap relevant request fields in a C-friendly datastructure.
        creq := C.Request{
            Method: C.CString(req.Method),
            Host:   C.CString(req.Host),
            URL:    C.CString(req.URL.String()),
        }
        // Convert the ResponseWriter interface instance to an opaque C integer
        // that we can safely pass along.
        wPtr := cpointers.Ref(unsafe.Pointer(&w))
        // Call our C function pointer using our C shim.
        C.Call_HandleFunc(C.ResponseWriterPtr(wPtr), &creq, cfn)
        // Release the ResponseWriter from the registry since we're done with
        // this response.
        cpointers.Free(wPtr)
    })
}

func main() {}

Let's break it down.

We're exporting two functions into the C API:

• ListenAndServe which is used to start our server.
• HandleFunc which is used to register a callback handler for some route pattern.

Note that the exported functions all take in C.* types, because they'll be called from the C side of things.

There is a tricky bit here: The HandleFunc callback function pointer accepts two parameters: http.ResponseWriter and *http.Request. This isn't going to fly for two reasons. The first parameter is a Go interface and both C and Python have no idea what those are. The second parameter is a pointer to an instance which we can't share across the runtime boundary, because that's a big no-no (see the Considerations section).

For the *http.Request data structure, we make a C equivalent (typedef struct Request_ { ... }), populate it by copying the necessary values, and pass it forward instead.

For the http.ResponseWriter interface, we work around it by creating additional exported shims in Go. These shims call the interface's function in Go on behalf of C (more on that in a moment).

There is one more weird trick here: We need to pass some kind of reference to whichever interface instance we're talking about, get that back, and call the original interface without passing any memory pointers across the runtime. How do we do that safely? With our own pointer registry!

There are three operations we care about, each one is just a couple of lines that saves a key in a lookup and the reverse.

type ptrProxy struct {
    sync.Mutex
    count  uint
    lookup map[uint]unsafe.Pointer
}

// Ref registers the given pointer and returns a corresponding id that can be
// used to retrieve it later.
func (p *ptrProxy) Ref(ptr unsafe.Pointer) C.uint { ... }

// Deref takes an id and returns the corresponding pointer if it exists.
func (p *ptrProxy) Deref(id C.uint) (unsafe.Pointer, bool) { ... }

// Free releases a registered pointer by its id.
func (p *ptrProxy) Free(id C.uint) { ... }

Now rather than passing in the entire http.ResponseWriter interface instance, we can register it in our pointer registry and pass along the id which will later come back to us. This is nice and safe because nothing outside of the Go runtime can modify the original memory space, all it can do is hang onto some opaque integer and hand it back later.

Let's take a quick look at our interface shims:

// C interface shim for ResponseWriter.Write([]byte) (int, error)
//export ResponseWriter_Write
func ResponseWriter_Write(wPtr C.uint, cbuf *C.char, length C.int) C.int {
    buf := C.GoBytes(unsafe.Pointer(cbuf), length)

    w, ok := cpointers.Deref(wPtr)
    if !ok {
        return C.EOF
    }

    n, err := (*(*http.ResponseWriter)(w)).Write(buf)
    if err != nil {
        return C.EOF
    }
    return C.int(n)
}

// C interface shim for ResponseWriter.WriteHeader(int)
//export ResponseWriter_WriteHeader
func ResponseWriter_WriteHeader(wPtr C.uint, header C.int) {
    w, ok := cpointers.Deref(wPtr)
    if !ok {
        return
    }
    (*(*http.ResponseWriter)(w)).WriteHeader(int(header))
}

Now, given our proxied pointer ID in C, we'll be able to call Write and WriteHeader on the underlying ResponseWriter that continues to live in Go. We could expand this to cover the full interface but, for the sake of our prototype, these two will do.

The Python

Let's skip the CFFI boilerplate we've seen before and dive right into our C-to-Python wrapper:

import os
from ._gohttplib import ffi

lib = ffi.dlopen(os.path.join(os.path.dirname(__file__), "libgohttp.so"))


class ResponseWriter:
    def __init__(self, w):
        self._w = w

    def write(self, body):
        n = lib.ResponseWriter_Write(self._w, body, len(body))
        if n != len(body):
            raise IOError("Failed to write to ResponseWriter.")

    def set_status(self, code):
        lib.ResponseWriter_WriteHeader(self._w, code)


class Request:
    def __init__(self, req):
        self._req = req

    @property
    def method(self):
        return ffi.string(self._req.Method)

    @property
    def host(self):
        return ffi.string(self._req.Host)

    @property
    def url(self):
        return ffi.string(self._req.URL)

    def __repr__(self):
        return "{self.method} {self.url}".format(self=self)


def route(pattern, fn=None):
    """
    Can be used as a decorator.

    :param pattern:
        Address pattern to match against.

    :param fn:
        Handler to call when pattern is matched. Handler is given a
        ResponseWriter and Request object.
    """
    def wrapped(fn):
        @ffi.callback("void(ResponseWriter*, Request*)")
        def handler(w, req):
            fn(ResponseWriter(w), Request(req))

        lib.HandleFunc(pattern, handler)

    if fn:
        return wrapped(fn)

    return wrapped


def run(host='127.0.0.1', port=5000):
    bind = '{}:{}'.format(host or '', port)
    print(" * Running on http://{}/".format(bind))
    lib.ListenAndServe(bind)

Just like we made C-friendly wrappers around Go's ResponseWriter and Request, here we're making Python-friendly wrappers around C's Request_ struct and ResponseWriter_* interface shims we made. All of this is just to hide CFFI's warts and make the exposed API somewhat idiomatic.

The final trick is converting our Python callback handler function into a C function pointer that can be passed back to Go. Luckily, CFFI has a convenient @ffi.callback(...) decorator which does all the nasty work for us.

Off to the races we go!

lolbenchmarks

It's fun to take a look at the performance characteristics of this kind of approach. Yes, yes, of course, this isn't Production Ready or anything, but for the sake of some laughs:

Conditions: ab doing 10,000 requests with 10 concurrency on my 💻. If you want to see how it stands up on Heroku dynos, just hit the deploy button and see how it works for you:

These are all basic "Hello, world\n" handlers. The first one is straight-up Go, then it's Go-to-C, then it's Go-to-C-to-Python (gohttp-python). It does pretty well.

Keep in mind that this is with 10 concurrent requests, so werkzeug-flask probably chokes more on the concurrency than the response time being slow.

Again, this is for-fun off-the-cuff unscientific lolbenchmarks.

What's left?

We've discussed 80% of what's involved, but the remaining 80% is still available as an exercise for the reader (or maybe a future blog post):

• The gohttp Python dependency comes pre-published to PyPI for your convenience, but you'll need build and distribute the dependency yourself if you want to tweak it further.
• Play whack-a-mole with memory leaks. The current prototype is not safe or battle-tested by any means. Any time a C variable gets declared, we'll need to free it.
• Implement the rest of the interfaces that we need. Right now there are only a couple of functions available but there is much more to build a full server. Pull requests welcome!

Handy links

• https://shazow.net/ is where you can see more fun experiments by me, and @shazow is me on Twitter.

• https://github.com/shazow/gohttplib/ is the source code for this project.

More posts on this topic:

• https://words.filippo.io/building-python-modules-with-go-1-5/

• https://feisky.xyz/posts/2016-04-19-cgo-in-go-1-6/

Automagic code generating:

• https://github.com/sbinet/go-python

• https://github.com/go-python/gopy