In my previous post I’ve introduced opium in a beginner friendly way, while in this post I’ll try to show something that’s a little more interesting to experienced OCaml programmers or those are well versed with protocols such as Rack, WSGI, Ring, from Ruby, Python, Clojure respectively.
First, let’s start with some history. I’ll be recounting from memory so I
apologize for any inaccuracies. Back in the stone ages of web application
development, a big problem was (still is) creating reusable stand alone
components. For example, caching, authentication, static pages. One solution to
this problem was invented by the python community, (WSGI) and popularized by
the Ruby community (Rack). Since then, it caught on like wild fire and has been
ported to almost every other langauge. I attribute Rack’s success to its
extreme simplicity. In Rack, an application is an object that takes an
environment, an all-encompassing dictionary that includes the http request
among other things, usually called
env and returns a tuple of three elements,
the status code, the headers, and the body of the response. Translating this
to OCaml, a rack application is just:
1 2 3 4 5 6 7 8 9 10
Accepting Rack’s proposition that applications are simply functions that return that 3 element tuple, how do we create reusable components? In Rack the solution was to create so called “middleware”. Middleware is an object with a call method (of type application) and is constructed by passing the next application down the middleware chain. A very literal translation of a typical rack middleware to OCaml gives:
1 2 3 4 5 6 7
One can imagine chaining multiple middleware to provide various functionality. For example checking credentials in the header of the request (found in env) to decide whether to authenticate the user to proceed with the next step or to return a not authorized status.
Middleware in Opium
Of course, nobody sane writes code like that in OCaml. So if we get rid of the classes and store state using closures instead of instance variables (if necessary) we’d get just a function of type:
1 2 3
In this viewpoint, middleware is simply a higher order function that transforms an application to another.
OK, so is this good enough for OCaml? No. Putting back my statically typed functional programming hat on I can poke a couple of holes in this approach.
env is mutable. In fact, middleware is encouraged to treat it as request local storage and pass information between middleware. This means that env may not be the same before and after calling a middleware.
env offers no encapsulation. Middleware can easily pry at each other’s internals. Sometimes this is necessary, but many times it’s not. Ring in clojure offers namespaced keywords as the keys to the env hash, but this is only a gentleman’s agreement.
env offers no type safety. In rack, doing env[‘xxx’] is like trying to pull a rabbit out of hat. There’s no guarantee that the value obtained will of a certain type.
All of this points us towards using something other than an untyped hash table
for the environment hash. But what do we use in OCaml if we want, an openly
extensible, heteregoneus, immutable map? We use core’s
Univ_map. I won’t go
into the details of how it works but I’ll say that a univ map supports the
following two operations:
In addition to the creation of new
Univ_map.Key.t that are associated
with the types a key would extract.
If you’d like to know more I recommend the following resource , , .
In Opium, we throw away Rack’s env and simply put everything under the
same umbrella and call it a Request. Similarly, an opium response will
subsume the 3 element response tuple.
env will then be the extensible
part of a request/response. Which gives us:
1 2 3 4 5 6 7 8 9 10 11
Now middleware is able to:
store stuff in env and always know the type of a value pulled out of env.
middleware can encapsulate its private data by not exposing its
envkeys in the public interface
And of course, there’s no sight of side effects anywhere.
Enough theory crafting, let’s build some middleware. Let’s start with a trivial
example. First of all, our middleware need not even use
env at all. For
example here’s a middleware that uppercases the body:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
As you can tell, a middleware knows of 2 bits of information:
handler : Request.t -> Response.t Deferred.t. This is the application request handler.
req : Request.t. The current request.
In our example, the middleware runs the handler and returns a response with the uppercased body. But of course a general middleware doesn’t have to run handler at all, it can change the request before feeding the handler, or it can simple add a logging message and let the handler proceeed with the request.
You can tell that middleware is flexible, but to make it do something more
interesting, you must be able to store stuff along the request/response. As
I’ve mentioned before, the
env bag is the perfect place for that.
Here’s another common use case. Suppose we’d like our webapp to automatically authenticate users that provide their credentials using the HTTP Basic scheme. For example, we can export a export function like:
Here’s how we could implement that:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
The middleware above might be basic as well but it should give you an idea on how to create a wide variety of middleware. For example, serving static pages, caching, throttling, routing, etc.
There are of course downsides however that I’ve yet to solve in Opium. The main downside is that middleware does not commute. This means that it must be executed in serial for every request. Much more worse however is that middleware order will affect application behaviour. In fact, a common source of bugs in Rack is executing middleware in the wrong order. One obvious solution to this is to explictly specify dependencies between middleware. Unfortunately it’s pretty ugly and heavyweight and it makes middleware less composable.
Final Note: After writing the first version of Opium I realized I was copying the core of twitter’s finnagle/finatra. I don’t mention them here however since finnagle is a lot more general and I don’t know much about it other than this excellent paper. I did end up borrowing their terminology for the lower level plumbing though.
 Rack Spec.
 Ring Spec. A huge improvement of Rack in my opinion. Rock could be thought of as a typed Ring.
 If you’re coming from the haskell WAI world I believe
env would be
vault. In fact, the way Rock is designed should be very
similar to the old WAI (< 3.0). Except that WAI use IO instead of Deferred.t.
 mixtbl An implementation without core. It’s also a hashtable instead of a map.
 Haskell Vault. Same concept but implemented in the Haskell world.
 Univeral Type. If you’d like to dig deeper to see how the basic for such a map can be implemented.