Synchronicity kills
I have quite a back log of technical material I want to post here, so let me just start with one of fundamentals of the tech behind notify.me: Asynchronous Programming
Behind the stories of internet service outages is almost always a bottleneck. And usually this bottleneck is due to synchronous access, i.e. some resource is requested and the requestor ties up more resources, waiting for a response. If the requested resource can’t be delivered in a timely manner, more and more requests pile up until the server can’t accept any new ones. Nobody gets what they want and you have an outage.
Now, there’s plenty of efforts of increasing capacity by adding more horsepower and mitigating bottleneck resources by adding caching at various levels of the architecture. These techniques are proven and solve many problems. However, if the data was delivered asynchronously, instead of synchronously, apparent capacity is drastically increased, since resources aren’t tied up waiting. Because HTTP is a synchronous pull protocol and the internet is basically plumbed by HTTP, the first step in to asynchronicity is usually polling. Instead of asking for a resource and then holding the line until the demand is met, ajax (when talking browser as a client) or simply repeated calls (when the client is another service) keep asking “you got my data?” and little bit later “how about now?” and so on. As long as it’s cheap to say no, polling reduces bottlenecks.
Ideally, the conversation should follow a pattern of “tell me when my data is ready”. In the browser scenario this isn’t strictly possible, although techniques such as comet, are repurposing HTTP to do just that even it it does in fact hold the line. However, holding a couple of thousand sockets open from a simple socket server is still a much better option than doing the same against the webserver.
But I digress, because the inner workings of notify.me only touch web pages in small ways. While we will be looking at implementing some version of Comet on the site itself, our greater concern is our message traffic. And here we can easily use asynchronous methods to avoid having resource scarcity turn into resource overload. Breaking synchronous operations into asynchronous operations by separating request and response into separate message passing actions, stops the resource overload. Instead of a system going down from too many parallel requests, it can works its way through a backlog of requests as fast as it can. And in most cases the request/response cycles are so fast that they appear like a linear sequence of events.
In the the next couple of posts, I will go over the three different types of message passing systems we use:
- Store-and-forward message queueing using simpleMQ
- REST calls with REST callback URI registration
- Xmpp message bus using message stanzas as well as IQ based RPC
I’ll discuss each in detail, including previews of our external REST and XMPP APIs, currently in internal testing.
1 year ago