[From the last episode: We looked at the request/responseOne way machines can communicate, where one machine sends a request (or issues a command) to another machine; that other machine responds with data or maybe with an acknowledgment to a command. way of handling machineIn our context, a machine is anything that isn't human (or living). That includes electronic equipment like computers and phones. communications.]

As we wind down our discussion on communications, there are a few straggling issues that also cross over into the computing area. Today we look at the notion of load balancingA way of handling heavy server traffic by having multiple servers that can do exactly the same thing and then using another server as a dispatcher to assign tasks to the least busy servers..

This is a topic that most likely applies to the cloudA generic phrase referring to large numbers of computers located somewhere far away and accessed over the internet. For the IoT, computing may be local, done in the same system or building, or in the cloud, with data shipped up to the cloud and then the result shipped back down., where there are lots of serversA computer with a dedicated purpose. Older familiar examples are print servers (a computer that controls local printing) and file servers (a computer used for storing files centrally). More modern examples are web servers (the computers that handle your web requests when you use your browser) or application servers (computers dedicated to handling the computing needs of a specific application). Servers are often powerful, expensive machines since they have to handle a heavy load. that have to handle tasks from lots of IoTThe Internet of Things. A broad term covering many different applications where "things"  are interconnected through the internet. devices that are connected to the cloud. But it’s probably easier to discuss in another familiar context: web browsing. The concepts hold equally well for both IoT and the web.

Browsing Means Requesting Pages

Let’s take an example scenario: looking up a recipe online on your favorite recipe site. First you go to the site, which will generally land you on the home page. Then perhaps you search the site for a particular recipe. Then, from the results, you click on one of the recipes.

Here’s what’s happening: first, your browser sends a request for the home page out into the internet. That request comes in the form of one or more packets, and those packets contain the IP addressAn address used by the Internet Protocol (IP) to identify a machine on the network. of the website you want.

Using the req/rep way of communicating, the request ends up at the server hosting that web page, having been routed through the internet to get there. That server looks at the request, fetches the contents of the requested page, and sends it back to you.

When you then start the recipe search, you’re initiating another request: please search the site for your recipe. The server gets the request, performs the search, and sends back the search results.

Finally, when you click on one of the recipes, your browser sends yet another request for that recipe page; the server gets and fulfills that request, sending the recipe page back to you.

So that’s all well and good. But what happens if you and a few million of your closest friends try to do the same thing at the same time? That’s a lot of requests coming into one poor server; how’s it supposed to keep up?

Dispatching the Requests

In fact, it can’t. You may have heard of a hackingThis can mean a couple things. A quick-and-dirty (but not elegant) trick to get something done is a hack. A computer security break-in is also a hack (because inelegant tricks are used to break in). It can be a noun or a verb ("he hacked my computer"). attack called a distributed denial of service, or DDoS attack. This is where some attacker arranges for a bunch of… computers or something… to bombard a site with requests – way more than the site can handle. And so the site goes down.

So, one extreme is a single server that can do all the work; the other extreme is a site that’s overloaded and quits. Is there a happy medium? Why yes, of course there is. (Or else I wouldn’t have asked the question.)

The trick is to create a bunch of servers that can all do the same thing. Maybe 5, 10, 500 servers – all acting just like that one first server that could serve up the pages you requested. Your browser’s requests could go to any of those servers, and the result would be the same. But how to pick the server? After all, each server has a different address, and your browser (or the address lookup functions on the internet) don’t have multiple addresses for one website.

So what they do is to add one more server, and it has one job: get all the incoming requests and then send them to the least busy server at the time. This dispatching server is taking the huge request load and keeping all the servers busy – balancing the load. And so we call this function load balancing.

This means that those three requests you send – home page, search, and specific recipe – could well end up with three different servers in the end.

No History Please

It also means something really important: if you’re doing this, you can’t require that a server know what you’ve already done. Returning one page can’t depend on which pages were returned before. If that history were necessary, then it would create all kinds of complication.

Let’s imagine that it did have to keep up with the history. More technically put, let’s imagine that these servers have to maintain a notion of the stateA broad term that can apply to a lot of systems. Let’s say you have a system that can be one of two ways – on or off. That “on-ness” or “off-ness” is the state of the system. The system is either in an on state or an off state. Many different kinds of systems can have many different kinds of state. There are many complex branches of engineering or physics or even mathematics that deal with state. The easiest way to think of any of them is, “the way things are” (which may depend on the history of prior states, meaning how they got to this particular state). of the systemThis is a very generic term for any collection of components that, all together, can do something. Systems can be built from subsystems. Examples are your cell phone; your computer; the radio in your car; anything that seems like a "whole.", and history determines the state. Then, when one server returned a page, it would have to let all the other servers know so that, if another request came in that happened to get dispatched to a different server, that that new server could know the history. You can probably imagine, with all the browsing going on, that this would completely bog things down.

There actually is a way that your browser can keep track of such things: that’s what cookies are about. They keep some aspect of your history with a website, or perhaps settings you requested, or something. That information stays on your computer, not in the servers, so it can be provided to the servers if necessary.

Same with Cloud-Connected Devices

So that’s fine for web browsing, but what about the IoT? Well, if your refrigerator connects to the cloud, then lots of other refrigerators may also connect to the cloud. And too many might overwhelm a single server. After all, if you’re sending up a bunch of data, then that might take time, and some other devices might be locked out of connecting during that time.

So a load-balancing function could be useful here too, passing device connectionsThis refers to some kind of electrical connection. It might be through a network cable, a cable connection, a wireless connection, or a phone - just to name some options. The connection might be to the internet or to some other local device. from server to server so that everyone receives attention.

This leads to another important concept for the IoT – one that makes sure that requests are handled correct regardless of which server or machine handles it and whether or not the packets come through cleanly. We’ll talk about this next.