Computing at the Edge and in the Cloud

[From the last episode: We saw how we can combine computing components together on a single chip.]

We’ve looked at a number of basic computing notions, and we’ve seen how, more or less, these come together in computing systemsThis is a very generic term for any collection of components that, all together, can do something. Systems can be built from subsystems. Examples are your cell phone; your computer; the radio in your car; anything that seems like a "whole.". Now let’s put them into the context of IoTThe Internet of Things. A broad term covering many different applications where "things"  are interconnected through the internet. systems. Where are these things used, and what’s used where?

If we take a complete end-to-end look at an IoT system, we know that we’ve got:

• The IoT gadgets that you and I would use;
• The cloudA generic phrase referring to large numbers of computers located somewhere far away and accessed over the internet. For the IoT, computing may be local, done in the same system or building, or in the cloud, with data shipped up to the cloud and then the result shipped back down. computing (as well as the fog); and
• The communication stuff that gets us to and from the cloud and other IoT devices.

Ubiquitous Computing

All three of those things include some kind of computing.

• The IoT devices need computing to operate all the local sensorsA device that can measure something about its environment. Examples are movement, light, color, moisture, pressure, and many more. and actuatorsA way of controlling some device electronically. It might turn the device on or off or change a setting or property or do any other thing that the device is capable of.. They also participate in the communications, since we saw that some parts of the communication stack are done in softwareIn this context, "software" refers to functions in an IoT device that are implemented by running instructions through some kind of processor. It's distinct from "hardware," where functions are built into a silicon chip or some other component.. There may also be some computing involved in the securityRefers to whether or not IoT devices or data are protected from unauthorized viewers. aspects of the device. All of that said, it’s not necessarily a big computing burden.
• The cloud has the potential to do massive amounts of computing – way more than any device could do. That’s part of the reason for the cloud: what the device can’t do locally it can “out-source” to the cloud. Searching through massive databasesA structured way of storing data and relating different pieces of data to each other. (Like, which address belongs to which person.) There are “query” languages, the best-known of which is SQL, that let you enter data into the database, change data that’s already in the database, and retrieve data from the database., doing internet searches, running algorithmsA way of calculating something, typically as a step-by-step recipe. A familiar one to all of us is how we do long division on paper: that's an algorithm. Algorithms abound in the IoT. – these are all things that the cloud may have to do. So the cloud needs monster resources for computing.
• The fogRefers to some way of doing "cloud"-style computing without having to use the cloud. A local server (or router or something else) can act like the cloud; because it's local, it's referred to informally as the fog. also has to do way more computing than an IoT device. After all, it’s kind of like a mini-cloud capability that’s local instead of who-knows-where. It may have to do some tough computing, but at least it has to deal with only one application or one company, where it’s located. The cloud has to be able to handle all kinds of stuff from many applications and users, which the fog doesn’t have to do. So the fog can make do with far fewer resources than the cloud would need.
• Communication involves, as we’ve seen, routersAn electronic box that helps steer data on a network. For instance, you may have one in your home connecting your phone and computer and other devices to each other and to the internet. The data itself has information about where it's being sent; the router uses that information to send it in the right direction. At a really basic level, you can think of a router and a switch as being the same thing. If you want to get more technical, a switch creates a local subnetwork, and the router connects multiple subnetworks (or multiple networks). and switchesA switch helps direct network traffic to the right destination. At a high level, it's very similar to a router. Technically, switches are used to create local subnetworks; routers connect subnetworks together. that get our messages where they need to go as well as serversA computer with a dedicated purpose. Older familiar examples are print servers (a computer that controls local printing) and file servers (a computer used for storing files centrally). More modern examples are web servers (the computers that handle your web requests when you use your browser) or application servers (computers dedicated to handling the computing needs of a specific application). Servers are often powerful, expensive machines since they have to handle a heavy load. at every hop along the route that a message takes. All of those devices do computing as they look at messages and decide what to do with them and where to send them next. That can actually be a lot of computing, but it’s super specialized, and there are chipsAn electronic device made on a piece of silicon. These days, it could also involve a mechanical chip, but, to the outside world, everything looks electronic. The chip is usually in some kind of package; that package might contain multiple chips. "Integrated circuit," and "IC" mean the same thing, but refer only to electronic chips, not mechanical chips. dedicated to handling networkA collection of items like computers, printers, phones, and other electronic items that are connected together by switches and routers. A network allows the connected devices to talk to each other electronically. The internet is an example of an extremely large network. Your home network, if you have one, is an example of a small local network. trafficRefers to any kind of electronic message -- email, web request, streaming video, or anything else -- that travels over a network..

Who Uses What?

So… do these things use microcontrollers (MCUs), systems-on-chip (SoCs), or something else entirely? Let’s start from the least needs (ignoring communications for a sec).

• IoT devices don’t need much, and the computing doesn’t have to be super-fast. In addition, this is an area where new upstart companies may try to jump into the market, and they typically won’t have the budget needed to design a whole new custom computing SoC. Then again, most of them don’t need that. So you’re much more likely to find MCUs in these devices. Maybe very small ones.
• The fog has more work to do, so MCUs probably won’t be beefy enough. But such servers aren’t necessarily going to be shipped in huge quantities, so, here again, doing an SoC probably isn’t going to pay off. So if they’re not going to use MCUs or SoCs, then what? This will more likely be traditional computer boards, with CPUs and memory as separate chips – just like your laptop or desktop machine.
• The cloud, because it needs so much, has a couple of options. It can simply use many, many powerful desktop-like machinesIn our context, a machine is anything that isn't human (or living). That includes electronic equipment like computers and phones.. Those use top-of-the-line CPUs and tons of fast memory – all on separate chips. But for parts of the cloud – in particular, the thing they refer to as “acceleratorsA circuit, usually done with hardware instead of software, that can perform some task faster than a processor can. Instead of the processor handling the task itself, it offloads the work to the accelerator, and the accelerator sends the answer (whatever that means for the task) back to the processor.,” SoCs may make sense. Accelerators take particularly common, thorny problems and use specialized, dedicated circuits that do that thing very well (but not much else). So they “accelerate” those particular functions. There aren’t a lot of companies building big cloud facilities, and the ones that are are ginormous. So they can afford to do an SoC. Some other companies design accelerators for cloud companies, and they’re also likely to design SoCs that compete well.
• Communications boxes often involve SoCs that combine some typical CPUStands for "central processing unit." Basically, it's a microprocessor - the main one in the computer. Things get a bit more complicated because, these days, there may be more than one microprocessor. But you can safely think of all of them together as the CPU. with super-fast specialized circuits for processing network traffic. They’re not designed specifically for the IoT, but they get involved with all network traffic – internet browsing, cell-phone calls, anything – so they can afford to design SoCs.

There’s one particular area that we’ll talk about later that mostly happens in the cloud now, but is making a move towards devices: artificial intelligenceA broad term for technology that acts more human-like than a typical machine, especially when it comes to "thinking." Machine learning is one approach to AI. (AI). This is a whole different area, and we’ll spend some time on it in the future. It’s really interesting, but it departs significantly from what we’ve seen so far.