The Two Main Machine-Learning Functions

[From the last episode: We looked at artificial neural networksA type of neural network that’s loosely inspired by biological neurons, but operates very differently. as a practical solution that’s come available ahead of neuromorphicThis refers to systems that attempt to operate in the same way that the brain operates. Spiking neural networks are the main commercial example. networksA collection of items like computers, printers, phones, and other electronic items that are connected together by switches and routers. A network allows the connected devices to talk to each other electronically. The internet is an example of an extremely large network. Your home network, if you have one, is an example of a small local network..]

We talk about AIA broad term for technology that acts more human-like than a typical machine, especially when it comes to "thinking." Machine learning is one approach to AI. as allowing machinesIn our context, a machine is anything that isn't human (or living). That includes electronic equipment like computers and phones. to make decisions. While that might be true, it’s an imprecise description of what goes on. There are two main jobs that a neural networkA type of conceptual network organized in a manner inspired by our evolving understanding of how the brain works. has: to learn and then to apply that learning. We call the first one trainingWith machine learning, a model must be trained in order to perform its function (which is called inference). and the second one inferenceFor machine learning, this refers to the process of making decisions using a model. Before the decisions can be made accurately, training must occur., since the network infers some fact based on a combination of what it has learned and what it’s faced with now.

Training

There are several approaches to training, most of which are still in the research stage. Broadly, we talk about supervised and unsupervised training.

With supervised training, we show the network a large number of samples. It starts with some weights in the network – maybe randomly selected – as a starting point. The network gets the first sample and makes a decision (which will probably be wrong). If it’s wrong, then there’s this backwards process that more or less amounts to, “OK, that answer was wrong. What would we need to change in the last set of weights to get it right?” From there, you go to the prior layer to see what needs to change there to give the last layer what it needs. And so forth up to the first layer. In the industry, they call this back-propagation.

This happens over many samples, and, hopefully, in the end, you converge on a stable model. Not converging might mean that the categories you’re trying to teach the network to understand overlap too much. To use a ridiculous example, if you show it a picture of a cat-looking dog and tell it that it’s a dog, it will adjust the model accordingly. But if you then show it a picture of a cat that looks like that cat-looking dog, it might then try to undo some of those changes, since it now got it wrong.

Supervised Learning Is a Lot of Work!

The tough thing about supervised learningA machine learning approach where a machine is trained by giving it examples and, during the training, telling the machine what the answers are. A large number of varied examples provide the best results. is that you have to provide thousands or more of samples in order to cover all the variations in whatever you’re trying to do. If it’s photos of people, you need to have examples of all races and genders and haircuts and makeup and lighting and every other possible variation. If you don’t, then the network will be biased, as we noted last week.

The other thing about those samples is that someone has to label them. In other words, they have to come with the right answer. And that may not be simple: for a video camera in a self-driving car, for example, it has to be able to identify a large number of items – driving guidelines (like the center line or speed limit), other vehicles, or obstacles. So, for each frame in the video, someone has to note – by specific pixel locations – where various objects in the frame are located.

Training is also a finicky business. If you add or subtract one sample from the training set, the model will change (albeit slightly). If you simply change the order of the samples, you’ll also change the model. What that says is that there are many, many different variations on a model that will give adequate results. You do have to test the modelsA simplified representation of something real. We can create models of things in our heads without even realizing we're doing it. Technology often involves models because they let us simplify what would otherwise be extremely detailed, complicated concepts by focusing only on essential elements., however, to make sure you’ve done enough training. That means setting aside a part of the samples that you have to use for testing. Say, use 80% of the samples for training, and the remaining to test how well it has learned.

We’ve Donated to the Cause

You can also see why Facebook and Google are so far ahead in this game. They gave us an opportunity to give them enormous numbers of photos and videos and other artifacts, and we heartily heeded the call, giving them everything. That has helped them improve their accuracy far more than is possible for other companies that don’t have access to the same resources. You know those challenges where they ask for a picture of you 10 years ago and today? Yup, that’s simply helping them to train their aging algorithmsA way of calculating something, typically as a step-by-step recipe. A familiar one to all of us is how we do long division on paper: that's an algorithm. Algorithms abound in the IoT..

Unsupervised learningA machine learning approach where a machine is trained by learning as it goes. It doesn't get trained by examples ahead of time, and isn't explicitly taught the "right answers." There must be some other way of reinforcing the right and wrong answers as it learns. is not so straightforward – and is not in volume production today. In that case, a systemThis is a very generic term for any collection of components that, all together, can do something. Systems can be built from subsystems. Examples are your cell phone; your computer; the radio in your car; anything that seems like a "whole." doesn’t get a pre-training session. It learns as it goes – call it on-the-job training. We’ll come back to this as it evolves.

Inference

While training requires that signals first flow forward and then backward through the network, inference requires only a forward pass (loosely speaking). We’re going to dive more into some of the details of inference over the next few weeks.

Both training and inference can be extremely compute-intensive – training in particular. So they have mostly taken place in the CloudA generic phrase referring to large numbers of computers located somewhere far away and accessed over the internet. For the IoT, computing may be local, done in the same system or building, or in the cloud, with data shipped up to the cloud and then the result shipped back down.. That’s fine for training, but it can be a problem for inference. If the Cloud is the only place we can make a decision, then we have to send all the data to the Cloud for that decision. That has several problems:

• The cloud can be expensive to use.
• You have to send all of that information – that’s lots of data on the wire.
• There are privacyRefers to whether or not information gathered about your usage of IoT devices by authorized people can be made public, or shared with others, without your consent. Different from (although related to) security, which protects such data and devices from access by unauthorized people. Different from privacy, which is more concerned about use of data by authorized people. concerns if all data goes into the cloud rather than staying local.
• It takes longer for decisions that happen in the cloud. If you push a button to unlock your door, you’ll get a much faster response if that decision is made locally.

For these reasons, there are huge investments being made in ways to bring inference “to the edgeThis term is slightly confusing because, in different contexts, in means slightly different things. In general, when talking about networks, it refers to that part of the network where devices (computers, printers, etc.) are connected. That's in contrast to the core, which is the middle of the network where lots of traffic gets moved around.” – that is, to have the IoTThe Internet of Things. A broad term covering many different applications where "things"  are interconnected through the internet. device (or whatever) make the decision rather than asking the Cloud to make it. The Cloud serversA computer with a dedicated purpose. Older familiar examples are print servers (a computer that controls local printing) and file servers (a computer used for storing files centrally). More modern examples are web servers (the computers that handle your web requests when you use your browser) or application servers (computers dedicated to handling the computing needs of a specific application). Servers are often powerful, expensive machines since they have to handle a heavy load. chew up lots of powerThe rate of energy consumption. For electricity, it’s measured in watts (W). for inference – something that a battery-powered device couldn’t handle. So those developers have a big challenge in figuring out how to do inference at the edge with lower power but still with adequate accuracy.

Next we’ll start looking at the most popular kind of inference: vision-oriented, and the types of network that are better and worse for it.