[From the last episode: We looked at neuromorphicThis refers to systems that attempt to operate in the same way that the brain operates. Spiking neural networks are the main commercial example. machine learningMachine learning (or ML) is a process by which machines can be trained to perform tasks that required humans before. It's based on analysis of lots of data, and it might affect how some IoT devices work., which is intended to act more like the brain does.]
Our last topic to cover on machine learning (ML) is about trainingWith machine learning, a model must be trained in order to perform its function (which is called inference).. We talked about supervised learning, which means we’re training a modelA simplified representation of something real. We can create models of things in our heads without even realizing we're doing it. Technology often involves models because they let us simplify what would otherwise be extremely detailed, complicated concepts by focusing only on essential elements. based on a bunch of samples. That may sound simple enough, but here’s a question: how do you tell the training algorithm what’s in the sample? That’s done through what’s called labeling.
We can talk about machine visionTechnology that gives machines the ability to see things and make decisions based on what they see., since that’s the most prevalent type of ML, but the topic applies to all kinds of learning – natural language, reading handwriting, even dealing with obscure things like protein folding.
Gathering Pictures
So let’s talk about pictures to start with. You have a bunch of pictures to use for training. How did you get all of those pictures? Well, if you’re lucky and you’re a Google or Facebook, you have lots of users that happily turned over pictures for free.
You know those “challenges” on Facebook, like when someone says, “Hey, post a picture of yourself 10 years ago and today! And… go!” You know what that’s about? That helps Facebook train their aging algorithmsA way of calculating something, typically as a step-by-step recipe. A familiar one to all of us is how we do long division on paper: that's an algorithm. Algorithms abound in the IoT. so that they can look at a picture and imagine what someone looked like 10 years ago – or will look like in 10 years. Yeah, if you played that game, you played right into that.
So that’s challenge one: simply coming up with the samples.
Labeling Pictures
But here’s the really hard part. Let’s say you’re trying to identify cats. (I know, can’t we be done with cat pictures already???) On one level, you can simply plan for a system to have a thumbs-up/down decision: does the image contain a cat or not?
That’s easy to label: with each picture, you attach a “yes” or “no.” Problem solved. But, of course, that’s not usually what we’re doing.
In reality, you’re probably going to want to have the cat identified in the picture – amongst a bunch of other things that might be in the picture. What might happen there is that the machine vision system might place a so-called “bounding box” around the cat. That’s a box that goes around all of the parts of the cat that are visible in the picture. Obviously, since it’s a box (and a cat isn’t shaped like a box – unless it’s sitting in one that’s too small), it will also contain things that aren’t the cat.
Or perhaps you’re going to color the cat some color to set if off from the rest of the image. In this case, you could color only the cat and nothing else.
What’s literally needed here is to go into the picture and, by some means, identify every pixel that belongs to the cat. Yeah, that’s a lot of work.
Labeling Video
Let’s make the problem harder: let’s use a video instead of a still image. A video is essentially a sequence of still images (called frames) that move so fast that your eye perceives them as continuous. Now, if you want to identify a cat in a video, you need to label every frame. Every. Dang. One.
I heard an estimate within the last couple years that it takes (or took) as much as 90 minutes per frame to label videos. Because, in reality, you’re not just looking for cats. If you want to use a video for multiple purposes, you want to label everything in there.
The same idea holds for lots of other non-visual samples. You may not be labeling pixels, but you’re labeling something, and it may well be tedious.
Tools for Labeling
Things have gotten a bit easier with tools that help with labeling. For instance, in a video, if a car is moving down the street and the camera is still (that is, not panning as the car moves), then everything in the frame is going to stay the same (because it’s background) except for the car. So, while you might need to label the first frame carefully, a tool might help you label the second frame by looking only at what changed.
That’s overly simplistic, but it’s an example of how tools can make the job easier.
Creating Videos
There’s yet another approach, and it’s being offered up for use in the kinds of cameras that we might find in self-driving cars in the future. Such cameras have to be able to identify and respond to all manner of things – traffic lights, speed limits, lane markings, bicycles, other cars, buildings, sidewalks, balls and children and puppies running in front of the car, other pedestrians… You get the idea.
You also need this to work during daylight and at night and at dusk whether it’s sunny or raining or… or… or… And pedestrians may come with different skin tones and clothing colors and with or without hats and… and… and…
If you want to train such a system, imagine the number of different videos you’d have to come up with – and then label all the variants.
An alternative is to have a system that creates these videos. You build a background, and then you add the different moving pieces, and then you create scenarios. Going through this intersection. Turning left instead. Turning right. And then you apply lots of variations: change colors of things, change time of day, change weather. And “automatically” create all of the training videos. (Automatically is in quotes because it’s still a lot of work – just not as much.)
As long as the images being created by such a system are realistic-looking enough to provide high training accuracy (when a car door opens in the video, can you see the details of what’s inside the car?), you have a way to get all of those different permutations and combinations that are needed to cover all the possibilities.
Labeled Samples are Gold!
So data labeling is an important thing, and companies that have vast stores of labeled data have something of great value. It’s why the Googles and Facebooks are so far ahead in the AIA broad term for technology that acts more human-like than a typical machine, especially when it comes to "thinking." Machine learning is one approach to AI. game. We’ve been feeding them samples for, well, decades now.
Of course, this is also a reason why unsupervised learningA machine learning approach where a machine is trained by learning as it goes. It doesn't get trained by examples ahead of time, and isn't explicitly taught the "right answers." There must be some other way of reinforcing the right and wrong answers as it learns. (or semi-supervised learning) can be attractive: You need no (or fewer) labeled samples. There are other challenges, and you can’t necessarily substitute one for the other, but not having to label things would be really nice.
ML systemsThis is a very generic term for any collection of components that, all together, can do something. Systems can be built from subsystems. Examples are your cell phone; your computer; the radio in your car; anything that seems like a "whole." – especially those in the cloudA generic phrase referring to large numbers of computers located somewhere far away and accessed over the internet. For the IoT, computing may be local, done in the same system or building, or in the cloud, with data shipped up to the cloud and then the result shipped back down. – burn a lot of energy. So, as we leave the topic of ML, we’re going to explore a new one: how do we save energy in electronics? It’s a big focus these days, especially since so many IoTThe Internet of Things. A broad term covering many different applications where "things" are interconnected through the internet. devices are battery-powered. So we’ll explore the causes and fixes of high powerThe rate of energy consumption. For electricity, it’s measured in watts (W)..
Leave a Reply