[From the last episode: We looked at the reason for doing all the multiply-accumulate math with machine learningMachine learning (or ML) is a process by which machines can be trained to perform tasks that required humans before. It's based on analysis of lots of data, and it might affect how some IoT devices work..]
Before we leave this matrix multiplication thing, there’s one other notion floating about a lot these days. Anyone listening to conversations about how to make machine-learning (ML) algorithmsA way of calculating something, typically as a step-by-step recipe. A familiar one to all of us is how we do long division on paper: that's an algorithm. Algorithms abound in the IoT. faster will eventually hear the word sparsity. What the heck is that about?
All that Matrix Math
You’ve seen what matricesAn array of numbers, having both rows and columns of numbers. Widely used for many complex tasks. look like, and you’ve seen how we multiply them. It’s a fiddly business, with lots of little calculations. While each calculation is small, they all add up to taking lots of time. So the fewer of those things you have to do, the better.
With the matrices we looked at, most of the “entries” in the matrix were some number. We used simple numbers, but they might be numbers with decimals; some might be huge; some might be tiny; some might be positive numbers; some might be negative numbers. For those of you who like the deeper math, some might even have a relationship to complex numbers.
Sparse Matrices
But, in the world of engineering, you learn early on about the notion of a sparse matrix: that’s a matrix where many – most, in fact – of the entries are simply 0. For many engineering problems, this allows you to play tricks with the math to make it simpler, since there may be some structure to where the zeros are and are not.
With machine learning, it’s not so much about structure; it’s simply about not wanting to do any more math than necessary. If an entry in a matrix is 0, then you know that the product between it and any other number from any other matrix is going to be 0, and it’s not going to add anything to the accumulation you’re doing.
So, in a super-simple (but inefficient) approach, you would just blindly do all the multiplications and you find – surprise! – that the product is 0 and the sum didn’t change because you added 0. So that multiply-accumulate operation was a total waste of time. And yet you knew the answer when you saw the 0. So why waste time on going through the math? Just ignore it and move on to the next one.
So the first optimization was for the computing engines to notice when an entry is zero and simply ignore it. That takes some extra circuitry, since you have to test whether it’s zero rather than just running the computation. But that extra circuitry buys you some nice performance, so it’s worth it.
Making Matrices Sparser
Once you have that, now you can actually go and start trying to create more zeros in the matrix. Is a number super small? Then it’s not going to contribute much of anything to the accumulation (assuming most of the other numbers are large). So why not just zero out that number – and now you don’t have to do the math anymore. You already have hardwareIn this context, "hardware" refers to functions in an IoT device that are built into a silicon chip or some other dedicated component. It's distinct from "software," which refers to instructions running on a processor. that knows how to skip zeros; now you’re giving it more zeros to skip.
Of course, by doing that, you’ve “cheated” a bit on the math – the final result will now be slightly different than it would have been if you’d included all of those tiny numbers. The expectation (or hope) is that the difference won’t be substantial. But you’d need to run the new streamlined modelA simplified representation of something real. We can create models of things in our heads without even realizing we're doing it. Technology often involves models because they let us simplify what would otherwise be extremely detailed, complicated concepts by focusing only on essential elements. against your tests to see if the accuracy drops too much. If so, then you might have to put some of the original numbers back instead of zeroing them out.
Leave a Reply