Pipelines

[From the last episode: We looked at some of the characteristics of big and small processorsA computer chip that does computing work for a computer. It may do general work (like in your home computer) or it may do specialized work (like some of the processors in your smartphone)..]

A big question with processors is how fast they can work. But that’s not as easy a question as you might think. “Fast” can mean different things.

First, all mainstream processors use a clock to synchronize all the activity. So one of the high-level descriptive elements of a processor is how fast the clock can run. A fast clock might run in the gigahertz range; a slow clock in the megahertz range*. Does a faster clock mean a faster processor? Yes, but there’s a catch.

Prepping Apple Pies

Let’s start by making apple pies (a favorite analogy of mine from way back). You’re working by yourself, but you want to make 10 pies. There’s a lot of ways to attack this challenge, but let’s assume you make one pie fully at a time, starting over for the next pie. (We’ll assume they bake at the same time so we don’t have to worry about bake time.) So, for each pie, you need to:

1. Make the crusts
2. Lay the bottom crust in the pan
3. Wash the apples
4. Peel the applies
5. Slice the apples
6. Add sugar and spice to the apples
7. Place the apples in the pan
8. Lay the top crust over

Let’s just say that takes 24 minutes to complete. So you can prep 2 pies plus part of a pie per hour. What if that’s too slow? It would take 4 hours (10 x 24 = 240 minutes) to prep all 10 pies!

An Apple-Pie Pipeline

Now, what if you could get some help? Then you could start doing things assembly-line style. Let’s divide things into four stages that take roughly the same time:

1. Make the crust and put the bottom into the pan
2. Wash and peel the apples
3. Slice the apples and add sugar and spice
4. Put into the pan, and layer on the top crust

Now, if you have four people, the first person can do those first steps and then hand that pie off to the second person. The first person will then start working on the crust for the next pie. The second person, meanwhile, will prep the apples for the first pie**. When done, he’ll give the apples to the third person and then go start on the apples for the next pie. The third person – and then the fourth person – will finish off the last steps, and the first pie will be complete.

Did the Help Help?

Did we save any time with that first pie? Nope! Still takes 24 minutes. But… if each of those four workers takes 6 minutes each, then, 6 minutes after the first pie is done, the second pie is done. Why? Because while the 2^nd, 3^rd, and 4^th workers are finishing one pie, the 1^st worker for the next pie is already starting. And when the first pie is getting finished, the 2^nd and 3^rd workers on the next pie are also just finishing. So, once we get going, after the first pie is finished, we’ll get another pie every 6 minutes until we’re done. So that’s 1 pie in 24 minutes plus 9 pies in 6 more minutes each (total of 54 minutes for those 9) making a grand total of 78 minutes, or just over an hour – way faster than the original 4 hours.

This is an example of what we call a pipeline. There are multiple stages along the pipeline, and a different person handles each stage, freeing up the previous person to repeat what they just did. It takes a while for the first one to get done – we call that filling the pipeline – but, once the pipeline is full, then things come out much more quickly. It still takes 30 minutes for each pie to be made – we call that the latencyThis can have various meanings, but in the context of a pipeline, it’s how long a specific single task takes to get be completed., which is how long it takes a specific pie to go from start to finish. But because the pies are overlapped in time, we can be more efficient.

Pipelines in Processors

The same thing happens with a processor. There are several steps needed for every instruction. They may be somewhat different for different processors, but typical steps would be:

1. Fetch the instruction from memory.
2. Decode the instruction (the instruction is 1s and 0s, and the circuitry needs to figure out what each instruction means).
3. Fetch any operands from memory. For example, if you’re adding two numbers that are in memory, you need to get both numbers out of memory so you can add them.
4. Execute the instruction – meaning do the adding in our addition example.
5. Finally, store the result somewhere.

Really small processors might do all of this in sequence for each instruction, with no pipeline. That’s unusual these days. You get better speed by using a pipeline. The thing is, the more steps you make separate in the pipeline, the quicker each step completes. With our apple-pie example, if each of the 8 steps took 3 minutes, then you could do an 8-stage pipeline (using 8 people) and, after the first pie, you’d get another one every 3 minutes instead of 6 minutes, with all 10 done in 51 minutes.

So bigger, faster processors have longer pipelinesA way of speeding up a repeated task by breaking it up into stages and having different resources do each stage. It takes longer for the first task to be completed (that’s the latency); the others come out in quick succession after the first one., with a smaller amount of work being done in each stage. Smaller, simpler processors have a shorter pipeline. There’s another trick that bigger processors use to save time, and it affects the pipeline, but we’ll talk about that next time.

 

Notes

* If you’re new to clock speeds, a hertzThe unit used for frequencies. It’s abbreviated Hz. Equivalent to cycles per second. A periodic signal that completes a cycle in one second is said to have a frequency of 1 hertz, or 1 Hz. measures how many clock cycles can happen in a second. A normal clock simply oscillates between a HI and a LO value, back and forth. When it goes HI for some time and then LO for some time (usually the same amount of time for HI and LO), that’s one cycle. If it can complete one cycle in one second, that’s one hertz. “Mega” means millions, so 1 megahertz (1 MHz) means the clock can complete a million cycles in a second. “Giga” means billions, so 1 gigahertz (1 GHz) means the clock can complete a billion cycles in a second. Yeah, that’s fast!

** You might say, “Hey, the apple person doesn’t really need to wait for the crust to get started!” And you’d be absolutely right! But that brings us into way more complicated territory that we’ll look at sometime later. Let’s just go with this for now for simplicity.