Organizing Internal Working Memory

[From the last episode: We summarized the last several weeks of computing posts.]

We’ve talked about processors and we’ve talked about several kinds of memories. Today we’re going to focus in on the memory used when a program is running. We’re not talking about permanent storageThis usually refers to memory that doesn't lose its contents when powered off - like a thumb drive or a hard disk. It's a place to store data. here; we’re talking about volatile working memory – DRAMStands for "dynamic random access memory." This is temporary working memory in a computer. When the goes off, the memory contents are lost. It's not super fast, but it's very cheap, so there's lots of it. and SRAMStands for "static random access memory." This is also temporary memory in a computer. It's very fast compared to other kinds of memory, but it's also very expensive and burns a lot of energy, so you don't have nearly so much.. We’ve seen that SRAM can be used for cache, but we’re going to look at non-cache SRAM. If that’s not clear now, hopefully it will be in a moment.

And here’s the question we’re going to look at: if you’re a designer, how do you organize that memory? The answer, of course, will depend both on what the system is and what the application is. The system question gets to this: do you want this system to do one thing (or a few well-defined things)? If so, then an IoTThe Internet of Things. A broad term covering many different applications where "things"  are interconnected through the internet. device would qualify here. The other possibility is that the system is a general-purpose computing machineIn our context, a machine is anything that isn't human (or living). That includes electronic equipment like computers and phones., like a laptop. In that case, you don’t know what the system is going to do when you’re designing it.

For our discussion, the big question is: Do you know ahead of time what you’re going to be storing in memory? And do you know exactly how much memory space it’s going to take?

The Data You Know

If you know what your data is going to look like, then you could choose to create fixed locations for data in memory. If you’re creating an IoT thermometerA sensor that measures temperature., for example, then you can assign the location of the temperature reading within memory to one fixed place. Why is that good? Because then it’s easy and fast to get to. It makes the system simpler.

Should that memory be DRAM or SRAM? Well, that depends on how fast you need that access to be. If you’re not operating at blazing speeds – as in the case of a thermometer – then you could write and read directly from DRAM. If you need more speed, then you could do one of two things:

• Use DRAM for storage, but cacheA place to park data coming from slow memory so that you can access it quickly. It's built with SRAM memory, but it's organized so that you can figure out which parts haven't been used in a while so that they can be replaced with new cached items. Also, if you make a change to cached data, that change has to make its way back into the original slow storage. the data in SRAM. We looked mostly at instruction cache before, but you can also create a data cache that works the same basic way. If you do this, it’s faster to get the data from the cache, but if you get a cache missRefers to the situation where you need memory data that’s not already in the cache, so you first need to load it from slower memory into the cache., then you have to wait for it to be loaded into the cache from DRAM.
• Use SRAM directly. This is always the fastest approach – it’s just that SRAM is expensive and power-hungry, so you probably do this only if you have to.

The Data You Don’t Know

What if you don’t know what the data will look like? Here are two examples where that might be the case: a laptop computer, where you don’t know what programs will even run, and a networkA collection of items like computers, printers, phones, and other electronic items that are connected together by switches and routers. A network allows the connected devices to talk to each other electronically. The internet is an example of an extremely large network. Your home network, if you have one, is an example of a small local network. routerAn electronic box that helps steer data on a network. For instance, you may have one in your home connecting your phone and computer and other devices to each other and to the internet. The data itself has information about where it's being sent; the router uses that information to send it in the right direction. At a really basic level, you can think of a router and a switch as being the same thing. If you want to get more technical, a switch creates a local subnetwork, and the router connects multiple subnetworks (or multiple networks)., where the data packets being processed could be of different sizes.

In these cases, you will have things to store – you just don’t know what those things are going to look like. So instead of fixing the locations of things, you just create a big chunk of memory – typically DRAM, since there’s more of it – and then you allocate that memory when you need to store something in it.

Memory used this way is sometimes referred to as heap memoryA block of working memory reserved for use by programs as they request an allocation. The programs request and release memory from the heap as they go, and where specific data will end up in the heap is unpredictable.. As in, it’s just a heap of memory, and you want to put something somewhere in it. But where will it go? That depends on what’s happened before. If you assume for a moment that you’re going to start at the beginning of the big block and put things in as they come in, then where something goes depends on what came in already. If it’s the first thing in the door after powering on, then it will go at the beginning. If it’s not, then it will go at the start of whatever memory is still free.

This is something the operating system takes care of. When a program asks for memory, it tells the operating systemSoftware – or firmware – that handles the low-level aspects of a computer or IoT device. It gets direct access to all the resources. Software asks the OS for access to those resources., “I need this much memory to put stuff in; please find where that would go and send me the addressWhen referring to memory, a binary number that shows where, within some memory block, the data you want is located. If you want to read or change that data, you have to give the address so that the right data is read or changed..” The program can then refer to that address as it runs. (That address is literally referred to as a pointerA variable that, instead of holding data, holds an address where data is located. That is, it "points" to the data instead of being the data., because it points to the place in memory.)

Fixed vs. Flexible

The critical thing here is the following: with the “data you know” systemsThis is a very generic term for any collection of components that, all together, can do something. Systems can be built from subsystems. Examples are your cell phone; your computer; the radio in your car; anything that seems like a "whole.", you know where the data is going to go before even powering on the system. The location is fixed. With “data you don’t know” systems, you don’t know where that data is going to go literally until you ask for the memory allocation. If a system can run more than one program, then it depends on who requests what in what order, and that’s completely unpredictable. Here the location isn’t fixed, but it’s much more flexible – the data can go wherever there’s room. But it’s also a little slower, since you have to do the whole memory-allocation thing.

Here’s the other thing: when you know your data and you know you’re always going to have it – like where you store the temperature reading in the IoT thermometer – then that data is always there. With other systems, you may be allocating space for something that will be there for a while – like the data packetA group of bits being sent from one place to another. How big the group is may vary depending on what kind of packet it is. Long messages -- like an email -- will typically be broken up into many packets, each of which travels independently until it gets to the destination, where they're reassembled into the email. you’re working on now – and then you’ll be done with it. Once you’re done with it, you don’t need that memory space anymore, so you can release it for future use for something else.

Which brings up a challenge: when you release the memory, you may create a “hole.” We’ll talk about this next week.