Cache memory - what it is and why you need it

Cache memory is an essential part of the data transfer process that is often misunderstood. Ian Rayner, Kingston product manager at AXiZ, explains the importance of cache memory.

"Cache memory is a special high speed memory designed to supply the processor with the most frequently requested instructions and data," he says. "Instructions and data located in cache memory can be accessed many times faster than instructions and data located in main memory. The more instructions and data the processor can access directly from cache memory, the faster the computer runs as a whole. "

In general, there are two levels of cache memory; internal cache, which is typically located inside the CPU chip; and external cache, which is normally located on the system board. Internal cache is sometimes referred to as primary cache or level 1 (L1) cache. External cache is sometimes referred to as secondary cache or level 2 (L2) cache.

"When we talk about upgrading cache, we are most often talking about external cache," says Rayner. "Upgrading external cache may involve plugging individual cache components into sockets located on the system board or plugging a cache module into a dedicated cache expansion socket. In most cases, upgrading internal cache would require the replacement of the CPU."

An interesting way to look at cache is to imagine yourself at a party with a host that is required to serve you the exact beverage you request. The beverages are the data, the corner store is main memory, and the refrigerator is cache memory. If someone at the party requests a diet Pepsi, the host of the party makes a trip to the refrigerator first, to see if it is there. If the diet Pepsi is in the refrigerator, the requester can have it right away. However, if it is not in the refrigerator, the host has to run to the corner store to get it. This may take considerably longer. The host can save a lot of time by purchasing a 6-pack at the store. This logic insures that most of the time, the next request can be fulfilled directly from the refrigerator.

In the same way, when the cache controller retrieves an instruction from main memory, it also takes the next several instructions back to cache with it. This increases the chances that the next instruction requested by the CPU is already in cache. (When a request from the CPU is found in cache, this is referred to as a "cache hit").

On a typical 100MHz Intel motherboard, it takes the CPU as much as 180ns to get information from main memory versus as little as 45ns to get information from cache memory. This represents the total memory retrieval process, including request, verification, and data access time. With the incredible performance advantage cache memory offers, it would seem logical to use cache for all the computer`s main memory.

However, cache memory typically uses SRAM (Static RAM) chips, which cost more than six times as much as the DRAM chips normally used for main memory. Thus, it is not cost effective to use a large amount of cache in a system. In our party example, using cache as main memory would be similar to buying the corner store in order to stock every type of beverage that exists. While having one refrigerator saves a lot of time and inconvenience, the added benefit of having the corner store in the back yard may not be worth the investment.

"This is also how cache works," says Rayner. "The first 256K of cache saves the computer a lot of time by holding all the most frequently used instructions. However, adding 256K more of cache for a total of 512K does not increase the overall performance of the computer as much as the first 256K does."