IB HL/Software Engineering

The CPU, pipelining, and parallelism

50 words to know. Each one with what it means, some with an example of how it's used.

CPU

The part of the computer that runs instructions, made of a control unit, an ALU, and registers.

ALU

The part of the CPU that knows how to do arithmetic (addition, subtraction, etc.) and logic (and, or, not). Most of its operations take two inputs and produce one result.

The ALU is drawn as a chonky V because its operations take two inputs in the top and spit one result out the bottom.

Control unit

The big boss of the CPU: tells the ALU what to do, controls access to memory, and coordinates the fetch/decode/execute cycle.

RAM

Memory outside the CPU where data lives when the CPU isn’t operating on it. Not part of the CPU, but the CPU is pretty useless without it.

Register

Very fast memory that lives on the CPU and each one holds one piece of data. A simple CPU has just a few while modern CPUs add multiple general-purpose ones.

Accumulator

A register connected directly to the ALU in order-style CPUs used as the both an input to most operations and also where the result is stored.

PC

The register holding the address of the next instruction in memory.

A jump instruction works by changing the PC.

IR

The register holding the data representing the current instruction.

MAR

The register holding the address to be used for loading from and storing to RAM.

MDR

The register holding the data just loaded from RAM or about to be stored to RAM.

General-purpose register

One of a set of registers that give the compiler places to store values on the CPU so it doesn’t have to go to RAM as often.

Cache

Fast memory directly on the CPU where it keeps data that is used a lot, without the compiler having to do anything.

Flag

A one-bit value set by the ALU and read by the control unit to carry information about a computed result: zero, negative, carry, or overflow.

Z

The flag set by the ALU when the result it just computed is zero.

N

The flag set by the ALU when the result it just computed is negative.

C

The flag set by the ALU when an operation, treating the bits as unsigned numbers, had a carry out.

V

The flag set by the ALU when an operation, treating the bits as signed numbers, overflowed resulting in a value with a sign flipped from what it should have been mathematically.

Bus

A set of wires connecting different parts of the computer.

Control bus

The bus the control unit uses to signal the rest of the computer, for instance to tell RAM to do a read or a write.

Address bus

The bus that carries the address to be operated on from the CPU to RAM.

Data bus

The bus that carries data between RAM and the CPU, in either direction.

Fetch/decode/execute cycle

The three-phase cycle the control unit repeats to run instructions.

Fetch

To load the next instruction from memory.

Decode

To parse a fetched instruction into its op code and arguments.

Execute

To actually run a parsed instruction.

Op code

The part of an instruction that tells the control unit which operation to perform, e.g. ADD, LOAD, STORE, etc.

Immediate value

An argument whose value is written right in the instruction, as opposed to a memory address to load it from.

Jump instruction

An instruction (behind conditionals, loops, and gotos) executed by setting the PC to something other than the next instruction, possibly depending on the flags.

Pipelining

Taking a sequential process that is performed repeatedly and increasing the rate at which it can be completed by overlapping the parts that can be done in parallel.

Starting the next load of laundry in the washer while the first is in the dryer is pipelining.

Pipeline stage

One step of a pipeline, which works on one item while the other stages work on others; the classic instruction pipeline has five.

Five-stage pipeline

The classic breakdown of the instruction cycle into instruction fetch (IF), instruction decode (ID), execute (EX), memory (MEM), and write back (WB).

IF

The pipeline stage that fetches an instruction.

ID

The pipeline stage that decodes a fetched instruction.

EX

The pipeline stage that performs any ALU operation for an instruction.

MEM

The pipeline stage that reads from or writes to memory, if the instruction needs to.

WB

The pipeline stage that writes an instruction’s result back to the CPU’s registers if needed.

Split cache

Separate caches for instructions and for data — a win because different pipeline stages running at the same time care about one or the other.

Branch

A jump that sets the PC to something other than the next instruction.

Pipeline stall

A pipeline stage having to wait, for instance for the right instruction to be fetched after a branch, which wastes cycles and reduces throughput.

Branch prediction

A CPU keeping a table of which instruction address was actually executed most often after each branch instruction, and fetching that next rather than always fetching PC + 1.

Parallelism

Doing more than one thing at the same time, like two trains running along separate tracks next to each other.

Throughput

How much total work gets done in a given time: the number of trains that get from A to B in a day.

Two people eating a pizza doubles the throughput without eating any one slice faster.

Latency

How quickly any one piece of work gets done: how long one train takes to get from A to B.

Coordination cost

The time people or processors working in parallel spend deciding who does what and waiting for each other, which can make parallel work take more total time than doing it alone.

Amdahl’s Law

The formula for the overall speedup of a program from parallelism, given the proportion of it that is parallelizable and the amount of parallelism: if 90% of a program can run with infinite parallelism, it still only speeds up 10x.

Multi-core CPU

A CPU with several cores, each essentially a whole CPU that executes instructions independently, sharing an extra level of cache (L3) and a memory controller.

Memory controller

The part of a multi-core CPU through which all of the cores’ memory access is controlled.

Task switching

How the operating system on a single-core CPU gives the illusion of multiple things happening at once: by very quickly switching between tasks.

GPU

A processor built for inherently parallel work like computing the color of every pixel, with lots of ALUs that all execute the same instruction at the same time on different data.

Streaming multiprocessor

One of the many units a GPU is made of, each with several processing blocks that each run one instruction across dozens of ALUs at once.