Registers are very small, very fast, memory locations for holding instructions or units of data. Registers operate at the same speed as the CPU, whereas normal system memory can be many times slower.
During operations, CPU's store data and instructions in registers. That information is then transferred back to main system memory. To speed operations, the control unit can "prefetch" instructions and data from system memory and store it in the CPU's registers.
CPU's can have many registers, with groups of registers devoted to a specific purpose (and thus unavailable for other uses). Some modern processors can use registers as needed for the task at hand, rather than being limted by a limted quantity of special-purpose registers.