Why can a magnet separate iron atoms from a mixture but not from a compound?
Because the iron atoms are chemically bonded to other atoms.
Certain mixtures can be separated physically like filtering, boiling, magnetism, etc. When something is separable by magnetism, it is physically attracted to the magnet. There is no chemical change going on.
Compounds, on the other hand, are pure substances that require energy to break.
In the case of iron, it forms ionic compounds so the energy required to break the bonds of 1 mole of the compound is called the lattice energy. These are usually really high values in the thousands of kilojoules per mol of compound.