The problem centres on the need to create a theory of gravity that follows the rules of the quantum world.
To my (limited) understanding, three forces - electromagnetism, and the strong and weak nuclear forces have been unified in that each can be seen as being a particular aspect of a larger "electro-nuclear" force, or grand unified force, as I believe it is called.
These quantum versions of the three fundamental forces are gauge theories and centre of gauge particles that convey each force - photons, gluons, W and Z particles.
The problem is that no one has been able to come up with a suitable quantum version of gravity - one that does not create infinities as you try to apply it to the microscopic quantum world. Thus, this force remains as the single "split off" member of what must eventually be unified into a single set of laws - the theory of everything.
I'm not sure if this gives you any information you did not already have, but perhaps other contributors will add to it!