What is the main problem with hard water for most people?

In places where water hardness is a problem, the tap water is slightly alkaline (i.e. $p H$ $>$ $7$) (it might only be $7.05$, but this is enough). When you wash your clothes (unless you add a water softener) the detergent is not so effective; the tap water tastes a bit bitter. This can be traced to the presence of calcium and magnesium salts in the water that form insoluble carbonates (soap scum!).