Well, its because of the normal force.
You might think that gravity will make us fall, but remember that when we are riding a bike, the bike's seat exerts a normal force on us, counteracting the gravity. That is why we do not fall while riding a bicycle.
On a side note, the normal force is always perpendicular to the surfaces that are in contact. So, the gravity vectors and normal force vectors cancel out, and therefore making the net force