Microorganisms (yeasts, molds, bacteria), oxygen, and enzymes cause fruit to rot/spoil (changing color, texture, flavor, nutrient content in undesirable, unpalatable ways).
Fruits are nutrient-rich, at the right temperature range, and high in water content - optimal for microorganisms like bacteria, fungus, and mold to live and reproduce quickly. As they grow, these microorganisms start to break down the structure of the fruit, change the chemical environment, produce byproducts, and otherwise make it less desirable for human consumption. Fruits are generally too acidic in pH for most bacteria to thrive so they are more commonly subject to rotting by fungus and mold.
Oxygen can also accelerate the spoiling process by enabling enzyme actions and through oxidation. E.g. a cut avocado or apple slice left open to air will turn brown because of oxidation.
Enzymes that are naturally present in fruit can ripen and then over-ripen it over time. E.g. a banana ripens due to ethylene gas production and release. Ethylene binds to receptors that activate genes to make different ripening enzymes that soften the fruit by breaking down cell walls, changing its color by degrading chlorophyll, make it sweeter by converting complex carbohydrates into simple carbs, etc. More on ethylene and fruit ripening here: http://www.scientificamerican.com/article/origin-of-fruit-ripening/
As a relevant aside, there's an interesting idea proposed that microbes evolved to cause spoiling in food to make it unappetizing for bigger organisms as fast as possible, and in doing so, keep it for themselves. http://news.sciencemag.org/biology/2014/03/rotten-fruit-may-be-due-microbe-warfare