What determines the amount of gravitational contraction in a star?
A star is in hydrostatic balance if there is a force which can counter gravitational collapse.
In the case of main sequence stars, gravitational collapse is balanced by radiation pressure from the fusion reactions taking place in the star's core.
When a star of less than 8 solar masses has consumed all of the hydrogen and helium in its core, it can't achieve the temperatures and pressures required to start carbon fusion. At this point the star's carbon/oxygen core collapses into a white dwarf. In this case further gravitational collapse is held in balance by electron degeneracy pressure. This is a quantum effect which forbids two electrons being in the same state.
In larger stars fusion reactions created heavier elements until the core is mainly iron. Iron fusion requires more energy than is released. Once fusion reactions stop, the iron core collapses under gravity. In this case electron degeneracy pressure is not strong enough to stop gravitational collapse. The core collapses into a neutron star. Gravitation collapse in a neutron star is balanced by neutron degeneracy pressure which prevents two neutrons being in the same quantum state.
If the core has a mass of more than a few solar masses, neutron degeneracy pressure is not strong enough to stop gravitational collapse. The core collapses further into a black hole. In this case there is no balance as gravity has won. We don't really understand black holes as our laws of physics break down. General Relativity predicts that inside the black hole is a singularity which is a point of infinite density where all of the mass of the black hole resides.