Two factors are actually involved – density and miscibility. The density causes the more dense (heavier) fluid to sink below the less dense (lighter) fluid. But if the fluids can be mixed homogeneously they would quickly from a single fluid mass of uniform density and no separation. It is the immiscibility of oil and water that keeps the two solutions separate from each other, maintaining their individual densities.
That creates the effect of globules of one fluid floating around inside the other. With the application of heat (the Lava Lamp) the relative density of the fluids changes, making them swirl around each other in alternate rising and falling masses instead of simply settling into two distinct layers with a horizontal boundary layer.
The salt is just used as a density and interface modifier to produce the desired fluid flow and surface tension.