Does the density of the universe change over time?
Yes. But maybe no.
The universe, as we now observe it, is expanding at an ever increasing rate. The means the distance between constellations is becoming greater making space less dense.
EXCEPT, string theorist have this idea that everywhere we believe there is a void, outer space for example, is actually filled with "strings." Which means if they are right, in general terms, the density of the universe will remain relatively constant.