The most common source attributed for the exclamation mark is the use of the Latin "io" (with the
It is said that clerical copyists during medieval times would often end a passage with this. (In Latin "io" means "joy" or "hip-hip-hooray, I managed to copy that without making a mistake").
There is really no historical evidence to support (or refute) this claim.
Other claims (though less common) place the origin with early Greeks, Egyptians, or prehistory story tellers.