Can you have two apostrophes on one word? For example, is it OK to write y'all's?
Yes: a word with multiple contractions can have multiple apostrophes, which is commonly used when writing dialogue.
It's possible to have one word with multiple apostrophes. While it isn't recommended to use this form when writing something like an essay, when writing dialogue for a character, it can be helpful.
"I shouldn't've done this."
"We couldn't've been there."
The example given, "y'all's", is a bit unusual, but wouldn't be out of place in character dialogue and can be used correctly as such.