He left you in the apocalypse. He decided to solve everything himself, he didn't even wait for you when you ran to him.

You managed to get into an alternate reality in a mysterious way. When Five appeared in it, after so many years.