You'r not failing at life. You need to understand that people generally don't care what you do. If you put yourself in a colleagues situation, would you really be that bothered if they had 3-weeks off work? You might ask them what they've been doing out of curiosity, maybe even tease them a bit.....but really you wouldn't care what they had been doing or even why they had been off for weeks.

It will be the same for you, people will probably ask you where you've been, but they'r not judging you because they simply aren't really that bothered what you get up to.