That means that you had the opportunity to say you had other plans... "Thank you for the invitation, we do wish you a happy birthday, and would have loved to attend your party.... BUT - WE have already planned a camping trip this weekend"
of course, if you didn't want to go. You could say "Guys if it had been a little later, we'd go - We just came from camping in the OBX" I wouldn't have touched that because I get home sick after a while. But you didn't use that one, you used Birthday Part.
I'm just going to shake my head in the highest level of disappointment, and go to bed. Good night, Sir!