Basically, it's part of coloquial language. No one who says "it's gay" doesn't mean it totally sucks penises, ergo it's bad.
It's just how language evolved, and there is nothing homophobic about it. Now, here comes someone who tries to defend homosexual people by saying it aimed at them and it's clear remark. Now that's homophobic. Thank you, goodnight.