Same sex marriage is a concept that is intrinsically linked to homosexual persons.
The counterargument that two hetero men could marry too with SSM established is equivalent to saying that without SSM, a homosexual man can marry any woman he wishes, so he is not legally disadvantaged. In a mathematical sense true, but utterly irrelevant in the real world. In the real world, context is a thing, and you can't isolate a single aspect to the situation and make a generalization about it, unless you show that those other aspects are irrelevant. The right to marry a woman is worthless to a gay man, and the right to marry a man is worthless to a straight man, so a law banning SSM is not affecting heterosexual people in a meaningful way. Therefore, it is targeting homosexual people specifically. Speech supporting such a law follows in the same vein.