I believe the answer you're aiming for is race and I'm sure that was a factor. However, Cain's popularity stayed about as consistent as any other candidate's throughout the affair news, so race may not be it. Second, Cain left the race citing family concerns, which I would believe if his wife was just enlightened to Cain's philandering ass (I'd love to be a fly on that wall). Finally, Gingrich is old news. Everyone's heard of that asshole and I'd be surprised if he wasn't having an affair with some 25 year old right now. The news about Herman Cain was fresh and thus more damaging.
Then again, this is assuming that Cain actually intended to run for president and not to sell books.