Save for the occasional contrarian thinker, we all know that the likelihood of a die-hard politico swapping allegiance just because their candidate lost is never.
Similarly, when we A/B test, and assume that just because A “won” that everyone will be happy with A, we’re fooling ourselves. All you know is that the people who voted for A want A, and the people who voted for B want B; there is zero correlation to prove that, should B gain fewer voters, those voters will jump ship to A if its forced down their throats. If you don’t trust, just visit a rural Evangelical Christian community with some Obama posters in hand and see if the B pickers all of the sudden have heart for A.
This is why segmentation is the name of the game; you just have to be careful how you do it. Not everyone who picked B can be dumped into the 25-35 year old male with a $75k or more income bucket, much in the same way not all northern urbanites are Democrats.
Dig into personas, understand not just what people want, but why they want it, the benefit they see in the product, and how it will enhance their life. Bucket people based on how they think inside their brains, not superficial demographic data — which means figuring out what it is that makes them tick. It may not be obvious how to take the first step, but it doesn’t have to be hard. The more simple you make a test, the more clear your results will be.