How to interpret 50% vs. 50%

Be careful when you see a user study saying that preferences over two design alternatives are 50% vs. 50%. It might not tell you the clear truth.

You may need to interpret its result as neither of the two options are good. Both options have its clear flaws, so the results tend to become more or less random. If you have time and resources, you need to fundamentally reconsider the overall design structure, so that you need to remove such a situation to beget the two design alternative. If you do not have time and resources, you need to choose one and document why you made such a decision clearly. Because either one is equally bad, when users run into a the interface, they often complain about it, which drives the designer to choose the other choice. This will cause back-and-forth between the two bad alternatives. This add yet another bad taste on top of already inferior design choice: inconsistency. Be careful when you get a user study result of 49% vs. 51%. Something can go wrong.

Written on April 15, 2017