What does “Conclusive Results” mean?
When conducting a test reaches the state when one hypothesis significantly wins over the other and there is a clear winner, it’s obvious that their results are significantly more notable. That means you will choose one over the other in 100 % of the cases, thus you have reached conclusive results in your testing process.
A few things to keep in mind when conducting A/B test:
A. Has the effect of the variation been tested?
B. Have all elements expected for a real-world interaction been considered?
C. What is the percentage of probability that declares victory if 95% is considered an acceptable threshold (not 100%)
Don’t make big decisions without testing
Random traffic patterns could be a result of your competitor going out of business the week before and not knowing.
Just because a blue button did well on one page doesn’t mean it will do well for you on another. Avoid making big changes without testing.
Sometimes a change for the better is hard, but it’s worth being consistent. Customers also like to be able to identify you based on your brand identity and overall customer-facing image.