Becoming hypothesis driven, not idea led

All major developments in the history of humanity have been brought to us through the structured and scientific-approach of research and experimentation.

Somewhere along the way, we started believing opinions without being shown any proof to back them up (ironically, you shouldn’t believe everything you read on the internet).

An idea is the opinion, a hypothesis challenges you to prove the legitimacy of the opinion.

A hypothesis is a prediction or theory about what your research may find. It is a tentative answer to a research question that is yet to be tested.

If you’re not hypothesis driven, you’re being led by opinion and gut feel. Organisational hypotheses should be tested by a scientific process of experimentation to separate the signal from the noise.

Being idea-driven leads to false facts. This is when opinions are accepted as fact, without being proven or disproven.

There is a very simple step by step guide that can help ensure you’re being hypothesis driven:

Step 1: Define the business goal, for example: increase revenue

Step 2: Define an objective, let’s pick increase website traffic

Step 3: Either through an ideation session, research or learnings, come up with a list of ideas which could impact the objective

Step 3: Prioritise all your idea ideas based on how much impact you hypothesise each idea will create.

Then come up with a hypothesis “Ideas 2,3,7 will have the biggest impact on increasing conversions on the checkout page”

Step 4: Gather data and facts to prove or disprove your hypothesis.

Your hypothesis must be able to be proven wrong

This can be from any of the following methods:

  • Run Experiments
  • Research reports
  • Expert/Industry consulting
  • Blog posts
  • Books
  • Youtube etc

While there are many different ways to validate your hypothesis, each bullets above carries a different level of evidence strength. Some data sources hold a higher level of credibility and evidence strength, relative to others.

Running experiments will always be more credible than reading content on the internet, when attempting to understand how each of the ideas will perform.

It can be very dangerous to make your business decisions based on data from other companies or time periods…so always try to gather your own high-quality data.

This is an organisational issue but starts with establish facts, through research, observations and analysis.

  • Instead of saying “our customers aren’t interested in the prices of our products” – run experiments to understand the impacts of pricing structures on customers
  • Instead of saying “our customers need this shiny new feature” – take time to speak to customers and understand their goals and objectives
  • Instead of saying “we’re an industry-leading, trusted, wonderful brand” – conduct customer and industry research to understand if this is true

The sculpting of a hypothesis can be tricky; how can you measure impact, what metric will determine your success or failure?

There is a fool-proof way of creating a hypothesis to ensure you’re challenging the idea in the most effective way. You can use a hypothesis tool kit, which will help to standardise your process and your hypotheses.

Here’s one we recommend: http://www.experimentationhub.com/hypothesis-kit.html

A strong hypothesis is the heart of data-driven optimization. Hypothesis statements help you turn a wealth of data and insights about your visitors’ behaviour into focused proposals on which you will take action.

Programs that do not use hypotheses risk wasting resources on unfocused experimentation that fails to make a business impact.

Think in terms of hypotheses, not ideas.

Max Crawley-Moore

Customer Success Manager at Effective Experiments.

Making customers happy and frustrating our dev team :)