An experimentation program is more than just running tests and there are lots of different moving parts including the people involved that you need to watch out for.
When running your experimentation program properly, you need to keep a close eye on the metrics that can help you determine if your program is healthy or not.
In this final part of a three part series on Vanity Metrics in Experimentation programs, we will look at how the activities centred around Culture of Experimentation is a vanity metric.
There are clear indicators of successful experimentation programs. The biggest of these is how an organization manages the documentation aspect of their experimentation program.
Experimentation documentation cannot be an afterthought. There must be a clear strategy and plan behind it to ensure it is valid and useful to the organization. A spreadsheet is not a strategy. Neither is hooking up random project management tools together with APIs in your spare time.
What will separate a lot of organizations running mediocre and poor experimentation programs from those that do it well is how they view and structure documentation as part of their experimentation process.
Looking to increase your CRO knowledge and meet likeminded people in the industry?
We have compiled a list of all the CRO specific conferences you should attend in 2023.
The challenge with organizations trying to grow their experimentation program across the wider business is it faces an uphill challenge to get a disparate group of people to run experiments at the same level of quality. Often, there is no visibility of the quality of the experiments run.
This is why created the Experiment Health Score.
Experimentation program documentation – the process from idea to result – are (or rather should be) a significant part of an organization’s experimentation program. It is something we at Effective Experiments have been talking about since our inception in 2014 when the idea of tracking experiments seemed like an alien concept. Stakeholders are now starting […]
Experimentation teams could be changing their hypothesis or data after an experiment is run. Here’s how to stop this malpractice in your team.
Building an experiment roadmap can be tricky but can also be highly beneficial for your organisation and by the same token; having a badly maintained roadmap can be incredibly costly. A poorly designed roadmap is, for all intent and purpose, a poorly designed experimentation program. The roadmap is the core of why, what, who and […]
A hamster on a hamster wheel can run for long periods of time but at the end of the day, it doesn’t go anywhere beyond the confines of that place. Experimentation programs are stuck on their own kind of hamster wheel. They do good work and run lots of tests but ultimately, the real litmus […]
The quality of your decisions and the quality of your results are not always related. This post explores decision making and ‘resulting’.
Bartle House, Oxford Court,
Manchester M2 3WQ,
United Kingdom
Registered in England & Wales
Company number 07811720
VAT GB206256333
© Digital Tonic Ltd. All Rights Reserved
