Key Questions to ask when democratizing experimentation in an organization

Experimentation is extremely useful to validating new ideas and improving conversion rates (oh so, 2015) , learning about the customer and using insights gained to drive the business forward.

Predominantly the domain of a “CRO team”, there has been a wider interest in rolling out experimentation capabilities to the wider organization.

“We want everyone to be able to run a test”

“We want to experiment before we launch anything new”

These statements are made with gusto and met with applause by the experimentation teams who see this as a validation of their work. They are tasked with getting others in the organization designing and running their own experiments.

Finally experimentation can grow beyond the CRO silo.

We have observed hundreds of organizations and how they managed their experimentation programs.

Here’s the approach on how organizations currently do this.

• Hold workshops & roadshows for the teams
• Use presentations to show examples of tests done
• Technical training on tools
• How to prioritise ideas etc.

After an initial surge in interest, almost predictably, the activities within these newly trained teams starts dying down or worse still the experiments and ideas aren’t good quality or too basic.

Experimentation teams fail to account for one thing – the rollout and democratisation of testing isn’t a push button process. They can’t do a predictable set of training sessions (for example 3 sessions) and then expect the teams will suddenly change and adopt a new way of working.

Democratising testing is about PEOPLE first and understanding them.

These are the questions that teams should answer when planning the rollout of experimentation in their organization

Why?

This is perhaps, the single most important question that an organization should answer.

Why does it want everyone in the organization testing? What is the end goal once everyone is able to test?

Why should others in the organization care about testing?

A CRO practitioner will most likely be able to passionately answer all those Whys.

BUT…

The Why must be answered by those in senior management and those making the business strategies.

If they don’t know the Why and cannot articulate that into measurable outcomes, how will others follow suit.

The Why must come from top down with clear expectations, guidelines and accountability for all involved.

Who?

There are three different individuals needed for the “Who”

a) Who will set the standards? – Democratising experimentation programs at scale cannot be left to chance. Autonomy, whilst useful, can prevent scale unless clear guardrails are defined. There needs to be clear standards set up for the rest of the organization to follow.

b) Who is the audience? – Who are you going to educate and enable within the organization. Who are the individuals in this team? What are their capabilities?

c) Who will lead the rollout? This individual or individuals is going to be responsible for education, engagement and monitoring of the rollout program.

When?

Timing is key. Especially in large and enterprise organizations where teams are spread out all over the place (or the globe), it can be tricky knowing when’s the best time to bring them on board.

This is a nuanced process and requires an understanding of the different teams, their capabilities and motivations in wanting to be involved.

Using a prioritisation framework coupled with a SWOT analysis of the teams, you can create a rollout roadmap which shows you the timeline of educating and onboarding the teams.

How?

Along with the roadmap and SWOT analysis, it’s useful to know how different teams operate and how the individuals within those teams learn. Some prefer workshops, others may want a self serve course or a written document. Working within those parameters, you can create content that will engage, educate and ultimately persuade them to doing the work the new way.

This cannot be a set and forget process. Ensuring that you put in regular milestone checkins and monitoring will help make this a long term success.

This isn’t a sprint. It’s a marathon and will take time energy and effort.

At Effective Experiments, we know that software alone does not bring about a culture of experimentation. This is why we have combined our best in class enterprise experimentation management platform along with our knowledge of the industry, frameworks and consulting to drive change in the enterprise. This is the Experimentation Ops system

We believe Experimentation is the key driver for innovation and the more experiments run across the business, the more successful it can become.

With our Experimentation Ops system, we help with tailormade strategies for organizational rollout, continuous monitoring, growth of the experimentation related activities and democratising the insights across the business.

Interested to learn more?
TALK TO US AND SEE HOW WE CAN HELP YOU BUILD A SCALABLE EXPERIMENTATION SYSTEM WITH EXPERIMENTATION OPs