Launching the Experimentation Changemakers Roundtable

Organizations are looking to scale their experimentation practice to all parts of the business. As such, experimentation is a really important part of an innovation minded business’ toolkit.

CRO or at least the nature of current programs in organizations is very skewed towards technicians and specialists. These are the people who are ideating, launching tests, analyzing and reporting. They love their craft and who can blame them? It’s fun!

However, they are not best equipped to grow a culture of experimentation, let alone oversee the rollout and growth of experimentation across the business. We covered this in a webinar we ran recently, but I’ll summarize some of the points:

  • They already have enough on their plate. A CRO is already pressed for time planning, ideating, running tests and reporting on them.
  • Experimentation culture change is not about A/B testing technicalities. No one in the wider organization cares about p-values, statistical significance, etc. The change needs to happen in the organization structure and dynamics. It needs to be led by someone with seniority who has a stake in growing experimentation.
  • The process of adoption and growth of Experimentation practices takes time and needs to be strategically planned, governed and monitored. This needs its own set of specialists (we call them Orchestrators).
  • Experimentation evangelization must be done diligently. It’s not just about doing email blasts and roadshows. It’s an internal sales process that needs – you guessed it – strategic planning and execution. This is a role that we have defined as Ambassadors.

There is a dissonance in the experimentation industry. On one hand, you have awards and articles where practitioners and specialists talk about how they’re single-handedly changing the culture in organizations. On the other hand, when we speak to the leadership teams in these organizations, the story is a different one. 

You will find plenty of courses and blog posts on growing a culture of experimentation and managing experimentation programs. They are passive mediums where you can read and consume, but thereafter the execution is yours to do without any guidance or feedback.

Back in early 2019, we ran a 6 month training program called Experimentation Bootcamp, which was designed to teach CROs how to democratize experimentation and grow a culture of testing and learning.

Unfortunately, it failed. And through hindsight and experience we learnt why. The CROs – at least the ones who love the technical craft – are not and should not be the ones leading this change. Expecting them to do it, however enthusiastic they may be, is a folly.

With all the lessons learnt, we’re excited to announce a new initiative.

Experimentation Changemakers Roundtable

The Experimentation Changemakers Roundtable is designed for those in the experimentation field who are looking to do more than running tests. If you love ideating, running and reporting on tests, there’s no harm in that but this isn’t for you.

The Roundtable is for those who want to lead and grow high quality experimentation programs. It’s for those who want to evangelize and build relationships with teams and business units beyond the CRO silo. It’s for those who want to learn practical tried and tested strategies to be able to communicate the value of experimentation.

The Roundtable is a monthly 1-1.5 hour Zoom call with peers who are looking to achieve the same goals as you – growing Experimentation and helping their businesses innovate.

You will learn strategies from our Experimentation Ops framework, discuss it on call and then implement it. When you come back for the next session, you will present your learnings back to your group. This constant execution and feedback will help you and your organization get better in growing experimentation practices.

Participation is mandatory if you want to be involved in this roundtable. This won’t be a “sit back and observe” type of call. Organizational change is hard work and will need all your effort and focus.

This one is for the changemakers. Are you one of them?

How to apply to be part of this roundtable:

Manuel da Costa

A passionate evangelist of all things experimentation, Manuel da Costa founded Effective Experiments to help organizations to make experimentation a core part of every business. On the blog, he talks about experimentation as a driver of innovation, experimentation program management, change management and building better practices in A/B testing.