Experimentation Programs Need Better Evangelists In Their Teams

There are a lot of expectations piled onto an experimentation team (or the few testing specialists that operate in that capacity).

Not only are they meant to fulfil their role by ideating, prioritising, building a backlog of tests, building the test briefs, communicating it with devs, managing the experimentation pipeline, managing conflicts in the test calendar as well as building reports and dashboards to communicate value to the wider business.

Then along came added pressure.

CROs have to build a culture of experimentation! Mind you, no one has actually told them to do that nor has there been any expectation or remit given to them by the business. This is just the consensus in the echo chamber amplified on Linkedin, conferences and facebook pages dedicated to the craft.

Experimentation can’t succeed on its the merit of successful tests and learnings alone. That much is true.

Ask any CRO about how to best evangelise testing and bring about a “culture of experimentation” and they will parrot off a list as if they were programmed to do so.

• Educate them.
• Empower them
• Make them see the light about testing
• Shout out about testing and share everything you do.
• Bring them on your side

I have heard all of these statements before. It’s like CRO bingo.

At Effective Experiments, we have done extensive research in organisations large and small with varying levels of Experimentation maturity and we reached a conclusion.

The CRO should not be the person responsible for this.

The answers they have provided are true. To an extent. But it’s just empty words because the underlying actions are not powerful enough and yield no results.

It’s no wonder that the most common complaint of a CRO is how to get buy in for their work.

Culture is very hard to create especially from within a silo which most CROs sit in. It’s even harder to do that when you are not always in a position of authority.

Take this example –

Joanna (name changed for obvious reasons) wants to win stakeholder support and thinks of highlighting and sharing the winning tests and learnings. She believes that by holding a meeting with senior stakeholders and showing them all the tests and dashboards that she had been meticulously putting together for the last week, they would be wowed and would cheer on the testing team.

The meeting came, there were some questions and some claps but by and large she didn’t see any change in the organisational attitude.

What has Joanna done wrong here?

She has done exactly what her peers in here network had been advocating.

This is the area where the CRO’s skills do not quite fit the bill and it has nothing to do with the ability to present.

The ability is here is how to change the minds of the people sitting in the room. They are there thinking about their own workload, their own agenda and their KPIs. This meeting if anything is a minor inconvenience.

The CROs within their circles assume that meetings like this will bring others to their cause. That firing off some emails with KPIs and stats will show them the light.

Building a culture of experimentation has very little to do with experimentation.

Yes, you read that right.

A big part of building culture is change management and change is extremely hard and slow. It cannot be done in a passive way. It requires time and effort to build and nurture relationships, navigate tricky politics and general apathy towards change.

Change management is a hard task and a completely different skill set to that of a CRO specialist.

Experimentation teams need a dedicated person to be that champion and the liaison between the experimentation teams, the wider organisation and the senior management.

Whilst marketers and CROs love to look at a tool to automate everything, the evangelist is someone who realises that this work requires the hours to build and cultivate relations.

Organisations need to understand how experimentation fosters innovation at the highest levels and in order to make experimentation more powerful, there needs to be rethink in how experimentation teams are architected and orchestrated.