Why Sharks are better than Pandas: The art and science of failure

(This is a transcription of my talk for Drink Digital on 25th June 2020.)

When doing any public speaking you’re always told you need a strong opening. Something that really gets your audience to sit up and pay attention. You then use this attention to hammer your key point of what you’re going to talk about.

So here we go.

I hate pandas.

Look I know what you’re going to say. They’re so cute. Look at his little furry face, and those ears! OMG! And then after that, you’ll get righteous and tell me that how dare I hate Pandas they’re incredible creatures and after all, they’re a logo.

And you’re right, and look i’m as big a fan of wrestling entertainment as the next person.

By the way, this is the real reason they have black eyes, they can’t keep their hands off each other! They love a good punch up.

But let’s just be serious, for a moment, obviously, the Panda is under massive threat as a result of human expansion in china and in fact all across South East Asia. As they were once widespread amongst central and eastern China and even into Northern Vietnam and Myanmar. Clearly, the impact of human expansion has been and continues to be devastating on wildlife across the world, but for me, the Panda has kinda brought it on itself.

But the fact that they’re the logo of the WWF demonstrates that there is a level of commitment to this species which is far in excess of the value it brings. (Pandas branding agency were on point the day they pitched that one...)

But this is an animal that has backed itself into an evolutionary cul-de-sac regardless of what we do. And i quote.

Despite adaptations in the forepaws, teeth and jaws for bamboo consumption, the giant panda has retained the digestive system of its carnivore ancestry...

So they’ve still got the digestive system of a carnivore, they love meat, that’s how people trap them for tagging purposes and they can even be a nuisance for campers. But they choose to only eat Bamboo, a plant they can’t digest and that only grows in extremely remote areas.

So they solve this problem by just eating all day long. Fair play, who doesn’t like a binge. But just to dispel the cute argument for you too.

Elimination of waste occurs up to 50 times per day.

So if you're a panda, on average, you are taking a crap every 30 minutes of every hour, of every day.

And remember they can't digest bamboo so that's basically a tree they’re crapping out.

Someone just needs to have a word and we’ll let Pandas go.

Now, in stark contrast to the pointless Panda, the Great White shark is the Top Gun of fish. Which by the way is also heavily under threat from habitat destruction and the general impact of man.

Unlike the Panda, however, which has only been around a very short while, backing itself into an evolutionary cul-de-sac just 2m years, ago the Great White Shark has been owning the sea for 16 million years. And that’s because it’s descended from species of sharks that date back 100’s of m of years to the time of the dinosaurs.

The Great White Shark is the ultimate winning variant of the infinite multivariant split test of evolution. The armoured tank shark, the chainsaw mouthed shark, the hammer body shark they all died out because they were bad evolutionary ideas. (just like the panda) so the Great White Shark is the ultimate result of millions of years of failure. Which brings me on to actually what i’m here to talk about.

To be any good at anything you have to fail. Repeatedly. It's the ones that keep going after a failure that are successful.

So let's take the Great White Shark analogy just a little further, who is the evolutionary Top Gun in commerce today?

I’m sure for all of you, like me, there has been a fleet of poorly paid people in white vans turning up at your house and throwing Amazon packages at you from a Covid safe 2m distance.

And like me, I’m sure you’ve also ordered a bunch of stuff from places that aren’t Amazon. And how was it for you? Because I can tell you for me it’s been an absolute fucking disaster.

I could talk at length about my experience of ordering a fish tank from Pets at home or a paddling pool from Smyths. They were all shit which is when you realise just how good Amazon is.

And the reason they’re so good?

Like the great white shark, Amazon has been able to fail thousands of times, costing billions to make sure that it only keeps the stuff that works. Like delivering fucking fish tanks when they say they will.

It’s this epic scale of failure and of ruthlessly only keeping the products or services that are sharks and binning the Pandas that have allowed Amazon to get so big and so strong so quickly.

I know what you’re thinking, yes yes yes this is all lovely but we don’t work for Amazon, how do we do it? How do I go into work every day and murder Pandas?

Now Wayne’s already done a session on failure at a previous Drink Digital event and that most of the time you aren’t going to know why shit happens one way for one client and another way for another.

It just does. And that’s fine.

The key thing is not to place too many eggs in any one basket of failure and to be ready to grab back as many eggs as possible from any particular basket when you see someone’s going to kick it over.

So here is a real-life example of how I’m setting up to fail probably in the next 2-3 weeks.

So we’re currently 2,500 customers off our target (N.B. When I presented this I couldn't include the actual numbers in the slides for commercial sensitivity reasons) and the CEO and CFO are jumping up and down about how we’re going to close the gap. They have firmly landed this challenge at my door and I need to give them my plan tomorrow at 2pm.

The army has a saying that a plan never survives first contact.

So no matter how smart you think you are or how long you spend thinking through all the possible scenarios and outcomes, whatever you think will happen probably won’t. And when that happens you’ll need to change your plan, in the case of the army while people are trying to shoot you.

The inevitable problem comes when you don’t pay attention to that new information.

If you’re in the army and you ignore new information, there's a good chance you could end up dead. Thankfully for us, the risk of not paying attention to new information is lower which is why we’ve all come across people who say ‘that’s how we’ve always done it’.

Well no more!

If I’m trying something for the first time or if I'm looking for a different outcome I always try to dip my toe in first, figure out quickly what's going on and give myself the option to step back if I need to.

So we know the goal is 2,500 new customers, we know that whatever plan we come up with is unlikely to be right but what else have we got? Where are we today?

So we have:

1. A signup journey that converts at about 2% - Pants

2. A monthly digital budget of about £3k - Pants

3. A brand that’s been absolutely hammered recently - Pants

4. A product price that puts us about 40th in the market - Pants

So that's a quadruple stacker shit sandwich right there before we even start.

Ok, so let's take a look at the reality of our situation a bit more objectively and try to take stuff out of our own particular shit sandwich?

This is the bit where we go and get some data and figure out what we know and also what we don’t know so we can start to develop some hypotheses that we can test and hopefully not fail too hard at.

So the good news

• We’ve just acquired a new business that has a national presence - so we can remove some of that damaged brand slice from our shit sandwich before we start.

• They also have a different sign-up journey which currently converts at about 7% so that abysmal CVR% slice can be removed too

• We’ve just turned on a bunch of digital channels for that brand across search, social, and display so we do have some levers that we can play with and due to the challenge the CEO has given me I’ve now got some budget to use as well.

I’ve also taken a deep dive into the conversion rate (CVR%) performance and early results from the digital acquisition activity is giving us a cost per acquisition (CPA) of £10. - so it feels like plenty of headroom there.

So if my CEO wants 2,500 new customers even on today’s metrics he’s going to need to give me £25k to do that.

Me: So how quickly do you want these customers Mr CEO?

CEO: ASAP, please.

Me: Great.

Show me the money!

£25k please?

Hang on though - I’ll eat my hat if that £10 CPA scales. There are big players with very deep pockets in this market - Uswitch, Money Supermarket, compare the market as well as other energy players like Octopus.

What if as we scale our CPA goes up to £20? Now I need £50k or even higher £40 = 100k £50k = £125k

Feels like an easy way to fail big and lose my job! But I need to know what an acceptable CPA is.

So now I have a question for the CFO. tell me how high we could comfortably go in terms of our cost to acquire - £10 feels very cheap and if we were using a price comparison site (currently no point because we aren’t cheap) we’d pay between £60 and £90 for every new switch.

£30 feels like an acceptable number to me, but I want the money man to tell me.

So let's say that our CFO is happy to pay £30 for a new customer - awesome that feels like we can expand our digital channels and try to drive some volume with that extra £20 of CPA headroom.

Show me the money!

Now I need £75k please?

No chance.

You don’t become the CFO of anything by given some marketing dude £75k after a quick PowerPoint and some adding up.

So how do we structure this test?

From keyword research on ‘relatively’ non-competitive broad terms (some still have £10-£15 CPC for top of page positions), but to grab some of that traffic I would expect to need an extra £15k per month just for search.

But if we run it for a week and go top of page, then Mr CFO give me say £5k so we can hit this hard but in a very short and sharp stab to see what it delivers for us and what our actual CPA comes out at.

Show me the money!

Can I have 5k please?

£5k feels like a result for the CFO because if this goes catastrophically wrong I’ve not burned our budget or killed the business. Also, it shouldn’t cost me my job. Win/Win

Just one more thing.

If the CPA stays below £30 then I’m going to need more money.

So don’t get me wrong if I could deliver 166 sales in 1 week I’d be a hero. but if I COULD have delivered 332 then I’m a failure.

So rewind, rather than asking for £5k I’ll ask for £10k for 2 weeks' activity.

The CFO doesn’t open his wallet very often but his need is keenly felt now as he wants to close the gap. I also want some headroom with my budget, either to keep spending if it goes well or to try other activity if the CPA goes too high too quickly on paid search and I don’t want to have to come begging again for more money.

£10k still feels like a result for the CFO as it’s not £75k and if this goes catastrophically wrong I’ve still not burned our budget or killed the business. (or lost my job)

So for the record, I fully expect to fail here. I don't think we're going to magically acquire 2,500 customers at less than £30 from paid search, but I want to fail as fast as possible and gather the learnings so we can tweak our strategy.

As soon as our CPA starts to hit above the £30 target I’ll throttle back or even stop activity altogether if we really eat the budget.

Once we get to that point we take stock, our north star is 2,500 new customers so how many have we got so far and what has it cost us? Then based on that runway when would we expect to reach our target? Based on 100 a week about 6 months which doesn’t feel very exciting and we will likely lose other customers between now and then so the gap either stays the same or worse case grows.

Pants.

So what would happen if we took some of that price layer out of our shit sandwich?

If we took say £100 per year off our tariff price, where would this put us in the market? Would we move from C.40th to top 10? Even top 5?

What would that change in the market position do for our conversion rate and our CPA?

So Mr CFO, tell me how much margin we have and how much we could afford to sacrifice if it meant we could go back into the direct acquisition market but at a higher conversion rate that brings our our CPA back below £30.

Also, Is a CPA of £30 still ok if we're sacrificing £100 of margin?

This also adds another potential new addition to our shit sandwich as our existing customers could move off their current, more profitable product, to take our cheaper one. We could limit this to new customers only, but even negative feedback from our existing base could drive erosion of performance down the line that we might need to factor in.

Once I have the answer to the discount and CPA question, we can test launch a discounted tariff and we go again.

Same channels, same activity hopefully refined based on what we’ve already learned but now we offer a lower price. Obviously, price in a commodity market is a key driver of performance. Uswitch is called a price comparison site for a reason it’s not a lovely energy company comparison website.

Also, our main point of attrition through all of our acquisition journies is the tariff page ⅔ of people who see our price, currently, drop out of the journey. Offering a considerably cheaper product here is bound to drive improvements in conversion rate - right now this brand converts at about 7% but we don’t know what impact dropping the price will have.

Again I’m going to be hunting for the failure here. How hard can we push this and where is the sweet spot in terms of a consistent volume/value position?

We can dial up or down spend on channels, check conversion rate through the funnel and by channel, monitor product adoption and also keep an eye on existing customer base erosion and do it all incrementally without just setting fire to £75k.

If this all works I expect to spend way in excess of £75k and to deliver way more than 2,500 new customers - all being well this will become the new norm.

So I’ve whizzed through a process there but we all know that a plan is basically null and void the moment you start to execute.

So what are the key steps in this process and how do we attack them in a way which breeds positive failure.

Firstly the most important thing is knowing where we want to be - our 2,500 customer target. We know that the journey is going to be a nightmare but what I’m looking to do is break down this nightmare journey into stages that when we join them together will get us to our ultimate goal.

By breaking the journey down we’re always pointing in broadly the right direction to create our shark. And at each stage, we’re aiming to find the answers we need that allows us (or not) to move on confidently.

It's worth being very clear here. At every single point on this journey, we could completely change direction or stop completely. Each time we go on we test and learn and accept that we’re likely to fail but we have boundaries of what is and isn’t an acceptable failure point.

The thing with these incremental goals is it really helps to have people around you who’ve done it before or at least know one or two of the shortcuts. So for this project, we’re working with a digital agency to help us - cheers Aaron.

While we’re talking about teams for a moment, there’s an important point to make here about failing and what it can do to people. Failing is hard and no one likes to do it.

Our own evolution as human beings has hard-wired us to actively avoid failure as often as possible. And when you’re in a team or in a client-agency relationship doing something new, managing, and dealing with failure even if it’s a perceived failure, that’s felt by an individual can be tricky.

A good book on dealing with fear is Taming Tigers by Jim Lawless I’ll add it to the reading list at the end.

I don’t know the answer to pretty much anything.

If I’m left on my own to do anything I ‘might’ get there eventually but it’ll take me a while and generally be quite difficult and I’m far more likely to fail badly.

When you’re planning to fail It’s important to be clear with your team that you don’t know the answer and that you don’t expect them to know either. We’re just stacking the chips in our favour by working with people that might at least know one or two shortcuts to make the journey a bit faster and easier.

This is where honesty and trust are important with framing acceptable and unacceptable failure.

Going back to breaking down the challenge, it’s easier to help your people manage any sense of failure over the short term. Dealing with challenges in chunks that everyone in the team is responsible for. It can actually be exciting to fail when it's what you're actually looking for so you get an answer that you need to move on.

A good book on this The Five dysfunctions of a team - Patrick Lancioni - Again, I’ll put the reading list on at the end

This example that I'm living through right now is pretty binary in terms of here's a numerical objective given to me by my CEO. But what if your objective is more opaque or what if you don’t even know what your objective is?

There’s a framework for that which is entirely based on failing fast.

The design sprint came out of Google Ventures as a way to help the startups that they invested in short circuit the process of testing and validating business models or product market fit without the need to spend time, money and effort spinning up what could just be a useless Panda.

There’s a phrase. ‘Mothers love their own children’ which for anyone who has had a snotty, screaming, jam covered baby thrust at them from a doting parent will attest; it really doesn’t matter how ugly your kid is, you still love it. For the record, this is my daughter really not wanting to be dressed as R2D2.

There are many examples particularly in business, where a beloved idea, campaign or even a business is desperately kept on life support by whoever came up with it because they love it.

Even though everyone else just looks at it and goes wow, that’s fugly.

The design sprint methodology helps you be more objective than you might otherwise normally be with your ideas - basically it helps you figure out how ugly your baby is before you fall in love with it. And it does this in as quick, lightweight and most importantly as objectively data-led way as possible.

The design sprint process is crammed into 5 intense days Monday to Friday. As a team, you lock yourself in a room with some pens, paper and a challenge and go through cycles of divergent and then convergent thinking to come up with a testable prototype that you are going to put in front of real humans on the Friday.

Because you have to book in the testers who are normally real members of your target audience before you start, there is a palpable sense of urgency and also fear that you won’t have something your potential customers will be able to use.

This is the fear of failure. But it’s a good thing in this situation because the stakes are pretty low. It really helps focus everyone on the critical questions that they need answered by those customers and what the business needs to learn. It’s a really great way to strip the requirements of what you create to the absolute bare bones and makes sure that the test you structure with the prototype tells you exactly what you need to know.

I’ve used the design sprint methodology probably 20+ times to help formulate, test and validate all sorts of different things, from new propositions, branding, digital products and even a corporate intranet. It’s an exceptionally powerful framework and over time becomes a way of thinking and way to assess all sorts of different challenges. It also helps refine critical thinking e.g. I’ll run mini sprints with my team for half a day when we need to double down on a campaign or particularly challenging piece of work that needs collective and critical thinking and effort on a single or few items rather than playing endless circular email tennis.

It means we get more shit done and more shit right.

Of course, there is a Design Sprint book and again I’ll share all of these books at the end

This process of creating a very lightweight prototype leads me neatly into the next way you can fail.

The Pinocchio or Pretotype

We all know the story of Pinocchio, the boy who looked and acted like a real boy, but was actually made of wood.

Well, this is exactly the purpose of the Pinocchio and could actually be what you create at the end of your design sprint. A replica or mock-up of the real thing but which is actually dumb. Dan Cobley who is on the board at my previous place OpenWrks and who was the former MD of Google UK always used to trot out the palm pilot example when he talked about the Pinocchio or the prototype.

I don’t remember the palm pilot personally but then most 14-year-olds don’t need a… whatever the palm pilot was.

Anyway, the Pinocchio for this product was literally a block of wood and a pencil. The creator would carry it around in his pocket and just tap on it like he was using it. He was testing the form factor, so how it felt in his hand, how heavy it was in his pocket and also whether people looked at him like he was nuts when he used it in public.

While I was at OpenWrks and following a design sprint we created an interactive but dumb version of our proposed SME platform in InVision. We bought some cheaply branded t-shirts and took it to an SME event to get real feedback from people and also to see how they interacted with it, what they thought of it, what features they wanted, and most importantly how much they would pay for it.

Unfortunately, even with numerous design sprints and people making all the right noises in

market testing, our platform didn’t work out, and like the palm pilot is now dead.

Beware confirmation bias.

Even with perceived rigor and objective data points, because we are human we are always on the lookout for data points that back up our own views. Fake news anyone?

The Minimum Viable Product

A minimum viable product is a step beyond your Pinocchio.

Once you’ve validated that whatever idea you’ve had might be a good one and not a Panda, you don’t immediately start building the ultimate version. We’ve all been involved in projects particularly in the corporate world where they go on for years and never get delivered.

The MVP is designed so you deliver something that solves the challenge you have but is an incremental process that can change and adapt as more insight or data comes in from the market, the customer, or business priorities.

I’ve actually just launched an MVP of our quote journey. This has been created to fix the abysmal conversion rate we get. I won’t show you the old sign up journey as it’ll melt your face, but the new journey even though it’s an MVP has still got a punchy target of doubling sales.

Technically it’s a step backward, as it has less integration with our backend system but that meant we could ship it sooner for less money and (hopefully) realise the benefit of 100% more sales sooner.

We’ll then take an incremental approach to improvement and are already moving onto V2. development.

The MVP is such a powerful tool because even if what I’ve delivered is a car crash the sunk cost is not excessive and the time frame to deliver was three months, not 12 months.

As you can see from the last few examples all of this activity is aligned to gathering learnings as quickly as possible. Failing is an accepted risk, but both the chance and scale of the failure is so much more reduced.

This MVP example brings me neatly onto the next way to fail.

The Manual Turk

This is a really interesting story, the manual Turk was a traveling entertainer who claimed to have a machine that could beat anyone at chess. Customers would pay money to have a go, sit at the table in front of the pieces, and when they made a move watch in wonder as the machine moved the opponent pieces.

In reality, while it looked amazing it was actually just a very good chess player in a box with a magnet.

So the manual Turk is a way to develop your MVP or to imitate complex functionality that could take time effort and cash to build but that you want to test with real people first.

The new MVP quote journey that I’ve launched by the nature of its lack of integration with our system will require team members to manually carry out some of the work. But we can make a decision about which of those manual processes we want to remove or automate or enhance or even add new ones in if we want to test customer appetite for a particular service or offering etc. before we build anything clever.

The obvious caveat here is people. The more people you have involved doing stuff the higher the likelihood that the stuff they do could be dumb.

The provincial

This is a simple one that we can do anytime. Simply limit the audience to a specific location, great in physical retail where you can test new products in limited numbers of stores but I used this to test and validate proposition, messaging, acquisition costs and conversion through Facebook in 3 local target markets before considering rolling out nationally.

In fact, the experiment I ran through earlier to drive new sales for Robin Hood Energy I’ll be testing locally in Nottingham for the Robin Hood Energy brand alongside the national roll out of our other brand.

Just one more before I wrap up.

The false door

A useful digital tactic, but probably my least favourite tool. It’s basically a button or click to nothing. Normally a coming soon page.

It allows you to track clicks and interaction with your platform or service to test appetite and demand for features or propositions way before you actually commit to building them in your product.

Again we used them to test and validate hypotheses about product development but just beware of leaving them live for too long as you erode trust amongst your customers if your product is just full of dead ends.

So there we go.

I've stepped you through an in-flight example of how I'm looking to fail my way to success over the next few week and some ways you can test and fail in your own business.

In summary, the five things to remember are:

1. Know where you want to get to

2. If you don't know where you want to get to, use a design sprint to test quickly

3. Break down your challenge into smaller steps and constantly challenge your destination based on any new information, learnings or insight that you gather along the way.

4. Hunt for failure everywhere, do it fast and as cheaply as possible

5. Being scared is ok, but do it anyway.

Thanks for your time and happy failing.

1. Taming tigers - Jim Lawless

2. The five dysfunctions of a team - Patrick Lencioni

3. The design sprint - Jake Knapp, John Zeratsky & B raden Kowitz

4. The subtle art of not giving a f*ck - Mark Manson