Information is NOT evidence.

In this post, I want to describe a scenario I’ve seen many times.

A business follows one issue or dataset without gathering more evidence.

Let’s explore an example: enquiries into a business drop and fewer people visit the company’s website. The marketing team looks into this and comes into a meeting with a plan.

It seems reasonable enough. They suggest increasing the advertising budget and improving the site’s content. Nobody plucked this answer from the air; there’s data behind it, and the person presenting it understands marketing far better than the business leader.

The marketing team did what was expected of them: they’d noticed a problem, investigated it and recommended a response. After all, they were hired for their experience so that the business didn’t need the leader to be involved with every detail.

The plan was approved.

Several weeks later, the data showed that website traffic had improved, but sales hadn’t. More people were seeing the company’s services without becoming customers. The original data had been correct; it just didn’t answer every question.

The real problem emerged during conversations with people who’d made an enquiry but gone elsewhere. They’d struggled to understand the company’s offer. Services had been added as the business grew, and the website now made perfect sense to those who already knew how everything fitted together. To somebody encountering it for the first time, however, it was confusing.

In this example, the company had treated low website traffic as evidence that too few people were finding it. In reality, the more important problem was what happened when they arrived there.

Such a difference can prove expensive. It can also highlight a serious flaw in the way a business operates. It’s all well and good producing reports and making sensible recommendations, but if the dataset is narrow and/or the results left unchallenged, assumptions are typically made to fill in the gaps and/or underpin a specific viewpoint.

Let’s explore another scenario.

A department’s output falls, and the manager believes motivation is the problem. The figures show that tasks are taking longer, and several deadlines have been missed. Staff are reminded of the standards expected of them, and closer monitoring is introduced.

What the report doesn’t show is that a new system requires employees to enter the same customer details twice. Each task now takes several minutes longer than it did before. Across a full working week, those minutes have become hours.

The performance figures are accurate; the conclusion isn’t. In this scenario, employees are being managed for a motivation problem when the real obstacle is a process that has added work without adding value.

This is what we mean when we talk about critical thinking. It’s not just about engaging your brain. Of course you have to do this, but it’s what you use your brain to do.

There are a few interpretations of the word ‘critical’ when looking at the dictionary definition. In this context, we’re taking an analytical stance, to which the definition becomes ‘using careful thought and logical evaluation’.

Evaluating an outcome often means measuring it against other possible outcomes. It also involves looking at the problem from all angles. It can mean asking more questions to understand context, and it almost certainly means not accepting things at face value.

Because information is not evidence.

For a business leader or founder, this statement may be unsettling. For a business to scale, decisions need to be made through other people’s accounts of situations they haven’t witnessed. The larger the company becomes, the less often the leader sees the original problem for themselves.

By the time an issue reaches them, it’s usually been turned into something manageable. The relevant figures have been selected and the possible causes reduced to a recommendation. That’s necessary; nobody running a business wants every report to arrive with the entire history of the subject attached.

Yet something can be lost in the tidying. ‘This might suggest’ becomes ‘the figures indicate’. Before long, the company is acting upon something the evidence never established.

More information doesn’t always solve this; it can actually make it worse. AI could also perpetuate this issue, because:

1) it only works with the data and from the lens you provide

2) It won’t tell you what’s missing, nor what information it has disregarded to reach its outcome

3) It can’t apply human experience or gut instinct, nor can it relate one problem to a similar issue that exists in a different context or setting unless instructed

4) It cannot innovate or find a new way out of a problem; it only draws on what has already taken place

Another flaw is how confident it can sound. AI can turn notes into a polished summary that reads as though the matter has been settled. These tools may organise the information perfectly, but they can’t compensate for an explanation that was too narrow at the beginning.

Meetings can take people further down a rabbit-hole. Once an apparently evidence-based account is on the table, the conversation naturally moves towards action. People discuss how the recommendation will be implemented and what it will cost.

Those are sensible conversations, but they take place after the most important judgement may already have been made. A decision might feel thoroughly considered because several people have contributed to it. What they may actually have examined is the proposed solution, whilst the explanation behind it has passed through the meeting untouched.

None of this means that every decision requires a forensic investigation. Leaders can’t remove uncertainty from commercial life, and waiting for ‘perfect’ evidence would create its own problems.

The issue is whether the people making the decision can see where the facts end and their interpretation begins.

That distinction becomes more important as businesses improve their ability to collect information. Nowadays, a company can know more about its activity than ever before whilst still misunderstanding what’s happening within it. It can measure what’s easy to record and overlook the information held in conversations, customer histories or another department’s systems.

The uncomfortable part is that weak reasoning doesn’t always sound weak. It’s often prepared by trusted people who’ve done everything normally expected of them. There may be no obvious error in the report - just an explanation that hasn’t been exposed to enough pressure.

Make examining an interpretation part of how your company works. Before asking what should be done, train your team to establish what else the available facts could mean. What information would challenge the explanation, for example, rather than continuing to collect material that supports it?

People become better at this when they have opportunities to test their judgement in situations where the answer isn’t supplied for them. They need to experience what happens when a convincing account is disrupted by evidence they hadn’t considered, especially after they’ve become invested in their original position.

This is exactly what a Battle of the Barristers workshop can simulate.

Developing people who can consistently scrutinise a problem is likely to prevent far more expensive mistakes in a business than another reporting process ever will.

For more information about Evolve3’s scrutiny training, contact info@evolve3.org.uk