beyond evidence-based policy

“What this kind of dispute points to, is that it isn’t possible simply to say that policy should be evidence-based – research evidence can often come to very different conclusions.” (Ian Greener)

I put my hand up: I have called for “evidence-based policy” as if it were sufficient to contain ideologically driven policy, and in doing so fallen foul of HL Mencken’s maxim:

“To every complex problem there is a simple answer: neat, plausible, and wrong”

My three word answer – evidence-based policy – can be dismissed for a number of reasons, the first being that I haven’t explained what I mean by the term. A second reason is given in the quotation which opens this blog entry. Thirdly there is an unproven assumption that this answer is sufficient to contain ideologically driven policy, which itself begs the questions as to what that is and which ideology/ideologies I wish to contain.

Ian Greener sets about rescuing the answer as follows:

A way out of this is to use evidence in a different way, and try and aim for what we might call ‘argument-based policy’. Argument-based policy would ask researchers and perhaps more importantly, policymakers, not what their evidence is (that comes later), but instead to go through the argument for what they advocate step by step, and only when we they have made that clear should they present the evidence they have for each of those steps.

I would like to extend this, using processes and techniques from best-practise models of governance. Underlying these is a straightforward algorithm:

  1. Where do we want to be?
  2. Where are we now?
  3. How do we get where we want to be?
  4. How do we know we have arrived?
  5. Go to 1.

It’s necessary to distinguish policy from what I would call implementation. Policy is proven at Step 1, and implemented by Steps 2-4: but evidence AND argument are used at each of the Steps 1-4.

  1. This is where policy is tested. In business terms, the vision is proven against the business strategy (argument) and a vital test is that it is justified (evidence – consistent with the strategy and providing benefits)
  2. The current service is assessed against the output from 1 (evidence) and a gap analysis produced, showing where the current service does not match the required service (argument)
  3. An implementation plan is drawn up and approved (argument) which must be proven to close the gap from step 2 (evidence) and is then implemented
  4. To complete each step of the implementation plan, proof must be provided (evidence) that the intended outcome has been achieved (argument)

But this still looks like a neat, plausible, and wrong answer. A supporter of the Health and Social Care Bill might argue that they have done this:

  1. Been elected on a Manifesto
  2. Published and then gathered responses to Green paper(s)
  3. Published a White Paper and gathered responses. Published legislation
  4. Passed the legislation through Parliament

and I have not eliminated the (wrong) ideology because of the response to Step 1.

Taking the Health and Social Care Bill as the case study, as an opponent of the Bill I would counter:

  1. No you weren’t
  2. There might have been argument (or debate), but no-one really tested the evidence
  3. See 2. above. And anyway, this doesn’t match your gap analysis. This looks like your objectives aren’t what you say they are.
  4. Having a bunch of compliant Tory and Lib Dem MPs and Peers voting along party lines doesn’t count as a valid metric

In other words, no process works where the people owning and performing it are determined to circumvent it or to be misleading (which the ideological are). It seems our MPs and Peers are in general not fit for purpose.

So to avoid yet another simple, neat, plausible, and wrong answer to the question it is necessary to turn to another best-practise tool of governance: Root Cause Analysis. But before that, thoughts on Top-Down Reorganisations


2 thoughts on “beyond evidence-based policy

  1. Thanks for this piece. I agree with a lot of what you write, but think I disagree in two ways.

    First, I think policy needs to be broken down into separate arguments and even propositions beyond that to see what evidence there is for each, and also (and I didn’t say this in my piece) to see whether the bits have any chance of working together. For me, argument has to come before evidence – evidence is research that needs to be put to some kind of use, and that use is brought into focus (correctly or not) when it is used in policy.

    Second, I’m not sure the ‘policy-implementation-test-revise’ cycle really works for a large public organisation like the NHS. I think we need a series of goals to be agreed across the service, which themselves need to be based on clinical evidence, but for localities to be allowed to go their own way in finding the best way to meet those goals. Our country (and others) is very different from city to city, and we need greater flexibility in how we meet those needs – one organisational solution is not going to work. So policy can provide goals, but local areas need to be allowed to adapt and change to meet local needs. And that requires stronger local democracy (I agree with Phillip Blond (which is unusual for me) about this–graham-allen-we-need-a-magna-carta-for-true-local-government-6988803.html). Now that doesn’t preclude planning and measuring, but it does make it more complex and means it needs to be more dynamic.

    Thanks again. Look forward to trailed part two….

    • I certainly agree with your first point, and that is not well-addressed in this post. There are a number of issues starting with the confusion of policy with implementation (something I tried to distinguish in this piece). Perhaps because we now accept manifestos, press releases and political statements to be inaccurate or misleading, we are forced to reverse-engineer through analysis of the legislation, as with the Health and Social Care Bill. We need to demand that the arguments both justifying the need for legislation (policy) and supporting the implementation of it are cogent and coherent, and derive from the politicians’ ideology; and we need to demand that they explain that ideology, its logic, and its derivation. The Third Way and The Orange Book should be basic points of reference. The Tories need to provide one.

      The second criticism is a strong one, and I’ll begin to address that in mu second post on this topic, before looking again at root causes. The model I set out in this post is simplified; and I will explain it further and argue for its scaleability. I am making a leap here by inferring that government based on evidence-based policy will result in incremental change. That said, as I argue in the post, solution based on organisational change is not efficient or effective. Within a long-term framework of continuous improvement and solving of problems, organisation may be changed as (part of) the improvements required. Imposing reorganisation from above will not succeed.

      Where I will not concede is on planning and measurement of success based on agreed objectives. Without this, improvement becomes “change”. Change is not necessarily improvement: it is just change.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s