How much is enough?

I think it was the late Peter Osterio, former chief executive of The Osterio Group (whose work is being carried on by Focus on Risk Enterprises), who first posed this three-part Socratic exercise,

1. If you had a warehouse full of fluffy teddy bears costing $1 each, how much would you pay to ensure they didn't disappear?
2. What if you had a warehouse full of stereos (today this is a better example with iPods), and
3. Would your answer be the same if they were diamond rings?

Of course, the step-by-step answers are something like...

1. about $1,500 for a chain link fence
2. at minimum a security guard, several forms of locks, and regular inventories
3. some serious stuff, up to and including very thick steel walls and biometric devices

And, really, the generalized answer is that it depends on what they're worth to you.

Now most audit programs ignore this basic point, but if you're going to get truly risk-based, you've got to get knee-deep in this question. How do you do that?

Well, this is the tough part and it requires some thinking out of the old TQM (Total Quality Management) world. In those days, as we do now in the Six Sigma environment, we spent a lot of time looking at acceptable-variation-from-standard and number-of-errors-per-1000 (million, gazillion, you get my point). Having spent some time as a production floor manager, this kind of stuff was pretty well known to me, but when I got into auditing I somehow left it behind.

I think it was after a particularly difficult conversation with a claims center manager that I began to dust it back off. The issue was, how many badly managed claim files equaled an out-of-control claim operation? Today, I don't recall how many we found, but my team thought it seemed like a lot and we began to write our report to say so. But seeming like a lot and actually being out-of-specification are two very different things.

Now, to be honest, the claim center manager had no more idea about whether he was within spec than I did, but of course he had a vested interest in not receiving a poor audit grade (I hate grading, by the way)... hence the difficult discussion.

Where this leads is the idea that long before the audit work gets underway, you've got to determine the acceptable control parameters. And, you've got to do it with management.
This gets particularly difficult in IT auditing, because in so many places it would seem that a single error is unacceptable. Info security is one major example. Software change management; transaction processing, balancing, and reconciliations; and financial output reporting are others. If an organization had unlimited resources, this would in fact be true. Most don't. Therefore, management has to define acceptable tolerances for key risks (in advance of the audit), you've got to compare them to your audit committee's expectations, and then use them as guides for your audit work and audit opinion.

What is really cool though, is when you match this kind of thinking with dashboards. What you have then is good old Western Electric-style statistical process control charts. They've been around since 1924, but somehow we keep forgetting them in audit.