Writing better reports

I've been writing reports that describe the state of affairs, explain the issues, and provide recommendations for about 17 years. Over that time, I've picked up a couple of bits that clearly make them more effective.

So, I thought I'd share.

Close the loop - Audit reports are, almost by definition, alarming things. However, highly effective reports ring the alarm bells, explain which barn is on fire, then talk about how the fire got put out.

They close the loop.

I know it sounds simple, but it is commonly missed opportunity. In my experience, audit committees and senior executives appreciate being told about an issue and then being told how it was handled. It means they don't have to take action, beyond monitoring the situation and asking for updates. It also means that they will get alarmed and active, when you bring them something that truly warrants their involvement.

This is a matter both of process and of report writing.

It is a matter of process, because to be able to do this, you must have actually worked with your audit client to get him/her to pick up a bucket and fill it with water. I have known a number of audit units that create findings, shoot them at their audit clients, and sit down to write the report. Any audit client worth their salt will want their side of the story in the written record. Now you've got potential conflict and disagreement. No one is focusing on making things better. And, your Audit Committee is wondering why you can't handle these things yourself.

It is a matter of report writing, because all too often I see auditors simply leave closure out of the report. They worked very hard with the audit client to come to a workable solution and they forget to talk about it. Sometimes I've found this to be the result of auditors who have a "gotcha" approach. However, more often it is just poor story telling.

Anticipate Questions - This is not an easy thing to do, because it requires placing yourself in the shoes of your audit client's superior. However, it is a skill-worth-paying-for and good auditors, who want to become great auditors, will focus on this.

For example, you've written a report with a finding regarding unauthorized access. It seems that for a number of months the Security Unit has been granting access to an important system based on the user's supervisor's approval. Evidently, the system owner hasn't been getting notified or asked for their approval. Worse, "super user" access has been granted to a few individuals. The "risk" statement in your finding talks about the potential for data being viewed or changed by inappropriate users. And, you've followed the tip above and closed the loop by describing what was done.

Not quite finished yet, though.

A really effective finding would anticipate the next question. Which I estimate would regard the scope of damage the unauthorized super users might have caused.

Perhaps you were able to find logs, which would have recorded any "super user" activity (and which couldn't be altered.) In that case, you allow the senior IT executive to sleep at night by saying so, somewhat comfortable that you've determined that no inappropriate activity was likely to have taken place.

Conversely, you may have found that no logs exist that describe what the "super users" might have done. In that event, it is your obligation to anticipate the question and make a statement that it is impossible to know. Niether the IT executive nor the Finance executive are sleeping now, which is probably appropriate.

Either way, you've delivered a more effective finding, because it targets the energy of the leadership.

Compliment - Finally, I find including in a report a brief statement of appreciation and compliment about the cooperation of the team and their leadership to pay significant dividends. It seems minor, and often can feel forced, but it has always been worth doing for me.

Something along the lines of,

The Internal Audit team wishes to express their appreciation and thanks for the degree of openness and cooperation received during this review. We found the staff and leadership of the unit to be genuinely interested in improving controls and making the organization more effective.

Another example,

While this report could include lengthy descriptions of the many strengths we identified, its purpose is to convey ways in which the unit can improve its level of internal control. Therefore, by design, the report focuses on areas of potential improvement. This concentration on areas of improvement should in no
way be construed as a diminution of the quality of the unit.

Strangely, while everyone knows this statement is somewhat perfunctory, it helps them save face. And as an auditor, when you help a manager save face you have the grounds for a relationship. Having an extraordinary network of relationships is how great auditors become amazingly effective auditors.

-- Prescott Coleman, CIA, CISA

Be Short

You want credibility? You want to be relevant? You want your audit function to be asked to the "Big Table."

Be short.

And, by this I mean deliver audit reports that are brief. In my last post on audit report writing, I talked about being succinct. Brevity and succinctness are related, but are not the same thing.

The fact that I used to write 100+ page reports for colleges and universities notwithstanding, I believe an audit report should be no more than 10 pages.

That includes executive summary, grade (if you have one), and all major findings. It can't be done, you say. It must be done, I say.

And, here are three principles to help you.

Aggregate - Find ways to roll multiple findings into one. You do this based on identifying a common factor, usually the Cause statement. If you have more than one finding with the same root cause, they are candidates for aggregation. You can also aggregate based on Recommendation. If the thing you propose as a solution will fix several issues, roll them into one. It should come as no surprise that the two easiest aggregating factors are Cause and Reco. As I pointed out in my post "Cause and the Last Why" they are linked like tires and rims.

Seriousness Ratings - I've seen a number of approaches to this. For example, Material Weakness, Significant, Important. Also, Critical, Major, and Minor. These, and a hundred others, are ways of classifying issues so that senior folks can tell the wheat from the chaff. There are usually definitions of the level of risk of the issue, but in the end it is a pretty judgemental rating. If you make a policy decision to report only the most serious findings and provide separate documentation on the minor stuff, your reports will get shorter and become more impactful.

Of course, there is a risk. The minor stuff might still be important and the tendency is for bits not in the report to disappear from radar. To combat this, let the client know that you are tracking all issues in your tracking database, but only reporting the serious ones. Not foolproof, but it helps.

Write Fewer Words - Yes, I am as serious as an overturned Greek ferry. I know it sounds obvious and simple, but it usually isn't. This principle has alot to do with being succinct, but I consider it a more direct prescription. But, how to accomplish this simple, yet monumental task?

In the first place, put a little angel or devil (you choose which) on your shoulder as you are writing. Their job is to gripe in your ear the whole time you are typing, saying mostly, "stop writing so much stuff." Then, when you are done writing, edit out 1/3 of what you wrote and revisit it. Try again until it hurts, then try one more time. Your reports will get shorter and, if you do it well, they will get more effective.

Of course, there is another way to drive fewer words. Turn your audit report into a table or powerpoint, so you can't fit anymore words in the boxes or slides. More on this in a later post.

-- Prescott Coleman, CIA, CISA

Be Succinct

Audit report writing is one of the hardest things to teach new auditors. This can be particularly true when working with team members rotated in from the business (see my posts on Rockstars) or when you've got auditors fresh out of school.

It seems, therefore, worthwhile to spend some blog space talking about some "important safety tips" as they regard report writing. I'll probably do several posts on this in April.

Let's start with being succinct.

Why? After all, those pesky management types have to read our reports, right? We're internal audit, we've got that cool Audit Mandate from our Audit Committee. They'll be hanging on our every word, won't they?

Not in this universe or the next, I'm afraid.

Succinctness (if there is such a word) is the art of getting to the point and it shows intelligence, preparation, and respect. Even if we could force our audit clients to read every word, we need to demonstrate each of these points to build credibility. And, credibility is king.

There are a number of ways to be succinct.

Destroy Clutter. One of my favorite books on the subject, On Writing Well, The Classic Guide to Writing Non-fiction, by William Zinsser, puts it this way:

Fighting clutter is like fighting weeds, the writer is always behind. New varieties sprout overnight, and by noon they are part of American speech. Consider all the prepositions that are draped onto verbs that don't need any help. We no longer head committees. We head them up. Writing improves in direct ratio to the number of things we can keep out that shouldn't be there.

Organize your thoughts. It costs alot of money to have senior people organize your thoughts for you. I've yet to meet an internal auditor whose time was more valuable than the people they were auditing.

I know, it hurts. Deal with it.

That means the company is, in fact, paying you to make things quick and easy for your audit client. If it means outlining, rewriting, reoutlining, and re-rewriting an audit finding to cause the reader to understand your point quickly - you should do so.

I find the Five Part Approach to audit findings (from IIA Practice Advisory on Standard 2410-1 - you know, condition, cause, criteria, effect/risk, and recommendation) a literal model for organization of a finding. Allow no more than one (maybe two) sentences for each. If you can be that structured, it helps the reader to look at multiple findings and know that the risk statement for the next finding will probably follow the criteria statement. If you can be that brief, your finding will probably get read.

Directness. Audit reports are clear and decisive statements. I can't even recall the number of audit findings I've reviewed that talk around the issue. Sometimes this happens because the auditor knows the subject so well that they don't remember to state it outright to the audit client. Other times, the issue feels scary and so the natural tendency is to "break the news" to the client. Both are a waste of precious attention span and, worse, can lead to misunderstandings.

Voice and Tone. Alot of report writers forget they need to pay any attention to voice and tone. For the uninitiated, these are the way the reader hears your words in their head. This blog has a particularly voice and tone that is different than the one I would use in an audit report. It is significantly more energetic and informal.

Now, I went to a pretty uncompromising liberal arts college for my undergraduate degree. Austin College is the sort of place where every course (including accounting) includes a major paper or thesis. Yet, they spent little or no time discussing voice and tone. So, chances are your auditors are largely unfamiliar with this concept. If you introduce it to them, it can be an eye-opener. It can take their writing from sounding like Ben Stein to sounding, more appropriately, like Peter Jennings.

One is laborious to listen to, while the other can be a delight - even when he was delivering bad news.

More thoughts on report writing in coming posts.


-- Prescott Coleman, CIA, CISA

How to attract Rockstars

In the previous post, Rotate - Why Rockstars Should be in Audit, I made a pretty compelling case (I thought) for adding your company's "rockstars" to your audit team.

Of course, by Rockstars I mean the heavy hitters of your business. They are engineers in an engineering company, architects in an architecture company, doctors or RNs (a case could definitely be made that the RNs are the real stars) in a medical organization, and loan officers in banks.

Certainly, a business may have more than one kind of Rockstar and you want some in audit.

How in the world do you attract them?

For the basic answer, you have to look at the reasons they may not want to come to your corner of the organization in the first place.

Like most things, reasons are pretty simple. If your audit function has a reputation for being police-ish, you'll have a tough time. If your audit unit is irrelevant, it won't happen. If your audit department is staffed primarily of fresh-out-of-school young people, the odds are lower. And, if your audit department is a career dead-end, Rockstars won't touch you with a 50 foot red pen.

Now we have something to work with.

In my experience, Rockstars like anyone else want to be sure their career decisions advance their career. Silly them. As a result, they will be on the lookout for taking roles that damage their reputation.

Now, we could take each issue in turn, but its obvious that you'll want to start working on fixing the above - stop being police-ish, become relevant, increase the experience level of your staff, and avoid being a dead-end. I'd be bored if this post stopped here.

Rather, lets talk about some positive attractants. These bits have been known to ensare a Rockstar or two, in my experience.

• See the World - If your audit function is structured well, your people will have a unique perspective on the company. You get to see it all. You can offer to a Rockstar the opportunity to wander the halls of the entire organization, meet people, and understand the company in ways they never would from their engineering cubicle. At RSA, seeing the world wasn't just a figure of speech. We had one Rockstar get seconded (a British term for "borrowed") to New Zealand. As far as I know, she is still there.

• Meet the President - Exposure to senior leaders can be a tasty carrot. When you present your quarterly audit findings to the Audit Committee or the Senior Leadership, ensure that you bring along your Rockstars (maybe one at a time). Introduce them. They may just sit quietly next to you, but they are getting exposure that their colleagues are not. If they are ready, let them present a portion of the report. With a reputation for making these kinds of connections, Rockstars will line up at your door.

• Credibility - This is not an easy thing to offer, because it takes a long time to build. But, if your audit function is seen as a strategic asset, one that is helping to advance the company's competitive advantage, this can rub off on Rockstars that work with you. Build this, and they will come.
  

• Pre-arranged Return Agreements - I'd suggest not accepting a Rockstar without having a hard agreement for them to return to the business at a specified point in time. While this is important to the Rockstar at hand, it is critical to being able to get others in the future. It also signals that you really are borrowing them for their career development and not yours. You may have to make these agreements with some very senior people. Nothing is worse than two years later finding that all the players have changed and no one will accept your Rockstar back.

• Skills, skills, skills - I have found that promising to make a Rockstar a better writer, a better communicator, and a better analyst often can be very compelling. Most Rockstars have limited opportunity to develop skills outside their discipline. Yet, they understand that the top people in most industries are great communicators and wide-thinkers. Promise them that you'll give them these skills (and deliver), and the next one will be easier to recruit.

As a final thought, start small.

A junior Rockstar is better than none. They tend to be more eager, easier to dislodge, and they give you the opportunity to practice the above. Done right, eventually you'll reel in bigger fish.

-- Prescott Coleman, CIA, CISA

Privacy Auditing

Between 2008 and 2009, Privacy Management moved up two spots to #2 on the AICPA's annual review of Top Technology Initiatives.

The full list of the Top Ten was:

1. Information Security Management
2. Privacy Management
3. Secure Data File Storage, Transmission and Exchange (Formerly known as Securing and Controlling
4. Business Process Improvement, Work Flow and Process Exception Alerts
5. Mobile and Remote Computing
6. Training and Competency
7. Identity and Access Management
8. Improved Application and Data Integration
9. Document, Forms, Content and Knowledge Management
10. Electronic Data Retention Strategy

This is true even while an Ernst & Young Survey of last summer suggests that IT executives and chief audit executives still haven't become wise to the risk. Indeed, according to survey, "Sixty-five per cent internal audit chiefs do not recognise data privacy and IT fraud as a serious threat to their business."

Nevertheless, the lack of awareness of the risk doesn't change the nature of the risk. So, it was with this in mind that I went hunting for some good intel on auditing this risk for the benefit of my readers.

And I found a terrific source.

The American Association of Accountants (AAA) held a conference in Anaheim, California last year and I found the content of their presentation a nicely encapsulated primer for auditors to approach risk assessing and auditing Privacy.

I was particularly pleased with the slide that compares the various international standards on the subject. And, if the E&Y survey reflects your organization's awareness of the risk, the opening section presents a very compelling case for getting a bit more motivated about the subject.

Enjoy.




-- Prescott Coleman, CIA, CISA

Quick Update on Auditing the Cloud

She told us she would, and she did.

My post several days ago regarded a presentation by Leslie K. Lambert of Sun Microsystems that discussed auditing in a Cloud environment.

She told us she'd post the slides and indeed she has. The document is posted on the Denver ISACA website.

I've also repackaged it to appear here.

Isaca Issa Feb 2009 Final

Publish at Scribd or explore others:

All rights are reserved, so be sure to treat Leslie's (and Sun's) copyright appropriately.

Thanks Leslie.

-- Prescott Coleman, CIA, CISA

Auditing the Cloud

Readers, I enjoyed a nice luncheon yesterday put on by ISACA where the Vice President and Chief Information Security Officer of Sun Microsystems gave a talk.

Bucking the classic mobility trend of silicon valley, Leslie K. Lambert has been with Sun for 17 years and it is clear she knows her stuff.

She spoke on a number of topics, but one of the central themes regarded, "the cloud." For many, this is a fairly mature concept. Indeed she joked, as have I, that Web 2.0 is already in need of an upgrade to 2.5. Nevertheless, based on pretty weak responses from the crowd to her questions about cloud applications actually in use, this IT subject may be more something we all know about than something we are really doing.

http://www.go2web20.net/

But its only a matter of time.

Simply put, the cloud is comprised of services you may rely upon, but which are not run on your own premises - and which largely use the capabilities of the Web for delivery. From a personal perspective, I began engaging the cloud when I let my domain name expire, quit paying for an ISP for my old website, and began relying on Blogger to provide space for my thoughts. I did it again, when I shifted my personal calendar to Google Calendar.

Company's do it when the shift to tools such as Salesforce.com, which is a Customer Relationship Management (CRM) tool that lives within the cloud and is emblematic of the technology. Indeed, Bill Gates publically predicted the global use of something like the cloud in about 1998, as I recall. Of course, the idea that we wouldn't buy disks from the store and "load" the software seemed pretty scary then. "Who would have my data?, you're kidding right?"

Within the cloud, according to Leslie (and the profession) there are three categories of service.

• SaaS - Software as a Service. The personal tools (Google Calendar, Blogger, Jotform, etc.) that I'm using are generally regarded as SaaS.


• PaaS - Platform as a Service. The underlying Google App engine is the canonical example.


• IaaS - Infrastructure as a Service. Not dissimilar from using an external hosting environment, except that it uses cloud computing for the infrastructure. Amazon's online backup is an example of this.

Of course, for auditors the control questions these kinds of new approaches raise can seem daunting.

How do we make sure our data is safe? Can we be sure its not being tampered with? What is the service's reliability? While it is neat to be able to access our data from anywhere, what about access speed? Change management. Are we going to be held hostage? Business Continuity-disaster?

She had a number of very full slides detailing these and many more cloud-related risks.

However, as she went through them she began to ask whether these were "new" risks. Risks that IT auditors would, as she put it, "need to go back to school on."

The answer, by the end of her presentation was definitely, "no." In the end, though the tools and tactics are changing, the control objectives are not.

What was interesting, and it generated a bit of discussion before time cut it off, is whether the traditional 3rd party assurance methodologies on which we have relied in the past will be up to the task. For example, one camp suggested that the traditional SAS-70 regime will be innadequate in a cloud environment.

Indeed, there would appear to be a number of layers of responsibility in such a computing model. It may be necessary to dig fairly deeply to figure out all the infrastructure and application providers on which we would be relying.

A SAS-70 cloud, perhaps?

Or would our contracts require that we send our auditors into the cloud to obtain assurance. The VP from Sun suggested that, to an extent, this is what they did and it cost them money initially to demand a certain level of assurance. However, she did note that Sun provides much of the actual behind-the-scenes infrastructure to these services, so they have a unique position from which to view the risk landscape.

On the other hand, another camp suggested that just as the risks have not fundamentally changed, so too the tools for covering them remain the same. The argument was made that SAS-70's are customized tools to begin with, and need only receive normal adjustment to comprehend these emerging risks.

I'll resist the temptation to pull a Kent Brockman (The Simpsons news anchor) and say, "only time will tell." Instead, let me say that I welcome comment on this issue.

-- Prescott Coleman, CIA, CISA