Lighter Side of SDLC

I ran across this extraordinarily technical description of System Development Life Cycle (SDLC) risks and thought everyone interested in improving processes and controls should be familiar.



Definitely take notes.

-- Prescott B. Coleman, CIA, CISA

Information Organization

Twenty million dollar IT projects don't come along all that often.

This one provides a tremendous example of how effective information organization (discussed in my earlier blog - “Credibility”) makes all the difference.

In a multinational insurance and financial services company there are some truly unique business challenges. For example, if I have an agent in the UK who writes coverage for a South African client on a facility in the US. Who gets the premium? In what currency is the premium to be exchanged and how will the policy be valued? As of what date will we mark the currency transaction and how will we get ceded premium back to the country that earned it? These are some of the easier-to-track challenges.

Don't worry, you don't need to understand any of this to understand this post.

To resolve these issues, a project, we'll call it “Global WorkSpace,” was initiated (names have been changed to protect the innocent). It was a massive IT effort to sort all this out and it ostensibly had a budget of $20 million. However, more was probably spent. Teams were working in the UK, US, and in some of the company's other 11 countries. As I recall, Accenture was engaged to help guide the Rapid Application Development (RAD) processes being used.

After chugging along for a period of time, declarations of victory were being made. According to the project leadership in the UK, all was fine, things were coming online, and they were poised to deliver on-time and on-budget. Unfortunately, to the US senior leadership, these claims didn't exactly match their on-the-ground understanding. While you might think this is a simple project management problem, determining the condition of this globally integrated system was almost as difficult as the obstacles it was built to resolve.
So my Business Controls Consulting (BCC) team, a wing of Group Internal Audit, was tapped to go have a look. Our charge was to make plain the true status of the project. I had auditors working for me in the US and the UK.

While what we found was fascinating from an IT controls and SDLC perspective, the point of this post is how we chose to describe the information we uncovered.

After more than six weeks of studying, living with, and testing the project, we had a pile of information.

It was true. Some bits were tested and working well. Other functionalities weren't as operational as they were made out to be. A few weren't even close. We needed a way to dispassionately tell this story while doing our best to preserve international business harmony.
So, we chose to do it graphically and back it up with written detail.

Looking at the deliverables from the user's perspective, we defined four major functionalities and a pile of sub-functions. Each was given a rating – Red, Amber, Green. As you might imagine, green areas were working properly and red ones weren't. We drew linkages between the major functionalities to denote dependency and interconnectedness and got it all on one page. The illustration on this page is a mock-up. The real diagram is proprietary.

This was the core of our report and it allowed the US senior leadership team to confidently engage with the World Group Office (WGO) on the real steps needed to get the project up and running.

Simply put, while the back-up data was immense, getting it on a single page converted it instantly from information to understanding. People can do things when they understand.

-- Prescott B. Coleman, CIA, CISA

The War Room

Auditors need dashboards too.

In an earlier blog post (Dashboards), I discussed the joy of auditing parts of the organization that were actively using trustworthy dashboards. Even if these dashboards aren't comprehensive, the thought process required to establish and review them drives a control-minded management team.

But our audit clients aren't the only ones who need dashboards. So do we.

Why?

Right off, I can think of two reasons. First, we need to live what we preach. If we can't demonstrate highly effective control processes, how can we hold our audit clients to the same standard?

Second, demonstrating a well-controlled audit process to the audit committee and senior management drives credibility. Credibility drives appreciation for our efforts. That appreciation drives additional funding, interesting audit projects, and is, frankly, the gateway to making a difference.

The War Room strategy that we invented at RSA is a great illustration of an effective dashboard process. Along with a number of other techniques (coverage targets, planning memorandums, control self appraisals, service level agreement metrics, and scheduling tools), a War Room can be an excellent way to demonstrate a well-controlled audit process.

You see, at the time, Home Depot sold 4x8 sheets of melamine for about $15 apiece. This is the stuff from which white-boards are made. Stealing the idea from my days on NEODATA's production floor and combining it with some tactics we observed from Accenture, we took a corner of our office and covered the walls. We titled each board with an organizational risk area and it became the responsibility of the audit managers and their staff to keep the boards updated.

At any time, the Chief Auditor, Audit Committee, President, and Senior team could see what we were working on, the status of our last audits, upcoming audit work, and the key risks that were keeping us up at night. We filled the boards with dashboard details.

But most importantly and beyond the information contained on the boards, the act of maintaining the boards drove a control focus.

You see, adequately updating the boards required our team to ...

• stay ahead of the audit schedule and plan
• plan and hold regular conversations with our audit clients
• develop an informal network to gather intel
• understand our client's true risk profile
• prepare to discuss and defend our understanding and opinions

Every one of these things is key to moving audit from a compliance role to being a trusted partner. And, they are critical to providing a deeply informed audit opinion.

- Prescott Coleman, CIA, CISA

Credibility

“Mr. President, I have something to tell you and I need you to listen to me, because I know something about the subject.”

Credibility is tough to achieve, but more valuable than... dare I say it... an upward moving stock in today's market.

Nevertheless, in my experience there are a couple of keys to building credibility with senior leaders.

Information Organization
I've already talked about organizing your message into no more than five points (see post from November 25th – Presidents have Five Fingers.) The point here though is that you've got to demonstrate you understand the information well enough to make it simple. Just like it is tougher to give a five minute speech than an hour one, distilling the facts into digestible bites (not sound bites, that would be wrong) shows you know what you are talking about.

Anticipation
I almost titled this piece “Preparation,” but that doesn't really cover the subject. I'm talking about getting yourself elbows-deep in the data. So deep, that you can accurately anticipate tough questions. When I used to analyze budgets to present to the City of Aurora City Manager and City Council, I'd comb through the detail at an unbelievably fine level of detail. Any variation from one year to the next, I investigated.

Why? Because we had a rule.

If a meeting with these folks lasted more than 15 minutes, we'd failed. And in my time there, we never failed. We did this by having the answer to the tiniest and most insignificant variance question. Once a City Council member asks about a 1.5% change in a fairly trivial line-item and you have the answer, they won't ask another.

No Surprises
Contrary to what Hallmark may tell you, all important people hate surprises. Credibility and lack of surprise go hand-in-hand.

Avoiding surprises is, like most things, a structured process.

Tell people what you are going to do. In an audit context, that means sharing as many of the objectives, steps, information needs, and people agendas as you can before you invade their space. Have the process for these structured. Engagement letter six weeks ahead. Planning memorandum four weeks out. Pre-visit conference call, two weeks beforehand. Entrance meeting. Post-visit update conference call. Exit meeting. Preliminary drafts of the report.
Put all of this in a Service Level Agreement. Live the SLA, and report honestly when you don't.

Equally important, when you have tough news to share, get ahead of it unofficially. I mean that all leaders respond better when you stop by their office and share, without drama, that you're pretty sure you or your team have found something and you just want to give them a “heads-up.” You might learn something you didn't know and they get time to digest it before it gets in print -- even in preliminary print.

Occasionally, I've been burned by this (e.g. managers rushing out to do damage control or whatever). But, on balance, this technique has yielded far greater positive results than negative. And, you get the chance to make a friend. No auditor or analyst can have too many of those organizationally.

Obviously, with some kinds of news you can't do this. Clearly, you won't stop by to share info about the fraud you think you've just detected.

Relationships Happen First
This sounds simple, but it is the easiest mistake in the world to make and I've certainly made it. Talk to your clients regularly and *before* there is tough news. Human beings are universally horrible at building relationships while in the middle of conflict. If the first time you've really had a good conversation with a senior leader is after they've received a tough finding, the game is lost. The same is true of any kind of reporting or delivery of critical data.

Be Right
And with the above techniques and strategies, I've left out a critical point.

Be right.

Double-check your data. Then double-check your conclusions.


--Prescott Coleman, CIA, CISA

Simple stuff

Controls and process improvement are a state-of-mind, almost as much as they are a discipline. And they don't have to be fancy.

It occurred to me that my Boy Scout Troop provides a pretty good example.

For a long time, the boys of my Troop complained to me that they no longer wanted to participate in District events. For the uninitiated, Scouting is structured like a corporation with large regions (Councils) and smaller local areas (Districts). Within a District there are probably 10-15 Scout Troops, with between 15 and 100 boys each. Usually twice a year, these Troops come together at a campout to compete against each other.

As you might imagine, the competitions include things like knot tying, first aid, and compass skills - the kind of stuff for which Scouts have been known for nearly a century. Each of these challenges take place at stations run by adult volunteers from each Troop.

Simple, right? Here's the issue.

They're usually a mess. The volunteer adults forget to write down times and scores. They lose the scraps of paper on which they put the points awarded. The terms of the challenges change throughout the day of competition. At the end, no one can figure out who won and awards go out to teams without regard to their true success. Very frustrating.

Moreover, this mess of a process has been common in Districts from Colorado to Texas to the Carolinas since I was a Scout (a few years ago, indeed).

Now, I'm fortunate to have some pretty high performing young people in my Troop and they'd had it. So rather than permanently abandon the idea of competition, we introduced a simple change. One ripped from the world of process and controls.

I've got a fellow in my Troop who is a professional leather worker. He makes saddles and tack. He provided scraps of leather, on which we stamped a 1, 3, or a 5. If a team attempted the event they got one point and if they finished it with flying colors they got five. Most importantly, they walked away with their points in-hand. No scraps of paper, no forgotten times. If they lost the token, too bad. At the end of the day, they presented their fist-full of leather and the team with the most points won. No arguments and no feeling of being cheated.Aside from the obvious work-flow improvement, this example highlights two key points. First, better controls don't have to be fancy. Second, large numbers of otherwise brilliant people will often muddle through a broken process for a very long time until a process-minded person comes along and looks at it differently.

The second largest fraud in history?

You know, I wanted to blog today about cool planning approaches, but I realized that not commenting on the arrest of former Nasdaq Chairman Bernard Madoff regarding a $50 billion fraud, might indicate that I'm not in touch with current events.

In any other year, the second largest fraud in history would be the only story. Indeed, the first largest (Enron at between $60 billion [Boston Globe] and $80 billion [Senator Mark Dayton]) occupied the headlines in 2001 and 2002 for months - and it spawned the Sarbanes-Oxley Act. Yet, so far, this story is just one dish on a complete menu of bad financial news. So much for, "lets not let this sort of thing happen ever again."

Of course, the biggest question any CIA or CISA should be asking is, "where were the auditors?" The answer is unpleasant even as it is uncomplicated.

According to Philip Broughton commentator for Forbes.com, there were two SEC audits, one in 2005 and one in 2007. According to the AP, the 2005 audit yielded three violations of rules requiring brokers to obtain the best possible price for customer orders, but the 2007 yielded nothing. His assessment is that this is a demonstration of the poor quality of these audits, and others are raising the same point.

Perhaps more importantly, the minimal (if not entirely absent) auditing capability of the firm that signed the financial statements should have been screaming out warnings. Though, according to Bloomberg.com's Dec. 13 story, at least one investment advisor had read the warning signs, Swiss private banks and otherwise conservative foundations were duped.

Bloomberg's story describes,

Hedge fund investment adviser Aksia LLC warned clients not to put their money with Bernard Madoff after learning of “red flags” at his company, including that its books were audited by a three-person accounting firm.

Bernard L. Madoff Investment Securities LLC used Friehling & Horowitz, an auditor operating out of a 13-by-18 foot location in an office park in New York City’s northern suburbs. The auditor signed off on Madoff’s annual financial statement through Oct. 31, 2006, according to a copy obtained by Bloomberg News.

And it gets worse, apparently there is some question whether this firm is actually an ongoing enterprise. According to Bloomberg's story,

Friehling & Horowitz operates from a storefront office in the Georgetown Office Plaza in New City, sandwiched between a pediatrician’s office and another medical office. An office for the Rockland County Bar Association is also in the building.

A woman who works in a nearby office, who didn’t want to be identified, said Friehling doesn’t come to the office regularly. When he does, he is the only person there.

Another woman in a nearby office, Leslie Cousar, said the man who comes to the auditor’s office does so for 10-to-15 minute periods, and wears tight pants and tie-dyed shirts. Cousar said she never saw anyone else going to the office during the day, but at about 5:30 p.m., another man would show up and use the location.

It's hard to not be infuriated by this kind of fraud-in-plain-sight and the damage it will do to an already staggering financial system. But, the fact that so many obvious indicators, including a high degree of secrecy and consistent returns even when they didn't make sense, just makes it worse.

Anyone still thinking a good place to save money is your audit department?

Audit Approaches - Speed of Change

A few years ago, we began to examine the concept of the Speed of Change in greater depth. We realized that it could be instrumental in shifting the thinking of auditors, who had grown up in a compliance environment, to a risk-based mindset.

To accelerate that mind-shift, we crafted a model that placed various kinds of audits on a continuum of change. Across the top we drew an arrow. On the left hand were control environments not experiencing much change or disruption. On the right were control environments either in such a state of flux or not even yet fully formed.



Doing this allowed auditors with a great deal of experience (and some with very little) to see where the kind of auditing to which they were accustomed fit in the overall scheme of things.

As our rate of change increased, we, of course, shifted our audit approach mix further to the right.

To illustrate, I've included a presentation, which we used to speak publicly about this concept. Enjoy.

How much is enough?

I think it was the late Peter Osterio, former chief executive of The Osterio Group (whose work is being carried on by Focus on Risk Enterprises), who first posed this three-part Socratic exercise,

1. If you had a warehouse full of fluffy teddy bears costing $1 each, how much would you pay to ensure they didn't disappear?
2. What if you had a warehouse full of stereos (today this is a better example with iPods), and
3. Would your answer be the same if they were diamond rings?

Of course, the step-by-step answers are something like...

1. about $1,500 for a chain link fence
2. at minimum a security guard, several forms of locks, and regular inventories
3. some serious stuff, up to and including very thick steel walls and biometric devices

And, really, the generalized answer is that it depends on what they're worth to you.

Now most audit programs ignore this basic point, but if you're going to get truly risk-based, you've got to get knee-deep in this question. How do you do that?

Well, this is the tough part and it requires some thinking out of the old TQM (Total Quality Management) world. In those days, as we do now in the Six Sigma environment, we spent a lot of time looking at acceptable-variation-from-standard and number-of-errors-per-1000 (million, gazillion, you get my point). Having spent some time as a production floor manager, this kind of stuff was pretty well known to me, but when I got into auditing I somehow left it behind.

I think it was after a particularly difficult conversation with a claims center manager that I began to dust it back off. The issue was, how many badly managed claim files equaled an out-of-control claim operation? Today, I don't recall how many we found, but my team thought it seemed like a lot and we began to write our report to say so. But seeming like a lot and actually being out-of-specification are two very different things.

Now, to be honest, the claim center manager had no more idea about whether he was within spec than I did, but of course he had a vested interest in not receiving a poor audit grade (I hate grading, by the way)... hence the difficult discussion.

Where this leads is the idea that long before the audit work gets underway, you've got to determine the acceptable control parameters. And, you've got to do it with management.
This gets particularly difficult in IT auditing, because in so many places it would seem that a single error is unacceptable. Info security is one major example. Software change management; transaction processing, balancing, and reconciliations; and financial output reporting are others. If an organization had unlimited resources, this would in fact be true. Most don't. Therefore, management has to define acceptable tolerances for key risks (in advance of the audit), you've got to compare them to your audit committee's expectations, and then use them as guides for your audit work and audit opinion.

What is really cool though, is when you match this kind of thinking with dashboards. What you have then is good old Western Electric-style statistical process control charts. They've been around since 1924, but somehow we keep forgetting them in audit.

That Ernst & Young Survey

You know you're on to something when CFO magazine picks it up and reports it.

I was delighted back on Thanksgiving day to have uncovered the release of the results of E&Y's recent survey on internal auditing around the globe. I picked it up off a finance and accountancy site in the UK (I guess not having a Thanksgiving - no British version of Plymouth Rock, you know - has its advantages). And, I've been even more heartened as I've seen the results popping up in articles in the weeks following.

Today, CFO Magazine has a piece on the survey at - http://www.cfo.com/article.cfm/12747841/c_12724274?f=home_todayinfinance - confirming what we believed in 2002 and 2003, that the pendulum had swung too far to one set of risks.

Probably the greatest alarm bell the survey sounds is:

More than a third of respondents said it was "very difficult" to recruit people skilled at enterprise risk assessment. Similar percentages said the same for auditing skills in specialized areas such as mergers and acquisitions, tax, and fraud detection. A total of 68 percent said it was either very difficult or somewhat difficult to find people knowledgeable about operational auditing or process improvement, compared to 51 percent for compliance auditing.

In 2003 at RSA, we were already concerned about the evolutionary impact that SOX would have on the profession. To a huge degree, we were able to segregate our SOX work from our risk assessment and assurance activities, but we watched many of our colleagues in the local IIA and ISACA chapters giving their entire Audit Plan over to SOX work. We knew then, and it appears validated now, that the skill set needed to conduct pure Sarbanes-Oxley work is a much different one than is required to properly assess an organizations portfolio of operational, financial, and information technology risks.

Kudos to E&Y for getting this back into the discussion.

Dashboards

I've always been a huge fan of dashboards and I'm not talking about in cars.

After all, the only reason the folks in Star Trek can fly all over the universe at breakneck speed is because they can tell you the temperature of cargo bay three at any moment from their seat on the bridge of the ship. When the bad guys start doing something down there, a little beep goes off at their console that lets them know to send expendable guys in red uniforms down to have a look.

So what does that have to do with anything?

When I conduct an audit and find the management has dashboard reports, charts, or whatever in place I know I've got folks who understand controls.

Indeed, I once reviewed a college admissions unit that employed dashboard reports (a combination of charts and numbers) for each of their eight colleges and nearly 100 majors. Each year they projected a trend based on history and computed each college's capacity for new students, and from this knew whether they were ahead or behind at any given moment. Seeing this level of management information, I was hardly surprised when I discovered they received close to 39,000 applicants for only a couple of thousand spots.

The best I've ever seen, outside the USS Enterprise, was the weekly output of the Enterprise Project Management Office (EPMO) at RSA. Each project at work within the US division had a single dashboard page in a “deck” of dashboards. Each, had an inset for planned deliverables over time. Right next to that were actual deliverables and variance. There were insets for budget, for issues, and ultimately a notation whether the EPMO believed the status of the project was red, amber, or green. The amber, as opposed to yellow, designation helped mark us as a British company.

The CIO summoned, each week, the project managers of any projects marked amber or red to explain themselves. It wasn't pleasant, I'm sure, but it was amazingly effective. Through an online repository, as IT Audit Manager, I had access to this entire “deck” and the supporting documentation on each project.

As a result, in terms of my audit opinion, once I audited the workings of the EPMO and found them sound, I could rely on these dashboards. This gave me tremendous amounts of breadth to my opinion, far more than if I tried to send a team of auditors to each project. And, prehaps more importantly, I knew where things were at the same time management did.

More on this in future posts.

Prescott Coleman, CIA, CISA

Fraud Indicators

One of the benefits of working as an internal auditor for a global insurance company is that you aren't the only one looking for fraud. Claim fraud and other kinds of insurance fraud are extremely important parts of the risk profile of an insurance company, so there is usually a team of investigators on hand to look into these things.

The good news for IA is that you have access to and training on this team's knowledge base regarding fraud indicators, which has been developed over many years. Below I've included some selected indicators from our training that, if properly modified, could be useful to other industries as well.

Obviously, the presence of these sorts of things doesn't indicate a fraud. Nevertheless, they can suggest greater investigation is needed.

Indicators often connected with property or auto:

1. Insured contacts agent to verify coverage or extent of coverage just prior to loss.
2. Misrepresentations and fabrications by the insured.
3. Suspicious comments and acts of the insured.
4. Insured acquires services of an attorney or public adjuster immediately after loss.
5. Insured threatens suit or use of an attorney immediately after loss.
6. Unreasonable pressure from insured for quick settlement.
7. Loss inventory indicates unusually high number of recent purchases.
8. Insured cannot recall place and/or date of purchase for newer items of significant value, and cannot provide bank or credit card records to substantiate these items.
9. Insured is unusually knowledgeable with regard to insurance terminology and the claims settlement process.
10. Insured handles all business in person, thus avoiding the use of mail.
11. Insured is willing to accept an unusually small settlement rather than document all claimed losses.
12. Attorney’s representation letter is dated same day as loss or shortly thereafter.
    No lienholder on expensive late model vehicle.

Indicators often connected with theft

1. Receipts are consecutively numbered.
2. Receipt amounts are in whole dollars.
3. Receipts show no tax or tax is incorrect.
4. Many expensive purchases in short period of time.
5. Receipts for common items are from a far distance.

And, I'll toss in a couple of my own that have served me well when auditing operations and IT. Sometimes these don't indicate fraud, just a weak control environment:

1. Manager is reluctant to let the audit team speak with their staff, or won't allow it to occur without them being present.
2. Files are missing. One version has the file is shipped away or destroyed as a matter of procedure.
3. Dates on security updates or settings changes on systems are just before the audit.
4. Manager keeps a set of everyone's passwords, "as a back-up."
5. Key individuals are known for heroically never taking vacation.

I encourage comments to this post and hope others will add their own most useful fraud indicators.

Prescott Coleman, CIA, CISA