How To Test Fewer Key Controls in a Sarbanes-Oxley Section 404 Project

The Opportunity to Rethink, Again

In 2007 the Securities and Exchange Commission (SEC) and Public Companies Accounting Oversight Board (PCAOB) have again issued documents urging companies and their external auditors to be flexible in applying the rules and guidance. Again, they are saying these reviews should be risk focused and top down, and again they are carefully contradicting myths and pseudo-rules invented by the big auditing firms and others.

They are also issuing slightly revised rules that remove some of the statements that helped prop up expensive and inefficient reviews and external audits in the past.

It is easy to feel overwhelmed by the complexity of the rules and the difficulty of interpreting the nuances of expression in the latest documents, but for most people, this is not necessary. The key point is that the regulators are inviting everyone to rethink what they have done in the past and arrive at new judgements about what is necessary. Even now there is no definition as to how much is enough, so it is all about using the new mood music in the renegotiations.

Key Controls: the Big Issue

At my course recently, the main issue people wanted to talk about was how to cut down the number of "key controls" they planned to test. In the jargon of Sarbanes-Oxley, a "key" control is not necessarily an important control. Your set of key controls is the set that, if tested and found to be operating effectively, gives sufficient assurance for a happy conclusion to the review.

For example, you might have 200 controls over a large scale activity but 5 of them are carried out toward the end of the process and confirm that the other 195 have done their work and that there are indeed no remaining reconciliation problems or other errors. Without the 195 earlier controls, there would be a huge number of errors coming through, and the 5 final checks would be little comfort. But if the 5 final checks usually find little or nothing requiring correction, then they alone might be enough for the key control set.

Most companies feel that they have in the past included far too many controls in their key controls set and therefore made work for themselves and their external auditors. Now they want to cut key controls out.

How To Cut Key Controls

The typical problem situation is that we have a set of thousands of key controls from an earlier year, collected with a very strong anticipation that these would become a standard set to be routinely tested according to an annual/quarterly cycle. We would like to cut that set down to size, but have to contend with the mindset of repetition and potential resistance from the external auditors. Here are some suggestions for how this might be done.

There are three periods during which key controls could be revised:

1. before we go out to review and retest,
2. while reviewing and retesting, and
3. next year when we take advantage of redesigned controls.

Even before setting out to review and retest, there is new information on which to base a revised selection, such as the new guidance from the regulators and views about possible mistakes made the previous year. Consequently, there are reasons that can be given for cutting out key controls:

• On reflection the risk involved is low and we simply do not need to test or continue documenting the control.
• Results from last year indicate that the risks in this area are such that we can reduce our requirement for testing this year, both in sample sizes and in the size of our key control set.
• Results from last year show that the overall level of error is low, and high level reconciliations and other last-check controls that simply confirm all is well are all we need to test this year. The preceding, low level controls can come out of the set.

Once you start reviewing your documentation and retesting, there are additional opportunities to remove key controls on the grounds that you can gain alternative evidence. Here are some reasons you can give for removing controls from the set:

• These controls can be dropped because earlier testing (e.g., inspection of documents) has confirmed that the inherent risk level is at or below the level we estimated in planning, so the need for controls evidence is lower.
• These controls can be dropped because inspection of error figures from a later stage in processing has already provided evidence as to the level of error.
• These controls can be dropped because we have uncovered more convincing, more detailed controls at entity level that we can rely on more than last time.

The following year, provided you made appropriate suggestions that were implemented, you can use those reasons again, but more so because the alternative controls are better value. For example:

• These additional controls can be cut out because we are now able to rely more heavily on some highly effective entity level controls (e.g., analytical review of performance using the "explanations before variances" method).
• Yet more controls can be cut out because we are relying more on newly automated controls that provide a final proof of correctness.

The "explanations before variances" method of analytical review of results is a stronger version of typical analytics. Typical analytical review involves looking at differences between the draft actual numbers and either last year's numbers or a budget. Having identified differences, you seek explanations for them. The weakness in this approach is that hindsight enables people to remember the explanations that work in the direction required to explain the differences, but other explanations that might be relevant are not remembered.

The stronger approach is to ask for the major things that have happened during the period, compute their likely impact on the numbers, and only then compare them with the draft actual numbers. Companies that base analytics on rolling forecasts may be doing this already.

Enabling These Changes

Don't forget that in addition to cutting down the key control set, you should also be shifting toward reliance on ongoing monitoring controls/assessment, powered by improved process health metrics, and more effective supervision conversations, all copied to the evidence database as well as fed up the line where it can help the business become more efficient and more reliable.

To do all this requires that the review/testing people are able to (1) change what they test as they go, and (2) make recommendations for control changes that will move toward a better design that is more efficient in every way, even though there is no deficiency in SOx terms. You will need to make necessary planning, training, and documentation changes to get people to think, and provide the flexibility to do so. You will also need to provide documentation that can capture the full range of evidence and resulting decisions, and prompt people to consider control improvements while keeping them separate from information about suspected deficiencies.

Finally, since this involves some innovation compared to previous years, the best project structure will be one with very rapid incremental delivery of completed sections of work. Forget the idea of having everyone throughout the company reaching the same point at the same time. You want to have small, pioneering teams completing narrow areas of work within a couple of weeks, right through to remediation, and learning rapidly from these complete experiences so that more and more increments can be kicked off and rapidly completed.

Summary

Most people agree that Section 404 compliance is still too expensive. Cutting out "key" controls is one way to reduce the cost and the regulators have provided some reasons for doing it. In addition, there are other reasons that have often been overlooked and underused. With no clear definition of how much work is enough, this is all helpful ammunition in the negotiations over what is truly key.