DOG IG Auditor: I have some good news and some bad news.

DCAA Director: Please tell me the good news first, if you would.

DOD IG Auditor: Congratulations! You passed your external quality review!

DCAA Director: That’s great! Now tell me the bad news.

DOD IG Auditor: Your auditors still don’t know how to audit.

Really, the fictitious dialog posted above tells the entire story. You can stop reading right here and get back to Facebook or whatever.

Would you like to know more? Okay, we can help you with that.

First, The Good News

On August, 21, 2014, the DOD Inspector General issued its long-overdue peer review report of DCAA’s quality assurance system. As we’ve noted in the past, DCAA has been without an externally-reviewed-and-determined-to-be-adequate quality assurance system since 2009, which is technically a violation of GAGAS. That problem was rectified when the DOD IG issued its report, and DCAA auditors will no longer have to report the expired quality assurance system in their audit reports. The DOD IG found DCAA’s quality assurance system to be adequate and it received a rating of “pass with deficiencies.”

Yay for DCAA!

Some aspects of the quality system review report caught our eye. Perhaps you will find them to be of interest as well.

• Instead of the “usual” 1-year period from which audit reports would normally be chosen for review, the IG instead reviewed only audit reports produced in a 6-month period. The IG stated that “we believe the volume of audits at DCAA creates a reasonable sample in a shorter time.” Left unexplained was the rationale for choosing a 1-year review period in prior quality system reviews of past years, when DCAA’s production of audit reports was significantly higher than it is now.

• The IG reviewed 92 engagements. In 11 of the 92 (12 percent of the sample universe) the IG “identified errors or a lack of sufficient documentation … that limited the reliability of the reports. … Specifically, the DCAA engagement documentation did not contain sufficient information to allow the peer review auditor to understand the judgments and conclusions drawn by the DCAA auditor based on the evidence in the work papers.” A 12 percent failure rate is not so swell, in our view. In contrast, the DOD IG judged it to be good enough. (We are tempted to say “ good enough for government work.”)

• In addition, the IG identified 3 additional reports “for which the engagement documentation did not support information in the report.” However, for these 3 additional defective reports, “the reliability … was not affected by the errors because DCAA adequately resolved our concerns … during interviews and provided additional information outside the engagement documentation.” We think it interesting that engagement files and working papers did not have to speak for themselves, and could be augmented by discussions and additional information created after-the-fact. That would seem to be a very, very lenient audit approach and we are sure DCAA appreciated the professional courtesy involved.

To sum up, then, the external peer review of DCAA’s quality control system identified defects in 14 of 92 engagements reviewed, for an error rate of fifteen percent (15%). The DOD IG found that error rate to be within tolerance and passed the system.

Yay for DCAA!

As a side note, those of you lucky enough to have your business systems audited by DCAA may want to remember that a 15% error rate is good enough for a passing score. We mean to say that if a 15% error rate is good enough for DCAA’s quality control system, we think a 15% error rate should be good enough for your business system. Your auditors, of course, may not agree.

Now For The Bad News

On September 8, 2014 – just a couple of weeks after giving DCAA a pass on its quality control system – the DOD IG issued another review of DCAA audits issued during Government Fiscal Years 2012 and 2013. The findings were not pretty.

Readers may recall the previous DOD IG review of DCAA audit quality, issued in March, 2013. The findings in that report weren’t pretty either. We reviewed that previous report (link provided), and we concluded as follows—

We could go on and on, just like the DOD IG audit report did, listing example after example of poor audit planning, poor communication, poor documentation, lack of professional competence, lack of adequate supervision, insufficient evidence, delayed reports, and other GAGAS violations. But why bother? The report is, unfortunately for DCAA, damning.

Just as the prior DOD IG and GAO reports on DCAA audit quality have been damning.

We’re not particularly surprised by the findings in the DOD IG report; nor do we suspect our readership is particularly surprised by them. We’ve asserted for some time that the DCAA initiatives intended to increase audit quality have not worked out as planned. As this report demonstrates, DCAA audit quality is still lacking.

In other words, DCAA has implemented its revised procedures and multiple reviews and, as a result, has dramatically delayed its audit report production for no good reason. They still suck.

So we think DCAA may as well just throw the audit reports over the transom to the customer just as quickly as it can. The quality will still be as poor; but at least the reports will be more timely.

But we can hear the chorus of cries from Fort Belvoir from here—‘just wait until the next review!’ Yes, things will be so much better then. Higher quality audit reports issued faster.

Sure.

We’ll be very happy to report on the assessed quality of DCAA audit reports when that next external peer review report is issued. If things have improved significantly, we’ll be first in line to say so.

And so here we are, eighteen months later, discussing the follow-up report. It gives us little pleasure to tell you that, as we expected, the quality of DCAA’s audit reports—as measured by the DOD IG—has not improved significantly.

In the latest report, the DOD IG evaluated 16 DCAA audit reports, including 5 audits of priced proposals and 11 audits of “incurred cost proposals.” The IG identified “1 or more significant inadequacies” in 13 of the 16 audit reports evaluated, for a failure rate of 81%. Yes, that’s correct . More than 8 out of 10 recently issued DCAA audit reports failed the DOD IG review.

We bet DCAA is glad those audit reports weren’t part of the sample used for the external peer review of the audit agency’s quality control system!

Following are some quotes from the DOD IG report. Each numbered item corresponds to one of the 16 Memos the IG issued to DCAA after the review. Since we are only quoting from Memos that interest us, we have to skip around a little bit.

1. Audit Report No. 3321-2009K10180035. The IG found “that 66 percent of the invoices and 59 percent of the claimed costs audited … had already been examined in an assist audit conducted by the DCAA Iraq Branch Office, under Assignment No. 2131-2007R10180002-S1. … In addition, 91 percent of the costs questioned [in the audit report reviewed] had previously been questioned by the DCAA Iraq Branch Office. On May 30, 2008, the DCAA issued a Form 1 … on July 15, 2010, the contracting officer reached a negotiated settlement with the contractor. ... Therefore, the Resident Office’s efforts to later re-examine the same invoices and requestion the same costs did not serve a useful purpose.” Further the IG reported that “We found that the Resident Office reported $8,725,017 of questioned costs in the DCAA DMIS, even though the Iraq Branch Office had previously reported $6,128,000 of the same questioned costs in the DMIS. … The accuracy of DMIS information is important because DCAA frequently uses it as a management tool, and DCAA reports key statistics from DMIS to Congress and various federal agencies.”

3. Audit Report No. 4551-2009B1101001. The IG found that “the field audit office (FAO) spend an excessive number of hours auditing a billing system that is no longer in use, reported on transaction tests that were not current or relevant, and recommended the withholding of contractor payments without sufficient evidence.” The IG reported that “FAO auditors spent 7,416 hours to complete the audit.” Although the testing on the billing system was “comprehensive and well documented,” the IG noted that the FAO “expended an excessive amount of time testing the legacy system [that was no longer being used] and reported on the results of tests that were not current.” Looking at the delays between the performance of audit procedures and issuance of the audit report, the IG reported that “the FAO did not issue its report … until one year and six months after the last tested transaction. The oldest tested transaction was nearly four years old [by the time the report was issued].” The IG looked further, and reported “while the audited took only one month and 19 days to prepare the initial draft report, the FAO spent the remaining time (over one year and two months) performing several management/technical reviews, editing the report format, and incorporating the contractor’s response. GAO noted that the same FAO took two years to issue the 2005 report after completion of testing.”

6. Audit Report No. 2701-2006A10100002. The IG reported that “In Exhibit A (G&A), Note 6 and Exhibit G (Penalties), the auditor incorrectly identified and applied a penalty to an unreasonable training cost which is not specifically unallowable under FAR 31.205 and is therefore not subject to a penalty.”

9. Audit Report No. 4151-2005T10100004. The IG found that “The working papers did not include any support for how the FAO computed its reported penalty participation rates.” In addition, the IG asserted that the auditor had mishandled the impact of the contractor’s adjustment of two unallowable bonus costs charged as direct contract costs.

11. Audit Report No. 482102011R21000012. The IG thought this audit did not accomplish very much. It reported that “After expending 4,807 hours the DCAA Audit Report … did not meet the needs of the AMCOM contracting officer. The DCAA work papers do not establish that DCAA complied with existing DCAA policy and communicated effectively with the contracting officer.” As a result of the issues the IG identified, “the AMCOM contracting officer had to expend additional DoD resources and convene a post-audit report issuance fact-finding summit to make the DCAA audit report useable for negotiating the contract.” According to the IG report, “the contracting officer advised the OIG that the purpose of the ‘summit’ was to sit with DCAA to reconcile the gaps [between] the audit findings and the request for audit and to make the audit report useable for negotiating the contract. The contracting officer identified the omission by DCAA of the review of the proposed indirect rates as one of the reasons for convening the fact-finding summit.” Moreover, the IG again found problems with DMIS reporting and reported that “the final amount reported in DMIS for net savings, $18.9 million, was not calculated in accordance with DMIS guidance.”

16. Audit Report No. 1261-2007J10100537. The IG reported that “The FAO took 4 years to complete the assignment... The FAO manager told us [the audit] was delayed to focus on ‘higher-priority’ work. However, there was no indication in the working papers that the audit had been significantly delayed for other priority work. In fact, the auditor continued to charge the assignment intermittently over the entire 4-year period.”

To sum up the obvious, despite the recent “pass with deficiencies” rating on its quality control system, it seems quite clear that the vast majority of DCAA audits still fail. Some lack quality, some lack value to the requestor, some are late. Some audits suffer from all three attributes of failure.

According to the DOD IG, DCAA still has room for improvement. We concur in that assessment.

Sometimes we wish we were making this stuff up.

But no, reality is strange enough without having to resort to fiction writing.

We recently had a conversation with a DCAA auditor. It was a professional conversation, not at all any sort of “off-the-record” discussion that you can sometimes have if you are lucky enough to develop a rapport with your auditor. Nope. This one was official and formal, and the tone was serious. This one was for real.

And we think it illustrates all-too-precisely what in the hell has happened to the audit agency that was once held up as the gold standard for all governmental audit agencies.

We think it illustrates, quite accurately, why audits take so much longer than they used to take, and why so many more hours are necessary to push an audit report out the door. We assert that it illustrates, quite accurately, why GAO has expressed concerns with the quality of the DCAA audit reports. We suggest that it evidences the current preoccupation with working paper documentation in lieu of actual audit procedures, and why that preoccupation is a fatal flaw for the agency. We feel it is an indicator that auditor training is seriously lacking and, as a result, professional judgment is seriously flawed.

It was a short conversation, really. But it confirmed so much.

Perhaps we are biased and lack objectivity. Perhaps we are reading too much into a simple conversation between a junior level DCAA auditor and an entity being audited.

So we’ll ask you to judge for yourself. What do you make of the following conversation?

Email from Auditor: Hello. I have been requested by your ACO to perform a 19100 review of your client’s recent Disclosure Statement revisions for compliance.

Apogee Consulting: Hi there! How can we help you?

Auditor: Before we have our walk-through, I need you to fill out this list of individual Cost Accounting Standards. For each Standard, please tell me the date the Standard was effective, as well as the date the Standard was applicable, for your entity.

Apogee: [WTF?] We’re not sure what you mean. Our client is a large business that has been fully CAS-covered for many years. At this point, all Standards are applicable. Perhaps we are misunderstanding your request? Let’s have a telephone call to clarify what you need. Perhaps if you tell us why you need it, we can give you what you really need, since it can’t be what you asked for in writing.

(Now switching from email to phone conversation.)

Auditor: I need what I asked for in writing.

Apogee: As we told you, our client is a large business. It has been doing business with DOD for many years. It is fully CAS-covered. All Standards are both effective and applicable for the entity.

Auditor: I understand what you are saying. But even so, I need what I requested.

Apogee: Isn’t that effort just a waste of time? Look, how about we stipulate in writing that the entity is fully CAS-covered? How about we put in an email that all Standards are applicable to the entity?

Auditor: No good. That’s not what the audit program calls for.

Apogee: You mean the new audit program that just went into effect? The one that focuses on Disclosure Statement compliance instead of adequacy?

Auditor: Yes, that’s the one. I’m required by the new audit program to obtain a list showing the effective date and applicability date for each individual Cost Accounting Standard.

Apogee: You can’t be serious, right?

Auditor: Don’t believe me? Check for yourself. See Section C-1, Step 1, found on page 5 of 7, of the new audit program.

Apogee: [Stunned silence.] Yeah. We see it. It says “ Prepare a list detailing CAS applicability dates for each standard used to evaluate compliance of the contractor’s disclosed cost accounting practices (CAM 8-301.c.). Note: Use the list to determine if the Cost Accounting Standard is applicable, based on the date of applicability, before citing a CAS noncompliance .” So basically, you need to make sure a Standard is applicable to our client before you assert a CAS noncompliance?

Auditor: Yes.

Apogee: Well, that makes sense, we guess. But as we previously mentioned, all Standards are applicable to this client. We’ll be happy to put that in writing, if you like. A management “representation” or “assertion,” if you will.

Auditor: Nope. I need a list for my working papers. That’s what the audit program calls for, and that’s what I need to prepare. And you will prepare it for me.

Apogee: But there’s no information in that list! All we are going to do is just read the CAS and tell you what the regulations say the applicability date and effectivity dates are. It’s meaningless.

Auditor: I need what I need. Did you read the CAM reference?

Apogee: You mean the part that says –

c. To facilitate the implementation process, each promulgated standard contains in subparagraph .63 an effective date and an applicability date. The CASB defers the applicability date beyond the effective date in order to provide contractors adequate time to prepare for compliance and make any required accounting changes. Under the regulation, a contractor becomes subject to a new standard only after receiving the first CAS-covered contract following the effective date.

(1) The distinction between the effective and applicability dates is important. The effective date designates when the pricing of future CAS-covered contracts must reflect the new standard. It also identifies those CAS-covered contracts eligible for an equitable adjustment, since only contracts in existence on the effective date can be equitably adjusted to reflect the prospective application of a new or revised standard.

(2) The applicability date marks the beginning of the period when the contractor's accounting and reporting systems must comply with a new or revised standard. Proposals for contracts to be awarded after the effective date of a standard should be evaluated carefully for compliance with the new or revised standard. The proposal need only reflect compliance with the standard from the applicability date forward. Therefore, it is important that the auditor determine the applicability date of the particular Standard (including any revisions) under audit. Any change resulting from early implementation by the contractor is to be administered as a unilateral change. It will result in an equitable adjustment under FAR 52.230-2(a)(4)(iii) for the period prior to the applicability if the CFAO determines that the unilateral change is a desirable change.

Auditor: Exactly. So please fill out the list I sent you, identifying the applicability date and effective date for each of the nineteen individual Cost Accounting Standards.

Apogee: (Oh, dear Lord.) Um, sure. Will do. Should take about an hour. First, we’ll need to find our CAS book.

Auditor: Thank you for your timely cooperation.

And that, dear readers, is exactly what in the hell is wrong with DCAA these days.

EDITOR’S NOTE: For those readers who may not be familiar with the Cost Accounting Standards, there hasn’t been a new Standard promulgated since 1980. Since then, a couple of Standards have been revised (notably CAS 412 and 413), but by 2014 any given contractor is either (a) exempt from CAS altogether; (b) subject only to CAS 401, 402, 405, and 406; or (c) subject to all 19 Standards.

ANOTHER EDITOR’S NOTE: Wouldn’t it make way more sense to tell an auditor: “Hey, before you assert a noncompliance with CAS, make sure the contractor is actually, y’know, subject to that particular Standard? Have the contractor confirm in writing that the Standard is applicable to it before asserting a noncompliance.” You betcha. It sure would make way more sense to do it that way. But if you did it that way, you wouldn’t get such a pretty working paper for the audit file.

A THIRD EDITOR’S NOTE: When the auditor requests that the contractor complete the list, identifying both CAS applicability and effective date for the entity being audited, what in the hell does that even mean from an audit perspective? What if we lied and said something like “CAS 403 is not yet applicable to this entity, so you cannot assert a noncompliance with CAS 403?” How would the auditor know if we were lying or telling the truth? How would the auditor test the accuracy of the dates on the list? When the audit program calls for the auditor to determine the applicability of the Standards to the entity being audited, and then the auditors asks the entity to prepare the actual working paper demonstrating the applicability without testing the validity of the assertions, how does that lead to a quality audit?

A FOURTH AND FINAL EDITOR’S NOTE: Mike Steen, Technical Director at Redstone Consulting, who is quite knowledgeable about DCAA, thinks we are perhaps overreacting a tad. (As is our wont.) In response to our whining post about the new 19100 audit program on LinkedIn, he wrote, “I interpret this as nothing more than DCAA not wanting to express an opinion on actual practices as compliant with CAS and/or with the CAS DS. Stat[ed] differently, DCAA is once again narrowly defining its audit objective, otherwise this would be one more perpetual audit involving hundreds of transactions to provide sufficient evidential matter supporting a conclusion that actual practices were or were not compliant.” He may have a point.