DOG IG Auditor: I have some good news and some bad news.

DCAA Director: Please tell me the good news first, if you would.

DOD IG Auditor: Congratulations! You passed your external quality review!

DCAA Director: That’s great! Now tell me the bad news.

DOD IG Auditor: Your auditors still don’t know how to audit.

Really, the fictitious dialog posted above tells the entire story. You can stop reading right here and get back to Facebook or whatever.

Would you like to know more? Okay, we can help you with that.

First, The Good News

On August, 21, 2014, the DOD Inspector General issued its long-overdue peer review report of DCAA’s quality assurance system. As we’ve noted in the past, DCAA has been without an externally-reviewed-and-determined-to-be-adequate quality assurance system since 2009, which is technically a violation of GAGAS. That problem was rectified when the DOD IG issued its report, and DCAA auditors will no longer have to report the expired quality assurance system in their audit reports. The DOD IG found DCAA’s quality assurance system to be adequate and it received a rating of “pass with deficiencies.”

Yay for DCAA!

Some aspects of the quality system review report caught our eye. Perhaps you will find them to be of interest as well.

• Instead of the “usual” 1-year period from which audit reports would normally be chosen for review, the IG instead reviewed only audit reports produced in a 6-month period. The IG stated that “we believe the volume of audits at DCAA creates a reasonable sample in a shorter time.” Left unexplained was the rationale for choosing a 1-year review period in prior quality system reviews of past years, when DCAA’s production of audit reports was significantly higher than it is now.

• The IG reviewed 92 engagements. In 11 of the 92 (12 percent of the sample universe) the IG “identified errors or a lack of sufficient documentation … that limited the reliability of the reports. … Specifically, the DCAA engagement documentation did not contain sufficient information to allow the peer review auditor to understand the judgments and conclusions drawn by the DCAA auditor based on the evidence in the work papers.” A 12 percent failure rate is not so swell, in our view. In contrast, the DOD IG judged it to be good enough. (We are tempted to say “ good enough for government work.”)

• In addition, the IG identified 3 additional reports “for which the engagement documentation did not support information in the report.” However, for these 3 additional defective reports, “the reliability … was not affected by the errors because DCAA adequately resolved our concerns … during interviews and provided additional information outside the engagement documentation.” We think it interesting that engagement files and working papers did not have to speak for themselves, and could be augmented by discussions and additional information created after-the-fact. That would seem to be a very, very lenient audit approach and we are sure DCAA appreciated the professional courtesy involved.

To sum up, then, the external peer review of DCAA’s quality control system identified defects in 14 of 92 engagements reviewed, for an error rate of fifteen percent (15%). The DOD IG found that error rate to be within tolerance and passed the system.

Yay for DCAA!

As a side note, those of you lucky enough to have your business systems audited by DCAA may want to remember that a 15% error rate is good enough for a passing score. We mean to say that if a 15% error rate is good enough for DCAA’s quality control system, we think a 15% error rate should be good enough for your business system. Your auditors, of course, may not agree.

Now For The Bad News

On September 8, 2014 – just a couple of weeks after giving DCAA a pass on its quality control system – the DOD IG issued another review of DCAA audits issued during Government Fiscal Years 2012 and 2013. The findings were not pretty.

Readers may recall the previous DOD IG review of DCAA audit quality, issued in March, 2013. The findings in that report weren’t pretty either. We reviewed that previous report (link provided), and we concluded as follows—

We could go on and on, just like the DOD IG audit report did, listing example after example of poor audit planning, poor communication, poor documentation, lack of professional competence, lack of adequate supervision, insufficient evidence, delayed reports, and other GAGAS violations. But why bother? The report is, unfortunately for DCAA, damning.

Just as the prior DOD IG and GAO reports on DCAA audit quality have been damning.

We’re not particularly surprised by the findings in the DOD IG report; nor do we suspect our readership is particularly surprised by them. We’ve asserted for some time that the DCAA initiatives intended to increase audit quality have not worked out as planned. As this report demonstrates, DCAA audit quality is still lacking.

In other words, DCAA has implemented its revised procedures and multiple reviews and, as a result, has dramatically delayed its audit report production for no good reason. They still suck.

So we think DCAA may as well just throw the audit reports over the transom to the customer just as quickly as it can. The quality will still be as poor; but at least the reports will be more timely.

But we can hear the chorus of cries from Fort Belvoir from here—‘just wait until the next review!’ Yes, things will be so much better then. Higher quality audit reports issued faster.

Sure.

We’ll be very happy to report on the assessed quality of DCAA audit reports when that next external peer review report is issued. If things have improved significantly, we’ll be first in line to say so.

And so here we are, eighteen months later, discussing the follow-up report. It gives us little pleasure to tell you that, as we expected, the quality of DCAA’s audit reports—as measured by the DOD IG—has not improved significantly.

In the latest report, the DOD IG evaluated 16 DCAA audit reports, including 5 audits of priced proposals and 11 audits of “incurred cost proposals.” The IG identified “1 or more significant inadequacies” in 13 of the 16 audit reports evaluated, for a failure rate of 81%. Yes, that’s correct . More than 8 out of 10 recently issued DCAA audit reports failed the DOD IG review.

We bet DCAA is glad those audit reports weren’t part of the sample used for the external peer review of the audit agency’s quality control system!

Following are some quotes from the DOD IG report. Each numbered item corresponds to one of the 16 Memos the IG issued to DCAA after the review. Since we are only quoting from Memos that interest us, we have to skip around a little bit.

1. Audit Report No. 3321-2009K10180035. The IG found “that 66 percent of the invoices and 59 percent of the claimed costs audited … had already been examined in an assist audit conducted by the DCAA Iraq Branch Office, under Assignment No. 2131-2007R10180002-S1. … In addition, 91 percent of the costs questioned [in the audit report reviewed] had previously been questioned by the DCAA Iraq Branch Office. On May 30, 2008, the DCAA issued a Form 1 … on July 15, 2010, the contracting officer reached a negotiated settlement with the contractor. ... Therefore, the Resident Office’s efforts to later re-examine the same invoices and requestion the same costs did not serve a useful purpose.” Further the IG reported that “We found that the Resident Office reported $8,725,017 of questioned costs in the DCAA DMIS, even though the Iraq Branch Office had previously reported $6,128,000 of the same questioned costs in the DMIS. … The accuracy of DMIS information is important because DCAA frequently uses it as a management tool, and DCAA reports key statistics from DMIS to Congress and various federal agencies.”

3. Audit Report No. 4551-2009B1101001. The IG found that “the field audit office (FAO) spend an excessive number of hours auditing a billing system that is no longer in use, reported on transaction tests that were not current or relevant, and recommended the withholding of contractor payments without sufficient evidence.” The IG reported that “FAO auditors spent 7,416 hours to complete the audit.” Although the testing on the billing system was “comprehensive and well documented,” the IG noted that the FAO “expended an excessive amount of time testing the legacy system [that was no longer being used] and reported on the results of tests that were not current.” Looking at the delays between the performance of audit procedures and issuance of the audit report, the IG reported that “the FAO did not issue its report … until one year and six months after the last tested transaction. The oldest tested transaction was nearly four years old [by the time the report was issued].” The IG looked further, and reported “while the audited took only one month and 19 days to prepare the initial draft report, the FAO spent the remaining time (over one year and two months) performing several management/technical reviews, editing the report format, and incorporating the contractor’s response. GAO noted that the same FAO took two years to issue the 2005 report after completion of testing.”

6. Audit Report No. 2701-2006A10100002. The IG reported that “In Exhibit A (G&A), Note 6 and Exhibit G (Penalties), the auditor incorrectly identified and applied a penalty to an unreasonable training cost which is not specifically unallowable under FAR 31.205 and is therefore not subject to a penalty.”

9. Audit Report No. 4151-2005T10100004. The IG found that “The working papers did not include any support for how the FAO computed its reported penalty participation rates.” In addition, the IG asserted that the auditor had mishandled the impact of the contractor’s adjustment of two unallowable bonus costs charged as direct contract costs.

11. Audit Report No. 482102011R21000012. The IG thought this audit did not accomplish very much. It reported that “After expending 4,807 hours the DCAA Audit Report … did not meet the needs of the AMCOM contracting officer. The DCAA work papers do not establish that DCAA complied with existing DCAA policy and communicated effectively with the contracting officer.” As a result of the issues the IG identified, “the AMCOM contracting officer had to expend additional DoD resources and convene a post-audit report issuance fact-finding summit to make the DCAA audit report useable for negotiating the contract.” According to the IG report, “the contracting officer advised the OIG that the purpose of the ‘summit’ was to sit with DCAA to reconcile the gaps [between] the audit findings and the request for audit and to make the audit report useable for negotiating the contract. The contracting officer identified the omission by DCAA of the review of the proposed indirect rates as one of the reasons for convening the fact-finding summit.” Moreover, the IG again found problems with DMIS reporting and reported that “the final amount reported in DMIS for net savings, $18.9 million, was not calculated in accordance with DMIS guidance.”

16. Audit Report No. 1261-2007J10100537. The IG reported that “The FAO took 4 years to complete the assignment... The FAO manager told us [the audit] was delayed to focus on ‘higher-priority’ work. However, there was no indication in the working papers that the audit had been significantly delayed for other priority work. In fact, the auditor continued to charge the assignment intermittently over the entire 4-year period.”

To sum up the obvious, despite the recent “pass with deficiencies” rating on its quality control system, it seems quite clear that the vast majority of DCAA audits still fail. Some lack quality, some lack value to the requestor, some are late. Some audits suffer from all three attributes of failure.

According to the DOD IG, DCAA still has room for improvement. We concur in that assessment.