Recently we devoted an entire article to the current state of DCAA publications. We noted quite a bit of activity in revising audit programs, but considerably less activity with respect to updated audit guidance via MRDs. In particular, we noted that the GFY 2016 Annual Report to Congress was late—as measured against publication dates in prior years. Within a week of our article’s publication, DCAA had remedied its problem, issuing the GFY 2016 Annual Report to Congress on its website.

This annual report, like the previous five annual reports, paints a rosy picture in which all signs point to an effective audit agency that’s on an uninterrupted course of improvement. For example, the annual report points out that sustainment of questioned costs (QC) in GFY 2016 matched the historic high set in GFY 2013, with nearly 53% of all QC being sustained. In another example of good news, the annual report states that the backlog of (adequate) incurred cost submissions awaiting audit was at an historic low of 4,677, down from last year’s backlog of 7,380. And the time it takes to perform incurred cost audits, according to the annual report, has dropped from 406 days in GFY 2012 to a quick 138 days in GFY 2016—a reduction of 66 percent!

Seems like very good news, indeed.

The only problem is that the numbers don’t seem correct.

As readers may know, we maintain a database of historic DCAA statistics, collected from the DoD IG’s Semi-annual Report to Congress and the more recent DCAA Annual Report to Congress. We track those stats and, from time to time, we publish them along with our thoughts on what they may mean for government contractors. For example, our most recent discussion of audit statistics can be found here. In that article, we asserted that the sustention rate was closer to 24 percent, in contrast to the DCAA report that claimed it was north of 50 percent.

The DoD OIG Semi-Annual Report to Congress is published, as you may well guess, every six months. Thus, to cover the same ground as the DCAA Annual Report to Congress, you need to look at two reports. Let’s do that!

The first report covered the period 01 October 2015 through 31 March 2016 (first half of GFY 2016). Looking at Appendix F (“Status of Action on Post-Award Contracts”) we see that, at the end of the period, there were 1,714 open audit reports with questioned costs worth $11,557.8 million (call it $12 billion). The audit reports cover the gamut of DCAA audit activity, from defective pricing to CAS compliance to incurred costs. Thus, the purpose of Appendix F is to put a value on DCAA audit reports for which a contracting officer follow-up action is required. The reports are aged, so that readers can see that 1,099 of the open reports are languishing and that the cognizant COs are late with their dispositions. The information regarding reports and dates and status is pulled from DCMA’s Contract Action Follow-up (CAFU) database.

More importantly, the Appendix F statistics also show closed reports—i.e., reports that were dispositioned within the reporting period. With respect to those closed reports, we see that 412 audit reports, cumulatively worth $1.7 billion, were closed in the first half of GFY 2016. We also see that the amount of QC that was sustained was $447.4 million—for a sustention rate of 26.0 percent. Footnote 9 reinforces the analysis. It states, quite clearly—

Contracting officers sustained $447.4 million (26 percent) of the $1,722.3 million questioned as a result of significant post-award contract audits during the period. The contracting officer sustention rate of 26.0 percent represents a decrease from the sustention rate of 31.3 percent for the prior reporting period

Okay. But that was only one half, not the whole year. So let’s look at the next Semi-Annual Report to Congress, the one covering 01 April through 30 September 2016. Appendix F tells a similar story. In the reporting period, 458 audit reports cumulatively worth $2.1 billion were closed—with a sustention rate of 22 percent. Again, Footnote 9 states—

Contracting officers sustained $468.9 million (22 percent) of the $2,140.2 million questioned during the period as a result of post-award contract audits. The contracting officer sustention rate of 22 percent represents a decrease from the sustention rate of 26 percent for the prior reporting period.

We did a rough average of the two reports (without weighting) and came up with a GFY 2016 sustention rate of 24 percent, not 52.5 percent as reported in the DCAA Annual Report to Congress. Where is the difference coming from?

DCAA is counting pre-award (“forward priced”) sustentions in its statistics.

Remember, the DoD OIG statistics only track “post-award” audit reports—i.e., the audit reports issued after a contract has been awarded. The statistics are not tracking pre-award audit reports, such as reviews of contractor cost proposals that may lead to a contract award. As DCAA itself notes, “forward pricing audits net the highest rate of return.” Thus, by including QC identified prior to contract award, DCAA is inflating the sustention rates.

Assuming for a moment that it would be correct to conflate pre-award QC with post-award QC, why would the sustention rates between the two be so radically different?

The answer to that question is easy: the COs have little or no incentive to disagree with DCAA. The COs are using DCAA’s audit reports as the basis for their negotiations, and they can (and will) use any QC to negotiate a lower price. There is little or no downside for them—no contractor is going to litigate a pre-award argument about QC. In contrast, once a contract has been awarded, the Contract Disputes Act comes into play. Contractors have the right of appeal, and COs know that. Further, there is more peer review and management oversight regarding disposition of DCAA audit reports in the post-award environment. The DCMA CAFU database tracks and ages only post-award audit reports. For all of these reasons (and perhaps more) the sustention rate of pre-award QC is going to be dramatically higher than the sustention rate for post-award QC.

Clearly, DCAA should be reporting the two sets of statistics separately. Until it starts doing so, the sustention rates reported by DCAA will always be inflated, and will never match the sustention rates reported by the DoD OIG. We suspect, however, that DCAA will be most reluctant to separate the two datasets unless Congress forces it to do so.

We also want to point out that DCAA has rejiggered the incurred cost audit duration statistics. Did you notice that the numbers look weirdly lower than they ever have before? What’s going on?

According to Figure 9 of the annual report, “The time to complete an incurred cost audit is the time between the entrance conference date and report issuance.” No, it’s not. For the previous five annual reports, incurred cost duration has been measured from the time the assignment was started to the time it was completed. However, starting in GFY 2016, DCAA changed the measurement. The new measurement is misleading, because (as every contractor knows) the period before the entrance conference is when the DCAA “risk assessment” takes place. That risk assessment is hard to differentiate from an actual audit, isn’t it? It’s filled with Request for Information (RFIs) and data calls and auditor questions—and it seems to take forever. By rejiggering the audit start date, DCAA has dropped its audit duration from nearly three years to a matter of a couple of months. The Note to Figure 9 states—

Had we maintained our previous measure to calculate elapsed days based on receipt day, our elapsed time would still reflect a 25 percent reduction from FY 2012 to FY 2016 (FY 2012: 1184 days, FY 2016: 885 days).

That Note doesn’t state that the incurred cost audit duration reported in GFY 2015 (Table 3) was 883 days—so it actually took DCAA two days longer in GFY 2016 to complete its incurred cost audits (on average). Granted, two days is a de minimis difference, but we think it’s important to note that fact.

Another DCAA-reported statistic that’s strange is the backlog of final billing rate proposals (“incurred cost proposals”) awaiting audit. In the GFY 2015 Report to Congress, DCAA stated (page 7) that its backlog at the end of GFY 2015 was 11,758 submissions, comprised of 8,979 “adequate” proposals and 2,779 proposals for which adequacy had not yet been determined. However, in the GFY 2016 Annual Report, DCAA stated (Figure 5) that its backlog at the end of GFY 2015 was 7,380 submissions. Clearly, some metric changed, because both figures cannot be true—but we can’t tell what magic DCAA did that made the numbers change. However, we are sure that some magic was performed, because the values reported in GFY 2016 Figure 5 do not match the numbers that we’ve been reporting over the years.

DCAA concluded its GFY 2016 report with a request for more auditors. The agency noted—and we agree—that headcount is down. At the end of GFY 2016, the audit agency had 4,023 auditors, down from GFY 2015’s level of 4,304. That’s a significant headcount drop. DCAA noted that the current hiring freeze imposed by the new Presidential Administration imperils its ability to continue to work down its audit backlog.

We here at Apogee Consulting, Inc., are not worried about that. Not in the slightest.

Regardless of headcount, we have confidence in DCAA’s ability to manipulate numbers, change metrics, and execute other bureaucratic tricks to make the audit agency seem like it is a great value for the U.S. taxpayer.