Recently the DOD Inspector General (DOD IG) released a July 28, 2010 audit report that criticized the Defense Contract Management Agency (DCMA) for “unsatisfactory conditions” related to how the oversight agency performed reviews of a defense contractor’s Earned Value Management System (EVMS).  The DOD IG criticized DCMA for three specific failings, as follows—

During two reviews of a DoD contractor in 2008, the Earned Value Management Center failed to (1) allow the Defense Contract Audit Agency (DCAA) sufficient time to perform an audit of the contractor’s system, (2) adequately resolve the DCAA findings, and (3) demonstrate independence and objectivity in fulfilling its oversight responsibilities.

Let’s peel the onion back a bit, shall we?

Both DOD oversight agencies (DCMA and DCAA) have been criticized heavily for both their dysfunctional relationship and their failure to accomplish their oversight missions.  We’ve posted several articles on the Commission on Wartime Contracting and its criticism of the agencies.  See, for example, here or here.  We even posted speculation that the two agencies might merge together.

We said at the time—

Reports are beginning to come out regarding the Commission's latest 2-day hearing on contractor business systems and DOD oversight -- and those reports indicate that the Commission thinks DCMA has failed in its oversight role and should possibly merge with DCAA in order to eliminate ‘dysfunctionality’ between the two DOD agencies.

For example, GovExec.com reports that ‘a lack of cooperation’ between DCMA and DCAA ‘is hindering the oversight of contractors' business systems.’ The article states that ‘members of the commission accused DCMA ... of almost uniformly ignoring recommendations from DCAA with regard to the business systems of logistical support contractors,’ and always siding with contractors by ignoring DCAA audit findings. …

The Commission called for the dysfunctional relationship between DCAA and DCMA to come to an end and, according to the report, put the onus on DCMA to resolve the problems. Co-Chair Shays was quoted as saying, ‘both of you up there, you're on the same team, but it doesn't sound like it and it doesn't look like it. ... With no disrespect to [DCMA], we think there needs to be more adjustment on DCMA's part than on DCAA's part. I think that's fairly clear.’

Recent proposed FAR revisions would affect how the DOD evaluates the adequacy of contractors’ “business systems.”  If the proposed rule is implemented as drafted (and rumors/gossip seem to indicate that some changes will be made), then there would be six (6) official business systems:  (1) Accounting, (2) Earned Value Management, (3) Estimating, (4) Material Management and Accounting (MMAS), (5) Property Control, and (6) Purchasing.  Some of those business systems would be the exclusive purview of the DCMA—e.g., property control and purchasing.  Others would seem to be the purview of the DCAA—e.g., Accounting and MMAS.  (Though it would appear that DCMA would retain the authority to issue final determinations of system adequacy—as it should.)  The authority/responsibility over the assessment of the other systems—notably Estimating and EVMS—would seem to be somewhat ambiguous.  And perhaps those two systems do require a joint approach to the assessment of adequacy, since costs and cost reporting accuracy play a prominent role within them.

But remember, DCAA has admitted publicly that “In fiscal 2008, the average time to complete a contractor pricing review was 28 days, compared with 72 days in fiscal 2010.”  For those in the know, DCAA reviews of contractor cost proposals are “demand audits,” and thus given the agency’s highest priority.  It used to be that such audits were to be completed in 30 days … but now they take more than twice as long as they used to—and that’s for the highest priority audits.  How long do you think DCAA takes to complete an assessment of contractors’ internal controls related to their business systems?

And remember that area for which DCAA has received the most criticism has been its review of contractor internal controls.  For example, in October 2009, we reported that the Government Accountability Office (GAO) reported “widespread audit quality problems” at DCAA.  As we reported at the time, “The GAO report stated that ‘we found audit quality problems … with all 37 audits of contractor internal controls and the 4 incurred cost and the 2 request for equitable adjustment audits we reviewed at 7 FAOs across the 5 DCAA regions ….’”  The Commission on Wartime Contracting reported that—

As a result of personnel shortfalls, DCAA system reviews and follow-ups are not always timely; therefore, the real-time status of contractor business systems cannot always be determined. As noted in our Interim Report to Congress, DCAA has not performed timely reviews of many contractor business systems.

The gadfly watchdog group, POGO, went so far as to send a letter to influential Senators saying, “We worry that these problems are indicative of a systemic strategy for reform that seeks to decrease congressional pressure rather than to institute meaningful reform.”  So given DCAA’s recent implementation of multiple layers of management review, its deferral of new system reviews—pending upcoming changes in how such reviews are to be performed—we invite readers to ask themselves how long DCAA actually takes to perform its evaluation of contractor internal controls related to business systems.  Perhaps you’ve experienced that duration first hand, as we have (many, many times).  Let’s ask how long DCAA should take to perform such reviews, how long DCMA should give DCAA to perform its share of joint system evaluations. What’s reasonable? 

A month?  Maybe a year?  How about a decade?

We can tell you, based on first-hand experience, that DCAA can’t perform such reviews in a month.  We don’t believe that anybody should expect those reviews to be performed in a month.  But certainly six months would seem reasonable, yes?  Most of us would expect DCAA to plan its work, perform its field work, and get a report reviewed within six months.  That seems reasonable—at least to us.

So when we tell you that you’ll be lucky to see a DCAA report evaluating the adequacy of a contractor’s internal controls related to a business system—a full scope audit—within the span a year, believe us.  A year is good.  A year is nominal.  Eighteen months would be highly probable.  Two years would not be out of the question.  Three years might be stretching it a bit.  Within five years?  Yeah, you’ll pretty much be certain of seeing that report within five years.  Virtual 100% certainty--but not quite 100%.  There may be some outliers….

Thus, contractors and other oversight agencies—notably DCMA—get frustrated at the slow pace of DCAA audits.  And DCMA starts looking to cut corners.  Does a Contracting Officer really need to see the DCAA audit report on the contractor’s proposed costs before sitting down at the negotiating table?  Can s/he afford to even wait that long before agreeing to a price and getting some weapon into the hands of warfighters in Southwest Asia who might desperately need it?  And can the DCMA functional specialists—the system evaluators—afford to hold up their official reports (the same reports that identify system deficiencies that require contractor corrective actions) for that period of time?  Or should they just go ahead and issue their report anyway?  After all, they have the final say in contractor business system adequacy.

Ponder those questions as we turn to the DOD IG report.  (Yeah, you were wondering when we would get back to the topic, weren’t you?  But we trust you’ll see how it all ties together.)

The DOD IG took DCMA to task for cutting DCAA out of their EVMS evaluation process.  First, a little background on that evaluation process, courtesy of the IG report:

The DCMA Contract Management Office … is one of 47 such offices established throughout the United States [with the authority and responsibility of determining the acceptability of the DoD contractor earned value management system under their cognizance.]  In addition, DCMA has established an Earned Value Management Center (the Center) to conduct initial ‘compliance’ reviews of DoD contractor systems for compliance with the ANSI/EIA-748 guidelines. The Center is currently comprised of a team of approximately 31 earned value management specialists who perform compliance reviews across the United States on a full time basis. The Center also performs follow-up reviews of significant deficiencies until they are corrected. Once the Center makes an initial determination that a contractor complies with the guidelines, the Contract Management Office will recognize such compliance through the issuance of either an Advance Agreement or a Letter of Acceptance. Thereafter, the Contract Management Office conducts ‘surveillance’ reviews on an ongoing basis to verify a contractor’s continued compliance with the guidelines. If a surveillance review detects significant noncompliances, the Contract Management Office may request that the Center perform another compliance review of all or parts of the contractor’s earned value management system and could withdraw acceptance of the system until the contractor reestablishes compliance.

The DOD IG reviewed two DCMA EVMS evaluations. The first was an April 2008 compliance review and the other was a follow-up review conducted in August of that same year.  “The April 2008 compliance review disclosed several noncompliances with the ANSI/EIA-748 guidelines. The contractor took corrective actions … and the DCMA follow-up review determined that those actions were adequate.”  DCAA participated in the initial compliance review but not the follow-up review. 

Somebody at DCAA didn’t like the way they were treated, so they made an Inspector General Hotline report. 

(Note that DCAA audit guidance encourages auditors to make such referrals when they encounter “mismanagement, a failure to comply with specific regulatory requirements or gross negligence in fulfilling his or her responsibility that result in substantial harm to the Government or taxpayers, or that frustrate public policy.”  See MRD 09-PAS-004(R), dated March 13, 2009, available at www.dcaa.mil, under “open audit guidance.”)

The DOD IG “substantiated” the allegations.  Specifically, the DOD IG found—

The Earned Value Management Center did not allow sufficient time for DCAA to meaningfully participate in the April and August 2008 reviews. The unreasonably short time frames effectively prevented DCAA from fulfilling its participation responsibilities … The Center established a 2-week time frame to complete a comprehensive review of the contractor’s system for compliance with all 32 ANSI/EIA-748 guidelines. As requested, DCAA participated in the review by auditing the contractor’s compliance with 8 of the accounting and financial related guidelines. Before receiving the DCAA audit results, DCMA held an exit conference with the contractor on … the last day of the 2-week period to advise them of the review results and provide a list of deficiencies. DCAA issued its audit report 6 weeks later … which reported additional noncompliances with 2 of the guidelines. The Center should have waited for the DCAA audit report before holding the exit conference with the contractor and providing a list of deficiencies. While the Center did consider an April 2008 draft of the DCAA report, the Center failed to adequately document its rationale for not upholding the DCAA findings … the Center should not have established an arbitrary and inflexible time frame to conduct reviews at all major DoD contractors. The established time frame should be based on a careful consideration of the risks and circumstances at each contractor location.

The Center established a 4-day time frame to complete the follow-up review. On August 12, 2008, DCAA notified the Center that it could not participate in the follow-up review because (i) the scope of the review had been restricted to the specific programs where the deficiencies were initially identified, (ii) the programs, cost reports, and period of time subjected to review had been selected and pre-announced to the contractor, and (iii) DCAA could not complete a follow-up review within the 4-day time frame established by the Center. Nevertheless, DCAA offered to perform a follow-up review in a timely manner that would still allow for incorporation into the final DCMA report. The Center did not adjust the 4-day requested due date for completing the DCAA follow-up. Rather than utilize DCAA, the Center used an ex-DCAA auditor on its staff to follow-up on the DCAA-reported noncompliances.

Let’s pause for a moment and consider whether the DOD IG might be on to something.  Was it unreasonable, as well as “arbitrary and inflexible” for DCMA to force DCAA to perform its share of the joint review within 2 weeks.  Well, yes.  There’s no way DCAA can comply with its existing audit guidance (both published and unpublished) in a 2-week period.  Either DCAA needs to change the way in which it performs its joint system reviews or DCMA needs to perform the reviews without the participation of DCAA.

In fact, we mustered the gumption to recommend to DCAA Director Fitzgerald that he might want to consider exempting some of DCAA’s audits from the rigors of GAO-interpreted GAGAS, just to avoid some of these issues.  We boldly recommended that Mr. Fitzgerald—

… consider whether all DCAA audits need to be subject to GAGAS. Reasonable people will disagree with GAO’s stringent definition of “independence” under GAGAS, but you can avoid the issue altogether if you make certain audits subject to GAGAS while others are not. There is precedent for this change: the AICPA has Consulting Standards that differ from Auditing Standards. Since DCAA performs both financial advisory services and audits, it would seem to make sense to apportion each type of audit into GAGAS-compliant and non-GAGAS-compliant groupings. And, by the way, DCMA really wants DCAA to participate in the process as an advisor; it wants your audits to offer value-added advice and to support the acquisition process. Contractors want to hear from auditors as well; they want to know where they need to improve and what should be done to fix system deficiencies. Your auditors can’t do this if GAO will allege they’ve compromised ‘independence’ whenever this happens—so change the rules of the game to eliminate the issue altogether.

What else did the DOD IG find?  The DOD IG found that DCMA compromised its own independence/ objectivity “by appearance” during the performance of the EVMS review.  The DOD IG reported—

First, the Center notified the contractor well in advance which programs and related cost reports it selected for review. As a result, the contractor’s compliance efforts could have focused exclusively on those pre-selected programs. Second, the Center did not vary the selection of programs reviewed. DCMA reviewed the same five programs during the April and August 2008 reviews as it did in a prior December 2006 review. Therefore, the Government has no reasonable assurance of the contractor’s compliance on other programs.

The [DCMA] representatives held joint activities with the contractor on two separate occasions which could have compromised DCMA’s independence and objectivity. In January 2007, following a December 2006 DCMA compliance review, DCMA … representatives participated in a week-long contractor review to identify root causes of various deficiencies and to develop a corrective action plan. In December 2007, representatives from DCMA … attended the contractor’s internal audit of its system in preparation for the April 2008 DCMA compliance review. DCMA officials charged with determining system compliance should not participate with the contractor in developing corrective action plans or performing internal audits. Participation in such activities may compromise the ability of DCMA to independently determine system compliance and continued acceptability of the earned value management system.

Moreover, the DOD IG asserted that “DCMA should not plan or conduct joint surveillance reviews with the contractor,” even though it admitted that “DCMA Instruction ‘Earned Value Management System (EVMS) System-Level Surveillance,’ January 2008, strongly encourages contractor participation in planning and conducting DCMA surveillance reviews.”

We have to disagree with the DOD IG on that one.  First, DCMA isn’t subject to rigid interpretations of “independence” under GAGAS and the IG shouldn’t hold the agency to such an inapplicable standard—especially when DCMA’s actions clearly fall within its own agency policy guidance.  Second, contractors and DCMA need to interact together, in order to identify root causes and appropriate corrective actions.  We wish DOD oversight agencies and their industry partners did more of that sort of behavior!  Finally, the IG report glosses over the fact that the same EVMS criteria that provide the standards for an adequate EVMS business system, the ones against which DCMA and DCAA were reviewing, were developed jointly by both industry and DOD officials, back in the days when such an approach was deemed to be in the public interest.

Talk about an independence problem!  Hey DOD IG—how about you deal with the historical fact that the “adequacy criteria” used for the reviews that you would like to subject to GAGAS-controlled independence were developed by the same people being reviewed, in concert with the people doing the reviewing. There’s a reason that’s not considered to be a real issue—and the reason is that none of the parties ever intended the oversight process to be subject to GAGAS—nor should it be.  Oops.  Missed that one, did you?

We aren’t the only ones to believe that the DOD IG missed the mark on this report.  The Director of DCMA, Mr. Charlie Williams, Jr., did not concur with much of the report either.  He told the IG—

• DCMA’s reviews lose their effectiveness and impact if DCMA’s results are presented to the contractor long after the review has been completed.

• The DCMA Director agreed that the DCAA findings of noncompliance were valid. However, DCMA did not consider the issues systemic. Therefore, the DCMA Contract Management Office worked with DCAA to correct the issues as part of the contractor’s Corrective Action Plan.

• DCMA has established and maintains a cooperative Government-contractor relationship that does not compromise the independence of their decisions. A Review for Cause is a focused review of specific elements of the contractor’s earned value management system in order to confirm the acceptability of the system. Advance notification to the contractor is necessary to ensure the contractor can support a review during the time period, key employees are available for interview, and the data requested is provided to the review team before the review.

• DCMA agrees with and is committed to ensuring that the decisions which result from its surveillance and compliance reviews represent an independent DCMA decision. DCMA does not agree that the only way to achieve this independence is by isolating the contractor from the review process.

Well, that was a breath of fresh, reasonable, air.  There yet may be hope for the DOD oversight process.  Oh, you want to see the DOD IG report for yourself?  Find it here.