Recently
the DOD Inspector General (DOD IG) released a July 28, 2010 audit
report that criticized the Defense Contract Management Agency (DCMA) for
“unsatisfactory conditions” related to how the oversight agency
performed reviews of a defense contractor’s Earned Value Management
System (EVMS). The DOD IG criticized DCMA for three specific failings,
as follows—
During
two reviews of a DoD contractor in 2008, the Earned Value Management
Center failed to (1) allow the Defense Contract Audit Agency (DCAA)
sufficient time to perform an audit of the contractor’s system, (2)
adequately resolve the DCAA findings, and (3) demonstrate independence
and objectivity in fulfilling its oversight responsibilities.
Let’s peel the onion back a bit, shall we?
Both
DOD oversight agencies (DCMA and DCAA) have been criticized heavily for
both their dysfunctional relationship and their failure to accomplish
their oversight missions. We’ve posted several articles on the
Commission on Wartime Contracting and its criticism of the agencies.
See, for example, here or here. We even posted speculation that the two agencies might merge together.
We said at the time—
Reports
are beginning to come out regarding the Commission's latest 2-day
hearing on contractor business systems and DOD oversight -- and those reports
indicate that the Commission thinks DCMA has failed in its oversight
role and should possibly merge with DCAA in order to eliminate
‘dysfunctionality’ between the two DOD agencies.
For example, GovExec.com reports
that ‘a lack of cooperation’ between DCMA and DCAA ‘is hindering the
oversight of contractors' business systems.’ The
article states that ‘members of the commission accused DCMA ... of
almost uniformly ignoring recommendations from DCAA with regard to the
business systems of logistical support contractors,’ and always siding
with contractors by ignoring DCAA audit findings. …
The
Commission called for the dysfunctional relationship between DCAA and
DCMA to come to an end and, according to the report, put the onus on
DCMA to resolve the problems. Co-Chair
Shays was quoted as saying, ‘both of you up there, you're on the same
team, but it doesn't sound like it and it doesn't look like it. ... With
no disrespect to [DCMA], we think there needs to be more adjustment on
DCMA's part than on DCAA's part. I think that's fairly clear.’
Recent proposed FAR revisions
would affect how the DOD evaluates the adequacy of contractors’
“business systems.” If the proposed rule is implemented as drafted (and
rumors/gossip seem to indicate that some changes will be made), then
there would be six (6) official business systems: (1) Accounting, (2)
Earned Value Management, (3) Estimating, (4) Material Management and
Accounting (MMAS), (5) Property Control, and (6) Purchasing. Some of
those business systems would be the exclusive purview of the DCMA—e.g.,
property control and purchasing. Others would seem to be the purview of
the DCAA—e.g., Accounting and MMAS. (Though it would appear that DCMA
would retain the authority to issue final determinations of system
adequacy—as it should.) The authority/responsibility over the
assessment of the other systems—notably Estimating and EVMS—would seem
to be somewhat ambiguous. And perhaps those two systems do
require a joint approach to the assessment of adequacy, since costs and
cost reporting accuracy play a prominent role within them.
But remember, DCAA has admitted publicly
that “In fiscal 2008, the average time to complete a contractor pricing
review was 28 days, compared with 72 days in fiscal 2010.” For those
in the know, DCAA reviews of contractor cost proposals are “demand
audits,” and thus given the agency’s highest priority. It used to be
that such audits were to be completed in 30 days … but now they take
more than twice as long as they used to—and that’s for the highest
priority audits. How long do you think DCAA takes to complete an
assessment of contractors’ internal controls related to their business
systems?
And
remember that area for which DCAA has received the most criticism has
been its review of contractor internal controls. For example, in
October 2009, we reported
that the Government Accountability Office (GAO) reported “widespread
audit quality problems” at DCAA. As we reported at the time, “The GAO
report stated that ‘we found audit quality problems … with all 37 audits
of contractor internal controls and the 4 incurred cost and the 2
request for equitable adjustment audits we reviewed at 7 FAOs across the
5 DCAA regions ….’” The Commission on Wartime Contracting reported
that—
As
a result of personnel shortfalls, DCAA system reviews and follow-ups
are not always timely; therefore, the real-time status of contractor
business systems cannot always be determined. As noted in our Interim
Report to Congress, DCAA has not performed timely reviews of many
contractor business systems.
The gadfly watchdog group, POGO, went so far as to send a letter
to influential Senators saying, “We worry that these problems are
indicative of a systemic strategy for reform that seeks to decrease
congressional pressure rather than to institute meaningful reform.” So
given DCAA’s recent implementation of multiple layers of management
review, its deferral of new system reviews—pending upcoming changes in
how such reviews are to be performed—we invite readers to ask themselves
how long DCAA actually takes to perform its evaluation of contractor
internal controls related to business systems. Perhaps you’ve
experienced that duration first hand, as we have (many, many times).
Let’s ask how long DCAA should take to perform such reviews, how long DCMA should give DCAA to perform its share of joint system evaluations. What’s reasonable?
A month? Maybe a year? How about a decade?
We
can tell you, based on first-hand experience, that DCAA can’t perform
such reviews in a month. We don’t believe that anybody should expect those
reviews to be performed in a month. But certainly six months would
seem reasonable, yes? Most of us would expect DCAA to plan its work,
perform its field work, and get a report reviewed within six months.
That seems reasonable—at least to us.
So when we tell you that you’ll be lucky
to see a DCAA report evaluating the adequacy of a contractor’s internal
controls related to a business system—a full scope audit—within the
span a year, believe us.
A year is good. A year is nominal. Eighteen months would be highly
probable. Two years would not be out of the question. Three years
might be stretching it a bit. Within five years? Yeah, you’ll pretty
much be certain of seeing that report within five years. Virtual 100%
certainty--but not quite 100%. There may be some outliers….
Thus,
contractors and other oversight agencies—notably DCMA—get frustrated at
the slow pace of DCAA audits. And DCMA starts looking to cut corners.
Does a Contracting Officer really
need to see the DCAA audit report on the contractor’s proposed costs
before sitting down at the negotiating table? Can s/he afford to even
wait that long before agreeing to a price and getting some weapon into
the hands of warfighters in Southwest Asia who might desperately need
it? And can the DCMA functional specialists—the system
evaluators—afford to hold up their official reports (the same reports
that identify system deficiencies that require contractor corrective
actions) for that period of time? Or should they just go ahead and
issue their report anyway? After all, they have the final say in
contractor business system adequacy.
Ponder
those questions as we turn to the DOD IG report. (Yeah, you were
wondering when we would get back to the topic, weren’t you? But we
trust you’ll see how it all ties together.)
The
DOD IG took DCMA to task for cutting DCAA out of their EVMS evaluation
process. First, a little background on that evaluation process,
courtesy of the IG report:
The
DCMA Contract Management Office … is one of 47 such offices established
throughout the United States [with the authority and responsibility of
determining the acceptability of the DoD contractor earned value
management system under their cognizance.] In addition, DCMA has
established an Earned Value Management Center (the Center) to conduct
initial ‘compliance’ reviews of DoD contractor systems for compliance
with the ANSI/EIA-748 guidelines. The Center is currently comprised of a
team of approximately 31 earned value management specialists who
perform compliance reviews across the United States on a full time
basis. The Center also performs follow-up reviews of significant
deficiencies until they are corrected. Once the Center makes an initial
determination that a contractor complies with the guidelines, the
Contract Management Office will recognize such compliance through the
issuance of either an Advance Agreement or a Letter of Acceptance.
Thereafter, the Contract Management Office conducts ‘surveillance’
reviews on an ongoing basis to verify a contractor’s continued
compliance with the guidelines. If a surveillance review detects
significant noncompliances, the Contract Management Office may request
that the Center perform another compliance review of all or parts of the
contractor’s earned value management system and could withdraw
acceptance of the system until the contractor reestablishes compliance.
The
DOD IG reviewed two DCMA EVMS evaluations. The first was an April 2008
compliance review and the other was a follow-up review conducted in
August of that same year. “The April 2008 compliance review disclosed
several noncompliances with the ANSI/EIA-748 guidelines. The contractor
took corrective actions … and the DCMA follow-up review determined that
those actions were adequate.” DCAA participated in the initial
compliance review but not the follow-up review.
Somebody at DCAA didn’t like the way they were treated, so they made an Inspector General Hotline report.
(Note
that DCAA audit guidance encourages auditors to make such referrals
when they encounter “mismanagement, a failure to comply with specific
regulatory requirements or gross negligence in fulfilling his or her
responsibility that result in substantial harm to the Government or
taxpayers, or that frustrate public policy.” See MRD 09-PAS-004(R),
dated March 13, 2009, available at www.dcaa.mil, under “open audit guidance.”)
The DOD IG “substantiated” the allegations. Specifically, the DOD IG found—
The
Earned Value Management Center did not allow sufficient time for DCAA
to meaningfully participate in the April and August 2008 reviews. The
unreasonably short time frames effectively prevented DCAA from
fulfilling its participation responsibilities … The Center established a
2-week time frame to complete a comprehensive review of the
contractor’s system for compliance with all 32 ANSI/EIA-748 guidelines.
As requested, DCAA participated in the review by auditing the
contractor’s compliance with 8 of the accounting and financial related
guidelines. Before receiving the DCAA audit results, DCMA held an exit
conference with the contractor on … the last day of the 2-week period to
advise them of the review results and provide a list of deficiencies.
DCAA issued its audit report 6 weeks later … which reported additional
noncompliances with 2 of the guidelines. The Center should have waited
for the DCAA audit report before holding the exit conference with the
contractor and providing a list of deficiencies. While
the Center did consider an April 2008 draft of the DCAA report, the
Center failed to adequately document its rationale for not upholding the
DCAA findings … the Center should not have established an arbitrary and
inflexible time frame to conduct reviews at all major DoD contractors.
The established time frame should be based on a careful consideration of
the risks and circumstances at each contractor location.
The
Center established a 4-day time frame to complete the follow-up review.
On August 12, 2008, DCAA notified the Center that it could not
participate in the follow-up review because (i) the scope of the review
had been restricted to the specific programs where the deficiencies were
initially identified, (ii) the programs, cost reports, and period of
time subjected to review had been selected and pre-announced to the
contractor, and (iii) DCAA could not complete a follow-up review within
the 4-day time frame established by the Center. Nevertheless, DCAA
offered to perform a follow-up review in a timely manner that would
still allow for incorporation into the final DCMA report. The Center did
not adjust the 4-day requested due date for completing the DCAA
follow-up. Rather than utilize DCAA, the Center used an ex-DCAA auditor
on its staff to follow-up on the DCAA-reported noncompliances.
Let’s
pause for a moment and consider whether the DOD IG might be on to
something. Was it unreasonable, as well as “arbitrary and inflexible”
for DCMA to force DCAA to perform its share of the joint review within 2
weeks. Well, yes. There’s no way DCAA can comply with its existing
audit guidance (both published and unpublished) in a 2-week period.
Either DCAA needs to change the way in which it performs its joint
system reviews or DCMA needs to perform the reviews without the
participation of DCAA.
In fact, we mustered the gumption to recommend
to DCAA Director Fitzgerald that he might want to consider exempting
some of DCAA’s audits from the rigors of GAO-interpreted GAGAS, just to
avoid some of these issues. We boldly recommended that Mr. Fitzgerald—
… consider whether all DCAA audits need to be subject to GAGAS.
Reasonable people will disagree with GAO’s stringent definition of
“independence” under GAGAS, but you can avoid the issue altogether if
you make certain audits subject to GAGAS while others are not. There is precedent for this change: the AICPA has Consulting Standards that differ from Auditing Standards.
Since DCAA performs both financial advisory services and audits, it
would seem to make sense to apportion each type of audit into
GAGAS-compliant and non-GAGAS-compliant groupings.
And, by the way, DCMA really wants DCAA to participate in the process
as an advisor; it wants your audits to offer value-added advice and to
support the acquisition process.
Contractors want to hear from auditors as well; they want to know where
they need to improve and what should be done to fix system
deficiencies.
Your auditors can’t do this if GAO will allege they’ve compromised
‘independence’ whenever this happens—so change the rules of the game to
eliminate the issue altogether.
What else did the DOD IG find? The DOD IG found that DCMA compromised its own independence/ objectivity “by appearance” during the performance of the EVMS review. The DOD IG reported—
First,
the Center notified the contractor well in advance which programs and
related cost reports it selected for review. As a result, the
contractor’s compliance efforts could have focused exclusively on those
pre-selected programs. Second, the Center did not vary the selection of
programs reviewed. DCMA reviewed the same five programs during the April
and August 2008 reviews as it did in a prior December 2006 review.
Therefore, the Government has no reasonable assurance of the
contractor’s compliance on other programs.
The
[DCMA] representatives held joint activities with the contractor on two
separate occasions which could have compromised DCMA’s independence and
objectivity. In January 2007, following a December 2006 DCMA compliance
review, DCMA … representatives participated in a week-long contractor
review to identify root causes of various deficiencies and to develop a
corrective action plan. In December 2007, representatives from DCMA …
attended the contractor’s internal audit of its system in preparation
for the April 2008 DCMA compliance review. DCMA officials charged with
determining system compliance should not participate with the contractor
in developing corrective action plans or performing internal audits.
Participation in such activities may compromise the ability of DCMA to
independently determine system compliance and continued acceptability of
the earned value management system.
Moreover,
the DOD IG asserted that “DCMA should not plan or conduct joint
surveillance reviews with the contractor,” even though it admitted that
“DCMA Instruction ‘Earned Value Management System (EVMS) System-Level
Surveillance,’ January 2008, strongly encourages contractor
participation in planning and conducting DCMA surveillance reviews.”
We
have to disagree with the DOD IG on that one. First, DCMA isn’t
subject to rigid interpretations of “independence” under GAGAS and the
IG shouldn’t hold the agency to such an inapplicable standard—especially
when DCMA’s actions clearly fall within its own agency policy
guidance. Second, contractors and DCMA need
to interact together, in order to identify root causes and appropriate
corrective actions. We wish DOD oversight agencies and their industry
partners did more of that sort of behavior! Finally, the IG report
glosses over the fact that the same EVMS criteria that provide the
standards for an adequate EVMS business system, the ones against which
DCMA and DCAA were reviewing, were developed jointly by both industry and DOD officials, back in the days when such an approach was deemed to be in the public interest.
Talk
about an independence problem! Hey DOD IG—how about you deal with the
historical fact that the “adequacy criteria” used for the reviews that
you would like to subject to GAGAS-controlled independence were developed by the same people being reviewed, in concert with the people doing the reviewing.
There’s a reason that’s not considered to be a real issue—and the
reason is that none of the parties ever intended the oversight process
to be subject to GAGAS—nor should it be. Oops. Missed that one, did
you?
We
aren’t the only ones to believe that the DOD IG missed the mark on this
report. The Director of DCMA, Mr. Charlie Williams, Jr., did not
concur with much of the report either. He told the IG—
- DCMA’s
reviews lose their effectiveness and impact if DCMA’s results are
presented to the contractor long after the review has been completed.
- The
DCMA Director agreed that the DCAA findings of noncompliance were
valid. However, DCMA did not consider the issues systemic. Therefore,
the DCMA Contract Management Office worked with DCAA to correct the
issues as part of the contractor’s Corrective Action Plan.
- DCMA
has established and maintains a cooperative Government-contractor
relationship that does not compromise the independence of their
decisions. A Review for Cause is a focused review of specific elements
of the contractor’s earned value management system in order to confirm
the acceptability of the system. Advance notification to the contractor
is necessary to ensure the contractor can support a review during the
time period, key employees are available for interview, and the data
requested is provided to the review team before the review.
- DCMA
agrees with and is committed to ensuring that the decisions which
result from its surveillance and compliance reviews represent an
independent DCMA decision. DCMA does not agree that the only way to achieve this independence is by isolating the contractor from the review process.
Well,
that was a breath of fresh, reasonable, air. There yet may be hope for
the DOD oversight process. Oh, you want to see the DOD IG report for
yourself? Find it here.