The ability to evaluate bidders’ proposed prices in order to determine the lowest probable price—the low bidder—is an art and a skill, and it requires the ability to engage in critical thinking.
Price analysis is always required. And when award will be made on a “best value” basis using FAR Part 15 procedures, the ability to perform cost/price analysis is critical. Part 15 states—
An agency can obtain best value in negotiated acquisitions by using any one or a combination of source selection approaches. In different types of acquisitions, the relative importance of cost or price may vary. For example, in acquisitions where the requirement is clearly definable and the risk of unsuccessful contract performance is minimal, cost or price may play a dominant role in source selection. The less definitive the requirement, the more development work required, or the greater the performance risk, the more technical or past performance considerations may play a dominant role in source selection. … When using a tradeoff process, the following apply:
(1) All evaluation factors and significant subfactors that will affect contract award and their relative importance shall be clearly stated in the solicitation; and
(2) The solicitation shall state whether all evaluation factors other than cost or price, when combined, are significantly more important than, approximately equal to, or significantly less important than cost or price.
This process permits tradeoffs among cost or price and non-cost factors and allows the Government to accept other than the lowest priced proposal.
The above points were emphasized in a recent GAO bid protest decision, as noted by Bob Antonio at WIFCON.
Although we’ve written before about concerns with DCMA’s skills in this area, the bid protest involved the General Services Administration (GSA) and not DCMA. But the lessons to be learned do not depend on the agency involved; they are universal. Let’s dig in, shall we?
GSA issued a small business set-aside RFP seeking up to 20 contractors to perform “repair and alteration” construction services. ID/IQ contracts were to be awarded. Under each ID/IQ contract, multiple task orders would be awarded, each valued at between $150,000 and $500,000. The ID/IQ contracts were to have ceilings of $50 million. In other words, the stakes were quite high for a small business; a winning award might well transform the small business into a successful large government contractor.
According to the GAO decision, the RFP provided that award would be made on a best-value basis, considering the following evaluation factors: prior experience on similar projects, past performance, evidence of local office, socio-economic status, and total evaluated price or cost. The RFP provided that the first two non-price factors were equal in weight and were more important than the remaining two non-price factors, which were equal to each other in weight. The RFP stated that all non-price factors, when combined, were significantly more important than cost or price; however, where the technical merit of competing proposals became more equal, price/cost would increase in importance in the award decision. Offerors were to submit separate technical and price proposals.
So far, so good. That’s all fairly standard stuff. If you have trouble following it, then you probably haven’t dealt with many government RFPs.
The kicker came when the RFP discussed the content of the bidders’ “price proposals.” According to GAO, “the solicitation required offerors to submit G&A rates, and stated that these rates alone would be used to evaluate price. Specifically, the solicitation stated that the offerors’ G&A rates would be evaluated using cost analysis to determine reasonableness, based upon verification of the offerors’ cost submissions for their G&A rates and confirming that the submissions are in accordance with the contract cost principles and procedures described in FAR part 31.” Obviously G&A rates are not costs nor do they correspond to an offeror’s proposed price. Moreover, a simple comparison of G&A rates is utterly meaningless.
The reason you can’t simply compare G&A rates is that G&A rates are calculated in different ways, using different allocation bases. CAS 410 permits three different G&A bases: Total Cost Input (TCI), Value-Added Base (VAB), and Single Element Base (SEB). Even if the G&A expense pool is exactly the same, you get an entirely different G&A rate with each allocation base. When we teach Cost/Price Analysis, calculating the G&A rate under each compliant G&A allocation base is one of the exercises—and it’s almost always an eye-opener for those analysts who think a higher G&A rate is “worse” than a lower one.
And in this case, the situation is even more dynamic because all bidders were small businesses and small businesses are exempt from CAS. Thus, while CAS would permit a choice of any of three compliant allocation bases, in this case the bidders were unconstrained and could literally choose any G&A allocation base that had a logical relationship to the expense pool.
All the above didn’t stop the GSA cost/price analysts from using bidders’ G&A rates as a (really poor) surrogate for total cost or total price being offered. Moreover, just to make matters worse, the RFP instructed offerors to submit “certified financial statements or DCAA report substantiating the offeror’s G&A rate.” The problem there is that you really can’t calculate a FAR Part 31 compliant G&A rate from financial statements—certified or otherwise—because the GAAP definition of G&A doesn’t necessarily match the FAR definition of G&A. For example, for financial statement purposes it’s called “S, G&A” (meaning selling, general & administrative) but it’s just G&A for government accounting purposes. Selling can be part of the G&A expense pool, or not. But that wasn’t the end of the problems GSA was creating for itself and its bidders.
A competitive range was established based on perceived technical risk, and then the G&A rates came into play. Per GAO—
The agency further determined that two of the T1 [Tier 1] technically-ranked proposals with G&A rates of 12% and 20.10%, and three of the T2 technically-ranked proposals with G&A rates of 11%, 11.40%, and 19.58%, did not have ‘the most favorable G&A rates when compared to the others.’ As a result, these proposals, including the protester’s proposal--which received a T2 technical ranking and proposed a G&A rate of [deleted]--were not further considered for award. With respect to the 12 remaining proposals, the agency contacted eight of the offerors and requested that they verify or confirm their G&A rates because of concerns about the rates. … The offerors generally responded by confirming their G&A rates. In these cases, the agency did not request any further substantiation, nor did the agency conduct any analysis of the ‘verified’ rates. The agency, nonetheless, accepted the rates provided in the responses as the offeror’s ‘verified’ rate.
(Internal citations and footnotes omitted.)
Needless to say, disappointed bidders had problems with the foregoing approach to cost/price analysis and the source selection decision. West Coast General Corporation was one such disappointed bidder, and decided to protest the award decision. “Specifically, the protester argues that the agency accepted G&A rates from some offerors that were unsubstantiated, and were not explained by the submitted financial data, as expressly required by the RFP.” Unsurprisingly, GAO sustained the protest.
Among the reasons cited by the GAO for sustaining the bid protest—
… the agency’s evaluation of price proposals was inconsistent with the RFP requirement that offerors’ proposed G&A rates be verified and substantiated using certified financial statements or DCAA reports, as described above. The protester asserts that instead, the agency accepted from eight of the 12 awardees ‘post-bid commitments’ of G&A rates that were either unsubstantiated or unexplained by financial data (and in some instances, directly contradicted by the financial data).
A fundamental lesson to be learned here is that past financial statements—certified or not—or DCAA audit reports confirming the accuracy of past G&A rates is really no basis for evaluating offerors. The G&A rates that were relevant to the source evaluation were future G&A rates to be incurred during task order performance. Past G&A rates were not good indicators of future G&A rates, especially given the fact that these were going to be relatively large task orders awarded to small businesses. One or more task order awards in a single year could result in a significant decrease to the historical G&A rate. The GSA evaluation scheme missed this point entirely.
This points to a larger problem: the notion that DCAA (or any government analyst) can “audit” a contractor’s estimate of future costs to be incurred. That’s not correct. A DCAA admission that such “forward priced proposal” reviews are not audits, and not really subject to the full GAGAS treatment, would go a long way to alleviating the audit agency’s well-known problems regarding quality, timeliness, and usefulness.
As colleague Darrell Oyer recently wrote in his newsletter, “One cannot audit something that has not yet happened. … Audit standards are not applicable to proposal audits. However, the audit attitude is evident by a statement of a DCAA auditor in a deposition: ‘Estimating does not require judgment—you just look at the books and records to see what it cost previously.”
So what value does DCAA provide to those charged with cost/price analysis and in making an informed source selection decision that will survive a bid protest? In theory, DCAA is capable of auditing historical cost data and the reasonableness of contractors’ projections that use that historical cost data as a starting point. DCAA can also audit vendor and subcontractor quotes and can verify the bidder used those quotes properly. DCAA’s audits of proposals don’t ever save the taxpayers any money, but they can certainly help the contracting officer better negotiate the price, which of course does save taxpayers money.
We recently noted a DoD Inspector General report that essentially found that DCMA Cost Monitors were less effective at evaluating contractor proposals than DCAA had been. We didn’t buy the DoD OIG conclusions. In retrospect, we could have and should have made our point better than we did in that article.
Our viewpoint is that nobody should expect DCMA Cost Monitors to perform audits of contractor proposals. Cost Monitors don’t actually, you know, perform audits. They are not trained to do so. (Well, except for all those ex-DCAA auditors who are now DCMA Cost Monitors.) Cost Monitors are supposed to comply with DCMA Instruction 120 (“Pricing and Negotiation”) and not with GAGAS. So it should absolutely not be surprising that Cost Monitors don’t perform audits as well as DCAA does.
On the other hand, it should also be clear that DCAA doesn’t perform audits well—especially when asked to perform “audits” of contractor cost proposals related to future work to be performed. Those cost proposals are essentially estimates. Estimates are not really subject to audit, for the reasons Darrell Oyer stated (which we quoted above). That being said, to the extent those estimates are based on cost information that can be audited, a DCAA auditor should be able to provide valuable assistance to those personnel performing cost realism analyses and who are negotiating contract prices.
Which leaves us right back where we started.
The ability to evaluate bidders’ proposed prices in order to determine the lowest probable price—the low bidder—is an art and a skill, and it requires the ability to engage in critical thinking.
It is not the exclusive province of DCAA, nor is it the exclusive province of a Cost Monitor or Contracting Officer. It is not a function of agency or position. To do a good evaluation—one that will survive a bid protest—is the province of a very few people who have the right skills, the right training, the right disposition, and the right attitude.
When done well, it saves the taxpayers money and results in a defensible contract award. When done poorly (as was the case with the GSA bid protest we summarized here), it hurts both government and contractor. The government ends up with delayed contract performance and possibly with the less-than-best contractor doing the work. The protestor ends up with wasted B&P dollars and possibly with unallowable legal bills. The lawyers, however, are quite happy.
Our previous article posited the notion that the Defense Contract Audit Agency would be better off if it stuck to performing audits, and quit providing “services that fall outside audit support” to its customers—especially those customers that are non-DoD agencies. We asserted that, if DCAA were to focus its staff of some 5,000 auditors on performing audits for DoD only, then it would be better positioned to reduce its embarrassing backlog of unperformed audits (especially including audits of contractors’ proposals to establish final billing rates) and thus get Congress and other critics (including us!) off its back. We opined that a relentless focus on performing audits—and only audits—would be beneficial to the audit agency, as well as to its customers, including DCMA.
