Readers know that we’ve taken the US Air Force to task—more than once—for botched source evaluation and award decisions. The debacle of the KC-X aerial tanker competition was not the only screw-up. More recently, legal maneuvering with regard to the Light Air Support (LAS) competition, evaluation, and award to the team of Sierra Nevada Corporation (SNC) and Embraer has generated several articles on this site—the most recent here.

The LAS story involves a USAF award decision that was “inadequately documented,” a refusal to provide the losing bidder with a debrief (because of an untimely request), a denied protest at GAO, litigation at the Court of Federal Claims, a “Commander Directed Investigation” into the source selection decision, an early termination of the inadequately documented contract award, and a decision to recompete the matter. As we reported, the USAF decided not to conduct a fly-off in the recompete, a decision that Under Secretary for Defense (AT&L) Kendall termed “acquisition malpractice” when it was made with respect to the F-35 JSF program.

It is that decision not to perform flight tests that sparked SNC to file an action of its own at the Court of Federal Claims, seeking to have its terminated contract reinstated. The various stories on SNC’s action all of appear to be strangely similar in wording, almost as if they were all recapping the same SNC press release. Here’s a quote from the story linked-to above—

According to SNC the cancellation of the contract was an extreme response to what appears to be paperwork errors on the part of the USAF. Moreover, the revised Request for Proposal (RFP) issued by the USAF is tilted in favor of the competition. … SNC’s filing also raises specific concerns with the source selection process and revisions to Amendment 8 of the RFP. The new source selection process eliminates any flight demonstration/evaluation and moves the completion of First Article Test (FAT) of production aircraft out until delivery in July 2014.  … The original source selection process included flight demonstrations of training and combat mission profiles and austere field operations. The competition sought non-developmental aircraft, which by definition should be available for evaluation.

Using language that was quite reminiscent of Hawker Beechcraft’s complaints regarding the original source selection decision, SNC Vice President Taco Gilbert was quoted as saying—

‘Despite repeated written and verbal attempts, we have not received adequate explanation – much less justification – for the termination of our contract, the reopening of the LAS competition or the readmission to the LAS competition of our competitor whose submission was previously found to be technically deficient and carry unacceptable mission capability risk.’

Gilbert added—

‘What we seek is a fair and open competition – one where there is a level playing field, one that provides transparency into the decision making process, and one that selects the best value as required by the Request for Proposal. Unfortunately, based on the information we have, we are concerned that this competition will not conform to these goals.’

But while the LAS debacle continues (depriving the warfighters in Southwest Asia with needed ISR capability), the USAF seems to have figured out how to evaluate complex proposals and make protest-proof source selection decisions in other areas. We are referring to the Dismount Detection Radar (DDR) contract award to Raytheon, a decision that was protested by the losing bidder (Northrop Grumman). Northrop’s protest was denied, and the lengthy GAO decision gives us enough details so that we think the USAF evaluators may have (re)learned how to do their job.

According to the GAO decision—

The DDR system is intended to provide a ground moving target indicator capability to detect and track vehicles and dismounts. The system, operating as a pod on the MQ-9 Block 5 Reaper, will allow combatant commanders and their forces to identify and eliminate threats before adversaries engage in harmful activities against the United States and Coalition Forces. … According to the agency, the DDR system is an urgent and compelling need of the warfighter.

The Air Force sought bids from only two companies, Raytheon and Northrop. The decision to limit competition was properly justified, according to the GAO. RFP evaluation factors were clearly articulated, as follows—

Award was to be made to the offeror whose proposal represented the best value to the government based on an integrated assessment of three evaluation factors: (1) schedule; (2) technical capability (comprised of two subfactors, technical performance, and engineering and management integrated processes--technical performance was considered more important); and (3) cost/price. … The schedule factor was more important than the technical capability factor; when combined, these factors were approximately equal to the cost/price factor. Each offeror’s technical solution would be assessed both technical and risk ratings for the schedule factor and for each technical capability subfactor. … For the technical ratings, the Air Force was to evaluate the quality of the technical solutions as outstanding, good, acceptable, and so on. In assessing the risk associated with each approach, which was to be evaluated as low, moderate, or high, the Air Force was to consider such things as the potential for disruption of schedule and the need for increased government oversight.

The bidders submitted proposals and made an oral presentation. The Source Selection Evaluation Team (SSET) made an initial evaluation and briefed the Source Selection Authority (SSA). Written and oral discussions were held with both bidders, and both bidders submitted final proposal revisions in response to those discussions. The final proposal revisions were evaluated by the Source Selection Evaluation Board (SSEB), briefed the Source Selection Advisory Council (SSAC) on its findings, and prepared a Proposal Analysis Report (PAR). The SSAC prepared a Comparative Analysis Report (CAR) and briefed the SSA on its award recommendation. The GAO decision stated—

In making his source selection decision, the SSA conducted an integrated assessment and found that Raytheon’s proposal presented a lower evaluated cost, less risk in the schedule factor, and a higher performing and more capable DDR system in the technical performance factor. He considered Raytheon’s proposal to be the best value for the government.

The GAO decision also reported that—

… the SSA first found that Raytheon had the stronger proposal under the schedule factor based on differences in the maturity of the offerors’ antenna array design/built/test efforts. … Second, under the technical performance subfactor, the SSA explained that Raytheon’s one weakness was based on its proposal of [DELETED] flight tests, an insufficient number. The SSA concurred with the technical team’s conclusion that four additional flight tests would be required; given Raytheon’s approach, [DELETED] flight tests would be adequate. Raytheon’s GEMPC [Government Estimate of Most Probable Cost] was adjusted upward to account for the additional flight tests. Overall, the SSA concluded that Raytheon’s proposal, and its combined performance in the areas of radar performance, non-developmental items reuse, scalability and upgradeability, and information assurance, offered significantly more benefit to the government than did Northrop Grumman’s proposal…. Although Northrop Grumman distinguished itself in certain areas, it was not enough to overcome Raytheon’s overall superior DDR system performance and capabilities. … The SSA also noted that Raytheon’s proposal had the lowest evaluated cost.

We won’t go into all the issues raised by Northrop. We’ll simply summarize the issues as the GAO did. To wit—

Northrop Grumman primarily challenges numerous aspects of the Air Force’s evaluation of the proposals and alleges that they were disparately evaluated.

The evaluation of technical proposals is a matter within the agency’s discretion, since the agency is responsible for defining its needs and for identifying the best methods of accommodating those needs. … Our Office will not reevaluate technical proposals; rather, we will review a challenge to an agency’s evaluation only to determine whether it was reasonable and consistent with the terms of the solicitation and applicable statutes and regulations. … A protester's mere disagreement with the agency's judgment regarding the relative merits of competing proposals does not establish that the evaluation was unreasonable. … Our decision does not specifically address all of Northrop Grumman’s arguments, but we have fully considered each of them and conclude that they do not provide a basis to sustain the protests.

Readers unfamiliar with the intricacies of government source selection and evaluation requirements may be put-off by the number of acronyms and the obviously bureaucratic processes involved. (We could have described even more bureaucratic processes and used even more acronyms, had we wished. They are certainly present in the GAO protest decision.) But readers need to understand that such seemingly bureaucratic processes are designed to (1) provide documentation, (2) support transparency, (3) facilitate oversight (including judicial review, if necessary), and (4) provide assurance that taxpayer funds are being wisely spent.

Contrast the robust documentation trail in the DDR source evaluation and selection decision with the “inadequate” documentation trail in the LAS competition. In the former case, a protest was denied and the winner could get back to performing the contract. In the latter case, a contract was terminated, an investigation was launched, and the parties are flailing about in court. In the former case, the warfighters will get their technical support on time; in the latter case, they will not.

Which do you think is the better outcome?