Tanker Fiasco: Five Steps to Fix the Problem

Issue Brief

The Government Accountability Office’s stinging rebuke of the Air Force tanker competition last week was so sweeping that some observers say a new award may not be made for years. That would be very unfortunate, because the 500 Eisenhower-era tankers in the force comprise the oldest large fleet of jets in the world. There’s no way of knowing how long they will last, and the joint force depends on having aerial refueling capabilities around the globe on a daily (indeed hourly) basis. So instead of engaging in an extended post-mortem, policymakers need to fix the problem fast. Looking at the five evaluation factors used in the competition, it is clear what changes need to be made.

Mission capability was the factor weighed most heavily in comparing the two proposals, and it should continue to occupy center stage. But GAO says evaluators did not apply capability measures as stated in the original request for proposals, and thus misled offerors as to what features would elicit the best scores. This is a simple problem to fix — not by developing new capability metrics, but by finding competent evaluators to apply the existing ones. If re-competition is used as a pretext for drafting revised metrics, it will introduce new uncertainty into a process that must be brought to closure.

Proposal risk was the second most highly ranked metric in the competition. The Air Force found the two proposals had equal risk, but only after forcing Boeing to raise the cost of developing its tanker. GAO says that insisting on such an increase was unwarranted because there was no evidence Boeing’s cost estimates were unrealistic. That finding seems to suggest that the Air Force assesses lower risk when companies spend more money on a project, even if there is no demonstrable need to spend the additional funds. The service will need to recalibrate how it measures risk.

Past performance was the third evaluation factor used, and it has proven to be a useful indicator of contractor competence in other competitions. Boeing was judged inferior to the Northrop Grumman team even though it built all the tankers in the current fleet, based on a comparison of more recent programs with characteristics similar to the envisioned future tanker. GAO did not question the application of this factor in its press release, implying that only minor adjustments may be needed. The offerors can propose that different past programs be used to rate their performance in the re-competition.

Cost/price was the fourth evaluation factor employed, although the Air Force always insisted it was less important than the previous three. The Air Force conceded errors in calculating the ownership costs of the competing planes prior to the GAO report, and the report found additional basis for doubt in the service’s estimate of what development and construction expenditures would be required by the two proposals. These problems can be corrected through more rigorous application of cost methodology.

Integrated Fleet Air Refueling Assessment was a complex computer simulation of operational performance used as the final evaluation factor, and it appears to have been the most flawed. Like other analytic tools employed by the military to model operational outcomes, it is only as good as the assumptions made at the outset, and in the case of the tanker competition those assumptions were not realistic. GAO did not attempt to assess the intricacies of how the model was applied, but if the Air Force wants to avoid another protest in the future, it needs to make the model conform more closely to real-world conditions, such as the actual characteristics and likely availability of wartime bases.