Competition is cranking up the pressure on everything the Defense Department does. The pace of change is so great and the cost of delay so dear that Vice Chairman of the Joint Chiefs of Staff Gen. John Hyten ranks among his highest priorities the drive “to insert speed into everything we do.”
As much as Hyten worries about Russian missiles or Chinese air, space and cyber capabilities, it’s the slow pace of acquisition that often frustrates him more. “We have adversaries now that are going faster than we are,” Hyten said at a recent Air Force Association address. “And it doesn’t matter how far ahead you are in the race, if somebody’s running faster than you are. they’re eventually going to catch you.”
Speed is an issue in developing and fielding new technology, but also in contracting for proven equipment. That’s why GE Aviation has developed a new approach to pricing designed to cut months of paperwork and manual effort from the process.
GE calls it Brilliant Pricing both because the system uses Artificial Neural Networks (ANNs) and because it can slash the time needed to produce pricing estimates by 85 percent.
When the government enters into a sole-source contract, suppliers must document how they arrived at a price, an excruciatingly time-consuming exercise that produces reams of paperwork and can take months to complete. If the Air Force or a foreign partner wants to buy more F110 jet engines, for instance, GE must document the cost of every single component—down to the last screw.
“In conventional pricing, that’s hundreds of people inputting data manually,” explains Karin Burns of GE Aviation. Documentation is required for each step in the supply chain, with each part’s supplier certifying its estimate of what they’ll charge GE dependent on each of their own suppliers’ certified estimates in turn.
“That’s a long and tedious process,” Burns says, noting that it can take “north of six months” to complete.
Next, Air Force acquisition specialists must review and analyze all those documents before they can approve a transaction.
With Brilliant Pricing, however, GE applies Artificial Neural Networks to historical data to forecast the costs of parts-based prices the company or its suppliers paid for that part in the past.
Thanks to new acquisition authorities approved by Congress, defense officials allowed GE to run a foreign military sales pricing effort for F110 engines to Bahrain, along side a similar conventional price estimation for Qatar. The conventional estimating technique for Qatar took six months to complete. The Bahrain bid was completed in just 58 days, saving about 1,000 hours of effort.
“And that was the first time we did it,” adds Burns. “We believe we can now produce a proposal within 15 days.”
Even the form of the Brilliant Pricing bid proposal is different. “Instead of receiving boxes of printed PDFs,” Burns says, “they’re receiving a real user-friendly interface with all the data available to them with a single click.”
This Brilliant Pricing interface, lets defense officials review the entire bill of material for the engine, and to click through into each part to access the historical data on which the estimate is based, “all displayed in easy to grasp charts and graphs right at their fingertips,” says Burns. “It’s very transparent, very auditable.”
GE based its process on an open-source ANN software model called Long Short-Term Memory, or LSTM, and trained it with the help of officials from the Defense Contract Audit Agency and the Defense Contract Management Agency. In biweekly working sessions, company data scientists and DOD auditors worked through how to account for the influence of all manner of factors to accurately project future costs.
Not only was Brilliant Pricing faster and more easily auditable, Burns says, it was also “just as accurate as traditional manual processes.”
GE used historical pricing sets to make Brilliant Pricing forecasts for past bids and compared the original audited estimates to what Brilliant Pricing projected using only the data and assumptions it would have had at the time. Consistently, its projections were more accurate than past methods.
Over four pricing exercises prepared using 2016-18 data, the new tool provided forecast estimates that were within 0.3 percent of the actual historical prices, says Burns. Generally, traditional pricing estimates turn out to be within 3 percent of the actual price. So Brilliant Pricing estimates were literally an order of magnitude more accurate.
Now GE is considering sharing Brilliant Pricing with other vendors. That’s really where the biggest value will come for the DOD—if it’s not just one contractor using this method, but multiple contractors.
For example, GE has briefed contracting teams at Boeing about the tool.
“We want the government to think of GE as a partner that helps to solve problems,” Burns says. “Brilliant Pricing can speed up acquisition. And the truth is, when we’re successful with initiatives like Brilliant Pricing, it’s a rising tide that’s going to lift all boats. But at the end of the day, what we want, what’s important, is that the Air Force and all the services win. And this will help them win.”
The next step is to move Brilliant Pricing from the exception to the rule. Defense agencies have the authority to accept this kind of pricing, but they aren’t required to do so, at least not yet.
GE is working with DOD officials to develop rules of the road for a new kind of certification where DOD audits the accuracy of the estimating technique and the integrity of the data rather than the individual pricing estimates.
That’s more efficient, Burns explains, and ultimately what GE’s defense customers need.
“We don’t want Brilliant Pricing to be used as an exception,” Burns says. “We want it to be the standard way of doing business.”