Decisions made using incomplete data can jeopardize the success of the mission and risk the lives of warfighters. Ensuring that personnel have complete, accurate, real-time data at the point and moment of need is crucial, but disconnected systems and sources limit this capability. The challenge is compounded by the growing requirement to coordinate data across domains to support joint operations—a central objective of the Air Force’s Advanced Battle Management System (ABMS).
The reasons for disparity are well-known: a combination of legacy systems, on-premises apps and multi-cloud environments, coupled with geographically dispersed data sources. Intensifying the issue is the sheer volume of data being generated daily across the DoD from sensors, satellites and connected devices and platforms. The solution is found in accessing, analyzing and applying the knowledge hidden in those mountains of data.
In the first of a three-part series, Elastic spells out the challenges to data access and the opportunities to use real-time information as a mission enabler without needing to move, consolidate or duplicate resources.
Overcoming Hurdles to Data Sharing, Real-Time Access
Substantial progress has been made by each branch of service to enable internal users to make full use of data. Meanwhile, the DoD Data Strategy aims to ensure data can be shared easily, producing a more comprehensive knowledge base for joint operations and supporting the Joint All-Domain Command and Control (JADC2) strategy — empowering everyone from command center to front lines with real-time situational awareness and decision support.
Still, the reality is some legacy systems will remain in service for the long term, as they provide specific capabilities that cannot yet be replicated by modernized systems. While legacy solutions will ultimately be phased out over time, for today, these stovepiped systems create obstacles to extracting, standardizing and consolidating data. The lack of standardization can cause issues at a basic level; if one system uses “F15” while another uses “F-15,” for example, searches and analysis can be delayed or derailed.
The AI and ML Connection: Spotting Trends
There’s another crucial use for shared data, one that is growing in importance: expanding the capabilities of artificial intelligence (AI) and machine learning (ML) processes, which are dependent on both a highly-skilled workforce and the ability to feed data to systems on-demand. ML can identify issues quickly, for example identifying if a vehicle is somewhere it shouldn’t be or going in and out of a specific zone.
Systems can also be trained using AI & ML to pinpoint data anomalies, such as whether the expected amount of data was received or transmitted on a specific day. In these cases, the system can alert humans or take predefined actions to minimize risks.
Making Data Available from End to End
ABMS is designed to capture and coordinate data spanning Air and Space Force’s connected sensors and devices, weapons and platforms — an Internet of Military Things — and reaches back to the services’ cloud infrastructure. ABMS is still evolving, but JADC2 integration is an essential goal, supporting the need to use data as a strategic asset across domains and services.
Connecting resources, data and people requires more than just new technologies. It starts with a data architecture that addresses version control, duplication and incomplete or out of date data. More fundamentally, it begins by determining how to share data.
Many data lake attempts fail because the data owner does not want to share everything, seeing the request as an all-or-nothing situation. While data access is a critical factor in mission success, some data simply should not be shared, and there is a concern that sharing everything may overwhelm the ability to use the data effectively. An effective strategy should allow data owners to identify what should be shared while allowing others to decide what they want to include in searches.
Search is the Key
Data’s usefulness is completely dependent on the user’s ability to query it, analyze it and apply it at the point and time of need, all of which hinges on the speed of search.
An enterprise search solution should enable users to analyze information and answer questions—in seconds—from all sources, either in place or through consolidation. For example, Elastic allows data to be collected and stored at endpoints. Since Elastic’s technology indexes data on ingestion, it is instantly ready to search in real-time. Leaving data in place eliminates version control while reducing the network congestion that would result from moving huge databases to a central location. Instead, the user can send the question to the data, instead of bringing the data to the question.
To address differences between how data is formatted, such as the “F15/F-15” issue mentioned earlier, an open source schema can assure all data is written the same way.
A search approach that can look across systems and locations should also reconcile any differences in user interfaces and data structure. A familiar example of this comes from ride-sharing platforms Uber and Lyft. Both companies use Elastic’s 2D data to sync drivers to riders. They deliver the same query speed despite using different maps and interfaces; each system uses geospatial data to map IP addresses to known locations. These same capabilities can be applied to logistics or troop movements, giving users much greater situational awareness.
Better Search Powers Better Decisions
The Air Force is moving to meet the growing demand for complete, timely data through programs such as VAULT, which allows personnel at all levels to directly access data using open source and commercial, cloud-based tools. The Mission Assurance Capability Kit (MACK) provides a similar capability, specifically focused on protecting aircraft systems from cyberthreats. MACK data can feed into ABMS to provide a crucial part of domain visibility.
For data to function as a strategic asset, it’s essential to ensure everyone is searching the same authoritative data, within and across all services and domains. In doing so, redundancies are avoided, as are issues with conflicting or out-of-date information. Most importantly, real-time search—delivering real-time knowledge—can inform smarter decisions at the moment of need.
For more information please contact airforce@elastic.co or visit elastic.co/federal to learn more.