Data transformation in the US Air Force: highlights from AFORS

What happens in Vegas...

Last month I visited the US Air Force Operations Research Symposium (AFORS) at Nellis Air Force Base in Las Vegas. The conference is an annual get-together for the Air Force's analyst community, with over 200 attendees talking about operations research, machine learning, and data science topics.

The Air Force has been investing heavily in modernizing its tools and training for working with data, and I was there to run a code-along workshop on using DataCamp Workspace.

Five key areas for data transformation

Dr. Angela Giddings, Technical Director of Studies, Analyses, and Assessments, summed up the Air Force's goal around its data transformation program. She stated that the big problem is figuring out how operations research analysts can become a force multiplier for the rest of the Air Force. Traditionally, operations research has been a very academic group within the Air Force, but that model is no longer viable in the current service.

Dr. Giddings listed five key areas for transformation.

• Ensuring projects have an impact on a real problem
• Ensuring results are delivered while they are still relevant
• Using the right data for the right problem
• Generating insights from math
• Making use of visualizations

Interestingly, all these areas are things that mirror challenges faced in organizations around the world, regardless of whether they are in the military or not.

The hiring challenge

A recurring theme from senior officers was that hiring and retaining top data talent is incredibly challenging. Both Mr. Wayne Schatz, Director for Studies, Analyses and Assessments, and Col Ron Annaballi, Air Force Chief Analyst, talked about addressing this hiring challenge by making the squadron a great place to do work, and training and tools are key contributors to this. The Air Force has access to three online education platforms—DataCamp, Coursera, and PluralSight—in addition to in-house training. A modern data stack is being developed through broad access to Python, DataBricks, Tableau, Plotly, Dataiku, and Palantir.

Impressive technical presentations

In the technical presentations, I was impressed by the range of USAF projects. Optimizing pilot training was a recurring theme, and I saw not one but two presentations involving strapping heart rate, eye-tracking, and other sensors to pilots to try and determine which flight maneuvers were the most cognitively challenging or terrifying.

Other presentations I saw ranged from bread-and-butter business problems—MSgt Nathan Shaw talked about using sentiment analysis to analyze survey feedback, to weird problems I never knew existed—Dr. Tyler Danley and Maj Brian Morrison talked about how to schedule training flights to reduce the number of bat strikes, to science fiction—Cdt Victor Rae discussed optimizing the size and position of satellite constellations to get the best coverage for missions.

For me, the best technical talk of the conference was by 2Lt Benjamin McCloskey. Unmanned aerial vehicles use machine learning on their camera data to detect buildings and vehicles in their field of view. According to 2Lt McCloskey, there's a problem that most of the model training data are captured during the day in good weather conditions. That means that the performance on real missions is poorer during night missions or in bad weather. 

His solution was to use a deep learning technique called "generative adversarial networks" to modify existing camera images to make them resemble nighttime or bad weather. Essentially, he created "deepfakes" of satellite data in order to improve the accuracy of the object detection models.

DataCamp training led to new data products

Finally, I saw an excellent presentation from Capt Dominick Speranza on the results of a DataCamp-based training program. The program's goal was to have more widespread skills to create data products from aircraft instrumentation data. The training program consisted of a curated series of DataCamp Python courses, followed by in-house training on aircraft instrument data. The capstone project to complete the training required participants to create a data product. Examples of these data products included an anomaly detection system for determining the data quality from F-22 fighter jet instrumentation and an automated report generator for making pilot debriefing sessions easier.

Wrap-up

The whole conference ran smoothly due to excellent organization by Maj Bryan Fagan. (I should have guessed that operations researchers would be good at operations!) After hearing all these examples of great data projects, I'm excited to see how the US Air Force data transformation program progresses.