IMAGES-AI: Generative AI to improve products for the US Navy warfighter
Abstract
Project IMAGES-AI uses generative AI (GenAI) to improve products for the US Navy warfighter. Recent advances in GenAI, and the ability to adapt it to a specific domain, have made it a useful tool for enhancing the interfaces that display information used by the warfighter. The overarching goal is to improve situational awareness and strengthen the decision-making process, primarily through visual GenAI image generation. GenAI image generation has become a fast-growing capability and has overcome many of the early errors and mistakes seen in the technology's early days. Much of this has been made possible with improvements to Large Language Models (LLMs). LLMs can be very large and cover a wide range of data to support correct image generation. However, for very specific domains, much of the data required, such as for the US Navy warfighter and related topics, can be very limited, which can impact the results of GenAI-generated imagery for this specific topic. Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that is used to customize pre-trained models (LLMs). It does this by training a small set of extra weights instead of altering the massive LLM model. The concept is to freeze the base weights of the original pre-trained model to preserve its coverage and capability. A domain-specific lightweight companion matrix or adapter layers can be added to inject domain-specific data into specific parts of the model with attention mechanisms. No retraining of the LLM is required; instead, a smaller add-on portion is used, where gradient updates modify only the much smaller set of adapter parameters. This produces a LoRA file, essentially a plug-in that adjusts the final output to better match the desired characteristics. In this case, applicable to US Navy applications. This technique is cost- and resource-efficient, small in file size compared to LLM data, and easy to swap out for different adapters to align with the desired output. IMAGES-AI has leveraged this technique to generate imagery that better illustrates Navy missions and courses of action. IMAGES-AI has been applied to applications to test and verify whether improvements are seen in warfighters through improved situational awareness and rapid decision-making. Domain and environmental information become embedded in the imagery. This has been examined through integration into applications and used in storyboarding workflows and scenario development. Initial prototype results indicate a valid return on investment.
Keywords: Generative AI, LoRAs, LLMs, Course of Action, Mission Planning, Image Generation, Storyboarding, ComfyUI, Shot List, Workflow
DOI: 10.54941/ahfe1008407
Cite this paper
More from this volume
- AI-Enhanced Live-Streamed English Language Learning: Designing and Evaluating an Intelligent Platform for Chinese Learners
- "They Will Move": Confirmation Bias, Expectation Frames, and Premature Monitoring Drop Off in COLREG Encounters
- Automation transparency communication does not improve supervisory control performance, but shapes operator trust
- The influence of neuroticism on startle and surprise response: a flight simulator study
- EEG-Based Classification of Reward, Penalty, and Movement Events Across Four Video Games of Varying Complexity
- The Effects of Task Similarity and Time Pressure on Task Performance Following Interruptions in Complex Human-Computer Interaction Systems
- Trusting the Chain, Not the Black Box: A Provenance-Based Architecture for Human-Machine Teaming in Military Autonomy
- Design of a Simplified Humanoid Robot Face: Cultural Differences and Emotional Synchronization
- From Visual Observation to Auditory Interaction: A Human-Centered Tangible Music System Based on Physical Objects
- The Impact of Scarcity-Based Dark Patterns on User Performance, Perceived Darkness, and Workload in Subscription Services
- A Behavioral Observation Tool for Diagnosing Human–AI Interaction Profiles in Engineering Design Education
- The Mediated Moon: Semantic Mediation Between Gesture and Generative Image


AHFE Open Access