Navigating the Seas of Automation: Human-Informed Synthetic Data Augmentation for Enhanced Maritime Object Detection

Abstract: Digitalization and increased autonomy in transportation have the potential to create sustainable, safer, and more efficient service chains, contributing to a better quality of life and global prosperity. Key technologies, including AI, sensor fusion, and deep learning, are already available for autonomous vessels. However, the challenge lies in effectively integrating these technologies, particularly in the complex and dynamic maritime environment.The demand for autonomous maritime systems has driven the integration of machine learning to enhance intelligence, particularly in object detection with computer vision. This task faces complexities due to factors such as lighting, weather conditions, and waves. However, ensuring the accuracy and trustworthiness of machine learning algorithms poses a significant challenge, primarily related to acquiring a well-prepared dataset. Creating a detailed dataset covering diverse scenarios proves difficult, time-consuming, and costly across various research areas. Data scarcity in maritime settings hampers progress, given the intricate and expensive nature of data collection and labeling. Additionally, the relatively new concept of autonomy in this domain limits the availability of relevant datasets, compounded by challenges posed by diverse weather conditions during data collection.In 2022, our aim was to build a comprehensive image dataset in Finland's maritime domain, consisting of 120,216 RGB annotated images. Evaluation by a maritime expert revealed a lack of diversity in weather conditions within our dataset, prompting the need to incorporate human opinions.To overcome data scarcity, especially in varying weather conditions, we propose a novel approach for maritime object detection. Our method employs human-informed synthetic data augmentation using Generative Adversarial Networks (GANs), implemented through 4Sessions-Net (4S-Net). This innovative strategy positively impacts labeled data and addresses challenges related to dataset imbalance and insufficiency.Synthetic data generation using GAN networks, such as 4S-Net, is a cutting-edge solution to overcome these limitations. This paper introduces 4S-Net, which augments labeled data, positively impacting results. However, the synthetic data's complexity may not match real-world scenarios, necessitating model evaluation with real data.The dataset, collected in the complex Finnish archipelago, was accurately labeled and extended with synthetic data representing different weather conditions. Comparative analysis involving three CNNs on the original and new datasets, including GAN-generated data, reveals superior accuracy in models trained on the new dataset.In summary, while digitalization and autonomy offer promise, data scarcity and environmental challenges in maritime settings hinder progress, requiring a high level of understanding and contribution from domain experts. Synthetic data generation through GAN networks based on expert opinion, as demonstrated with 4S-Net, is a key solution resulting in improved model accuracy. This approach not only addresses the limitations of real-world data collection but also contributes to advancing the application of machine learning in maritime autonomy. The results demonstrate significant improvements in accuracy and reliability while simultaneously reducing the cost and time of data collection through the incorporation of expert opinions in dataset creation.

Keywords: Autonomous vessels, human factor, synthetic data, GAN network, maritime environment

DOI: 10.54941/ahfe1005004

Cite this paper:

​