Deep-Learning Assisted Digital Twin of Stereo Camera for non-Invasive Underwater Fish Biomass Estimation

Abstract

Digital twins have become increasingly important in aquaculture, a sector traditionally dependent on manual, subjective, invasive, and labor-intensive practices. This trend is driven by the convergence of precision aquaculture and AIoT technologies, enabling a shift from experience-based practices to intelligent, data-driven systems. Existing AIoT architectures are highly specialized, addressing function-specific requirements. This work proposes a digital twin–enabled smart stereo camera system for monitoring fish growth through non-invasive biomass estimation of freely swimming fish. The digital twin framework includes a processing pipeline consisting of RGB-D video acquisition, 6D (RGB-XYZ) representation generation, 3D point cloud Transformer-based segmentation, and fish biomass regression. First, the stereo camera captures RGB-D video in the aquaculture environment, which is automatically transmitted to a cloud system for further processing. The RGB-D frames are then transformed into 6D (RGB-XYZ) representations for subsequent analysis. A 3D point cloud Transformer is then used to detect and segment fish objects from the 6D representations. Finally, the reconstructed 3D fish objects are used for k-nearest neighbors (KNN) regression to estimate fish biomass. The contributions of this work are as follows. First, the digital twin approach enables the transformation of aquaculture toward intelligent farm management. Second, the 3D computer-vision-based fish biomass estimation scheme is a non-invasive model for understanding the status of fish growth without disturbing fish schools. Third, the proposed 3D point cloud Transformer has low computational complexity and can be deployed on edge-computing platforms with limited GPU resources. Finally, the digital twin model synthesizes fish growth data based on the existing fish growth model to improve the estimation accuracy of fish biomass. To the best of our knowledge, this work presents one of the first digital twin–enabled smart camera systems deployed in real aquaculture environments for real-time fish growth monitoring.

Keywords: Smart Camera, Digital Twin, Deep Learning, Artificial Intelligence of Things (AIoT), Fish Biomass Estimation, RGB-D Video.

DOI: 10.54941/ahfe1007272