Verification of marine oil spills using aerial images based on deep learning methods

The article solves the problem of verifying oil spills on the water surfaces of rivers, seas and oceans using optical aerial photographs, which are obtained from cameras of unmanned aerial vehicles, based on deep learning methods. The specificity of this problem is the presence of areas visually similar to oil spills on water surfaces caused by blooms of specific algae, substances that do not cause environmental damage (for example, palm oil), or glare when shooting (so-called look-alikes). Many studies in this area are based on the analysis of synthetic aperture radars (SAR) images, which do not provide accurate classification and segmentation. Follow-up verification contributes to reducing environmental and property damage, and oil spill size monitoring is used to make further response decisions. A new approach to the verification of optical images as a binary classification problem based on the Siamese network is proposed, when a fragment of the original image is repeatedly compared with representative examples from the class of marine oil slicks. The Siamese network is based on the lightweight VGG16 network. When the threshold value of the output function is exceeded, a decision is made about the presence of an oil spill. To train the networks, we collected and labeled our own dataset from open Internet resources. A significant problem is an imbalance of classes in the dataset, which required the use of augmentation methods based not only on geometric and color manipulations, but also on the application of a Generative Adversarial Network (GAN). Experiments have shown that the classification accuracy of oil spills and look-alikes on the test set reaches values of 0.91 and 0.834, respectively. Further, an additional problem of accurate semantic segmentation of an oil spill is solved using convolutional neural networks (CNN) of the encoder-decoder type. Three deep network architectures U-Net, SegNet, and Poly-YOLOv3 have been explored for segmentation. The Poly-YOLOv3 network demonstrated the best results, reaching an accuracy of 0.97 and an average image processing time of 385 s with the Google Colab web service. A database was also designed to store both original and verified images with problem areas.