ABNORMAL PIG BEHAVIOR RECOGNITION BASED ON VIDEO-AUDIO DATA FUSION USING CONVOLUTIONAL NEURAL NETWORKS

Abstract

The pig breeding process in Vietnam has been relatively well-developed compared to the rest of the world. However, not all pig farms in Vietnam also have advanced facilities and techniques to monitor and prevent abnormal behavior on pigs. Data is collected from several farms both at home and abroad, through preprocessing and extracting important behavioral characteristics in terms of video and audio. Then, the 3D-CNN model, combined with the YOLOv11n model, was used to predict behavior. The HMM model was then utilized to analyze and recognize abnormal sounds in pigs. These two models were then merged to form a single model, enabling it to analyze both video and audio input data in parallel. The accuracy of abnormal video behavior recognition is 91.7%, and the accuracy of abnormal audio recognition is 98.2%. These results show the great potential of applying artificial intelligence to the pig breeding process. Keywords: 3D-CNN, HMM, Data Fusion, Multi-modal, Machine Learning.