[en] Monitoring multiple behavioral patterns and grazing trajectories of individual sheep can provide valuable information for various aspects of livestock production. The emergence of low-cost miniature sensors, coupled with the continuous advancement of deep learning technologies, has ushered in a new generation of intelligent solutions for precision livestock farming. This study aims to explore the deployment methods of motion sensors, selection of data collection frequencies, and choice of deep learning algorithms to provide accurate classification of multiple behaviors in grazing sheep. Based on this, in conjunction with the acquired location information, the goal is to comprehend the spatiotemporal distribution of grazing sheep behaviors. Devices capable of collecting Inertial Measurement Unit (IMU) and location data were attached to the jaw, neck, and hind leg of sheep. Four datasets were created using IMU data with a frequency of 20 Hz and a 5 s time window from different positions (neck, neck & leg, jaw, jaw & leg). Two deep learning models, Convolutional Neural Network (CNN) - Long Short Term Memory (LSTM) and Temporal Convolutional Network (TCN)-Transformer, were employed to classify six grazing sheep behaviors: walking, standing, grazing, lying, standing-ruminating, and lying-ruminating. The results indicate that by fusing data from the neck- and leg-mounted devices and utilizing the CNN-LSTM model, the accuracy reached the highest at 99.3 %. Furthermore, a comparison was made regarding the behavior classification accuracy of this fused data at different IMU data frequencies (20, 10, 5, and 1 Hz). It was found that even when the data frequency was reduced to 1 Hz, the classification accuracy for the six sheep behaviors still exceeded 96 %. Additionally, the trained model with the highest accuracy was applied to monitoring grazing sheep behavior under two different management procedures. Further analysis of the time budgets of sheep behavior revealed differences in the durations of behaviors under different management procedures. For extensively grazing sheep, location information was also monitored and combined with behavior classification results to generate the spatiotemporal distribution of sheep behavior. The technologies presented are important for gaining further insights into the health status of grazing livestock and grassland, thereby enhancing grazing management practices by informing grazing choices and optimizing grassland utilization.
Jin, Zhongming ; Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing, China
Shu, Hang ; Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing, China ; AgroBioChem/TERRA, Precision Livestock and Nutrition Unit, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
Hu, Tianci; College of Computer and Information Engineering, Xinjiang Agricultural University, Urumqi, China
Jiang, Chengxiang; College of Computer and Information Engineering, Xinjiang Agricultural University, Urumqi, China
Yan, Ruirui; Institute of Agricultural Resources and Regional Planning, Chinese Academy of Agricultural Sciences, Beijing, China
Qi, Jingwei; College of Animal Science, Inner Mongolia Agricultural University, Hohhot, China
Wang, Wensheng; Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing, China ; Big Data Development Center, Ministry of Agriculture and Rural Affairs of the People's Republic of China, China
Guo, Leifeng; Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing, China ; Xinjiang Wool Engineering Technology Research Center, Institute of Animal Husbandry Quality Standards, Xinjiang Academy of Animal Science, Urumqi, China
Language :
English
Title :
Behavior classification and spatiotemporal analysis of grazing sheep using deep learning
Ministry of Science and Technology of the People's Republic of China
Funding text :
This research was supported by the National Key Research and Development Program of China (2021YFD1300500); the Key Research and Development Program of Xinjiang Uygur Autonomous Region (2023B02013); and the Science and technology innovation project of Chinese Academy of Agricultural Sciences (CAAS-ASTIP-2023-AII).
Akasbi, Z., Oldeland, J., Dengler, J., Finckh, M., Analysis of GPS trajectories to assess goat grazing pattern and intensity in southern Morocco. Rangel. J. 34 (2012), 415–427, 10.1071/RJ12036.
Alvarenga, F.A.P., Borges, I., Palkovič, L., Rodina, J., Oddy, V.H., Dobos, R.C., Using a three-axis accelerometer to identify and classify sheep behaviour at pasture. Appl. Anim. Behav. Sci. 181 (2016), 91–99, 10.1016/j.applanim.2016.05.026.
Alvarenga, F.A.P., Borges, I., Oddy, V.H., Dobos, R.C., Discrimination of biting and chewing behaviour in sheep using a tri-axial accelerometer. Comput. Electron. Agric., 168, 2020, 105051, 10.1016/j.compag.2019.105051.
Anderson, D.M., Winters, C., Estell, R.E., Fredrickson, E.L., Doniec, M., Detweiler, C., Rus, D., James, D., Nolen, B., Characterising the spatial and temporal activities of free-ranging cows from GPS data. Rangel. J. 34:2 (2012), 149–161, 10.1071/RJ11062.
Bai, S., Kolter, J.Z., Koltun, V., An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. ArXiv., 10.48550/arXiv, 2018, 1803.01271.
Barwick, J., Lamb, D.W., Dobos, R., Welch, M., Trotter, M., Categorising sheep activity using a tri-axial accelerometer. Comput. Electron. Agric. 145 (2018), 289–297, 10.1016/j.compag.2018.01.007.
Barwick, J., Lamb, D.W., Dobos, R., Welch, M., Schneider, D., Trotter, M., Identifying sheep activity from tri-axial acceleration signals using a moving window classification model. Remote Sens., 12(4), 2020, 646, 10.3390/rs12040646.
Brennan, J., Johnson, P., Olson, K., Classifying season long livestock grazing behavior with the use of a low-cost GPS and accelerometer. Comput. Electron. Agric., 181, 2021, 105957, 10.1016/j.compag.2020.105957.
Casal, R., Di Persia, L.E., Schlotthauer, G., Temporal convolutional networks and transformers for classifying the sleep stage in awake or asleep using pulse oximetry signals. J. Comput. Sci., 59, 2022, 101544, 10.1016/j.jocs.2021.101544.
Cheng, M., Yuan, H., Wang, Q., Cai, Z., Liu, Y., Zhang, Y., Application of deep learning in sheep behaviors recognition and influence analysis of training data characteristics on the recognition effect. Comput. Electron. Agric., 198, 2022, 107010, 10.1016/j.compag.2022.107010.
Decandia, M., Giovanetti, V., Molle, G., Acciaro, M., Mameli, M., Cabiddu, A., Cossu, R., Serra, M.G., Manca, C., Rassu, S.P.G., Dimauro, C., The effect of different time epoch settings on the classification of sheep behaviour using tri-axial accelerometry. Comput. Electron. Agric. 154 (2018), 112–119, 10.1016/j.compag.2018.09.002.
Decandia, M., Rassu, S.P.G., Psiroukis, V., Hadjigeorgiou, I., Fountas, S., Molle, G., Acciaro, M., Cabiddu, A., Mameli, M., Dimauro, C., Giovanetti, V., Evaluation of proper sensor position for classification of sheep behaviour through accelerometers. Small Rumin. Res., 201, 2021, 106445, 10.1016/j.smallrumres.2021.106445.
Deep, S., Zheng, X., 2019. Hybrid Model Featuring CNN and LSTM Architecture for Human Activity Recognition on Smartphone Sensor Data. In: 2019 20th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT). Gold Coast, QLD, Australia, pp. 259-264. https:// doi.org/10.1109/PDCAT46702.2019.00055.
FAO, 2023. FAOSTAT. Available at https://www.fao.org/faostat/en/ (Accessed 13 July 2023).
Fogarty, E.S., Swain, D.L., Cronin, G., Trotter, M., Autonomous on-animal sensors in sheep research: a systematic review. Comput. Electron. Agric. 150 (2018), 245–256, 10.1016/j.compag.2018.04.017.
Fogarty, E.S., Swain, D.L., Cronin, G.M., Moraes, L.E., Trotter, M., Behaviour classification of extensively grazed sheep using machine learning. Comput. Electron. Agric., 169, 2020, 105175, 10.1016/j.compag.2019.105175.
Geng, L., Wu, H., Zhang, F., Xiao, Z., Li, X., Classification method of Heartbeat confusion signals of hatching eggs based on TCN and Transformer. Transactions of the Chinese Society for Agricultural Machinery 54:08 (2023), 296–308.
Giovanetti, V., Decandia, M., Molle, G., Acciaro, M., Mameli, M., Cabiddu, A., Cossu, R., Serra, M.G., Manca, C., Rassu, S.P.G., Dimauro, C., Automatic classification system for grazing, ruminating and resting behaviour of dairy sheep using a tri-axial accelerometer. Livest. Sci. 196 (2017), 42–48, 10.1016/j.livsci.2016.12.011.
Guo, L., Welch, M.C., Dobos, R.C., Kwan, P.W.H., Wang, W., Comparison of grazing behaviour of sheep on pasture with different sward surface heights using an inertial measurement unit sensor. Comput. Electron. Agric. 150 (2018), 394–401, 10.1016/j.compag.2018.05.004.
Hosseininoorbin, S., Layeghy, S., Kusy, B., Jurdak, R., Bishop-Hurley, G.J., Greenwood, P.L., Portmann, M., Deep learning-based cattle behaviour classification using joint time-frequency data representation. Comput. Electron. Agric., 187, 2021, 106241, 10.1016/j.compag.2021.106241.
Hu, S., Ingham, A., Schmoelzl, S., McNally, J., Little, B., Smith, D., Bishop-Hurley, G., Wang, Y.-G., Li, Y., Inclusion of features derived from a mixture of time window sizes improved classification accuracy of machine learning algorithms for sheep grazing behaviours. Comput. Electron. Agric., 179, 2020, 105857, 10.1016/j.compag.2020.105857.
Huang, Z., Xu, W., Yu, K., Bidirectional LSTM-CRF models for sequence tagging. ArXiv., 10.48550/arXiv, 2015, 1508.01991.
Hulbert, I.A.R., Wyllie, J.T.B., Waterhouse, A., French, J., McNulty, D., A note on the circadian rhythm and feeding behaviour of sheep fitted with a lightweight GPS collar. Appl. Anim. Behav. Sci. 60:4 (1998), 359–364, 10.1016/S0168-1591(98)00155-5.
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A., Deep learning for time series classification: a review. Data Min. Knowl. Discov. 33:4 (2019), 917–963, 10.1007/s10618-019-00619-1.
Kim, J., Moon, N., 2022. Dog Behavior Recognition Based on Multimodal Data from a Camera and Wearable Device. Appl. Sci. 2022, 12(6), 3199. 10.3390/app12063199.
Kleanthous, N., Hussain, A., Mason, A., Sneddon, J., Shaw, A., Fergus, P., Chalmers, C., Al-Jumeily, D., 2018. Machine Learning Techniques for Classification of Livestock Behavior. In: Cheng, L., Leung, A.C.S., Ozawa, S. (Eds.), Neural Information Processing, Lecture Notes in Computer Science. Springer, Cham, pp. 304–315. 10.1007/978-3-030-04212-7_26.
Kleanthous, N., Hussain, A., Khan, W., Sneddon, J., Liatsis, P., Deep transfer learning in sheep activity recognition using accelerometer data. Expert Syst. Appl., 207, 2022, 117925, 10.1016/j.eswa.2022.117925.
Li, T., Cui, L., Xu, Z., Hu, R., Joshi, P.K., Song, X., Tang, L., Xia, A., Wang, Y., Guo, D., Zhu, J., Hao, Y., Song, L., Cui, X., Quantitative analysis of the Research trends and areas in grassland remote sensing: a scientometrics analysis of web of science from 1980 to 2020. Remote Sens., 13(7), 2021, 1279, 10.3390/rs13071279.
Liseune, A., den Poel, D.V., Hut, P.R., van Eerdenburg, F.J.C.M., Hostens, M., Leveraging sequential information from multivariate behavioral sensor data to predict the moment of calving in dairy cattle using deep learning. Comput. Electron. Agric., 191, 2021, 106566, 10.1016/j.compag.2021.106566.
Liu, X., Zhang, J., Li, M., Chen, Q., Lu, P., Chen, Q., Wang, S., Feng, X., Observation on behaviors of the first filial generation goat of Zhongwei and angora. Journal of Domestic Animal Ecology 33:02 (2012), 55–59.
Lush, L., Wilson, R.P., Holton, M.D., Hopkins, P., Marsden, K.A., Chadwick, D.R., King, A.J., Classification of sheep urination events using accelerometers to aid improved measurements of livestock contributions to nitrous oxide emissions. Comput. Electron. Agric. 150 (2018), 170–177, 10.1016/j.compag.2018.04.018.
Mao, A., Huang, E., Wang, X., Liu, K., Deep learning-based animal activity recognition with wearable sensors: overview, challenges, and future directions. Comput. Electron. Agric., 211, 2023, 108043, 10.1016/j.compag.2023.108043.
Nobrega, L., Goncalves, P., Antunes, M., Corujo, D., Assessing sheep behavior through low-power microcontrollers in smart agriculture scenarios. Comput. Electron. Agric., 173, 2020, 105444, 10.1016/j.compag.2020.105444.
Ordóñez, F.J., Roggen, D., Deep convolutional and LSTM recurrent neural networks for multimodal Wearable activity recognition. Sensors, 16(1), 2016, 10.3390/s16010115.
Ouyang, Y., Wan, D., Gao, R., Ye, Z., Research on single-phase fault line selection in distribution network based on TCN+Transformer self-attention. Microelectronics & Computer 39:09 (2022), 89–97, 10.19304/j.Issn1000-7180.2021.1331.
Peng, Y., Kondo, N., Fujiura, T., Suzuki, T., Wulandari, Yoshioka, H., Itoyama, E., 2019. Classification of multiple cattle behavior patterns using a recurrent neural network with long short-term memory and inertial measurement units. Comput. Electron. Agric. 157, 247-253. 10.1016/j.compag.2018.12.023.
Prati, R.C., Batista, G.E.A.P.A., Silva, D.F., Class imbalance revisited: a new experimental setup to assess the performance of treatment methods. Knowl. Inf. Syst. 45:1 (2015), 247–270, 10.1007/s10115-014-0794-3.
Price, E., Langford, J., Fawcett, T.W., Wilson, A.J., Croft, D.P., Classifying the posture and activity of ewes and lambs using accelerometers and machine learning on a commercial flock. Appl. Anim. Behav. Sci., 251, 2022, 105630, 10.1016/j.applanim.2022.105630.
Radeski, M., Ilieski, V., Gait and posture discrimination in sheep using a tri-axial accelerometer. Animal 11:7 (2017), 1249–1257, 10.1017/S175173111600255X.
Ren, K., Karlsson, J., Liuska, M., Hartikainen, M., Hansen, I., Jørgensen, G.H., A sensor-fusion-system for tracking sheep location and behaviour. Int. J. Distrib. Sens. Netw., 16(5), 2020, 10.1177/1550147720921776.
Riaboff, L., Aubin, S., Bédère, N., Couvreur, S., Madouasse, A., Goumand, E., Chauvin, A., Plantier, G., Evaluation of pre-processing methods for the prediction of cattle behaviour from accelerometer data. Comput. Electron. Agric., 165, 2019, 104961, 10.1016/j.compag.2019.104961.
Riaboff, L., Shalloo, L., Smeaton, A.F., Couvreur, S., Madouasse, A., Keane, M.T., Predicting livestock behaviour using accelerometers: a systematic review of processing techniques for ruminant behaviour prediction from raw accelerometer data. Comput. Electron. Agric., 192, 2022, 106610, 10.1016/j.compag.2021.106610.
Roux, S. P. l., Wolhuter, R., Niesler, T., 2019. Energy-Aware Feature and Model Selection for Onboard Behavior Classification in Low-Power Animal Borne Sensor Applications. IEEE Sens. J. 19(7), 2722-2734. 10.1109/JSEN.2018.2886890 Sheng, H., Zhang, S., Zuo, L., Duan, G., Zhang, H., Okinda, C., Shen, M., Chen, K., Lu, M., Norton, T., 2020. Construction of sheep forage intake estimation models based on sound analysis. Biosyst. Eng. 192, 144-158. 10.1016/j.biosystemseng.2020.01.024.
Turner, K.E., Thompson, A., Harris, I., Ferguson, M., Sohel, F., Deep learning based classification of sheep behaviour from accelerometer data with imbalance. Inf. Process. Agric. 10:3 (2023), 377–390, 10.1016/j.inpa.2022.04.001.
Turner, K.E., Sohel, F., Harris, I., Ferguson, M., Thompson, A., Lambing event detection using deep learning from accelerometer data. Comput. Electron. Agric., 208, 2023, 107787, 10.1016/j.compag.2023.107787.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I., Attention is all you need. ArXiv., 10.48550/arXiv, 2017, 1706.03762.
Walton, E., Casey, C., Mitsch, J., Vázquez-Diosdado, J.A., Yan, J., Dottorini, T., Ellis, K.A., Winterlich, A., Kaler, J., Evaluation of sampling frequency, window size and sensor position for classification of sheep behaviour. R. Soc. Open Sci., 5(2), 2018, 171442, 10.1098/rsos.171442.
Wang, J., Li, Y., Bork, E.W., Richter, G.M., Chen, C., Hussain Shah, S.H., Mezbahuddin, S., Effects of grazing management on spatio-temporal heterogeneity of soil carbon and greenhouse gas emissions of grasslands and rangelands: monitoring, assessment and scaling-up. J. Clean. Prod., 288, 2021, 125737, 10.1016/j.jclepro.2020.125737.
Wang, K., Wu, P., Cui, H., Xuan, C., Su, H., Identification and classification for sheep foraging behavior based on acoustic signal and deep learning. Comput. Electron. Agric., 187, 2021, 106275, 10.1016/j.compag.2021.106275.
Zhang, F., Bennett, J.A., Zhang, B., Wang, Z., Li, Z., Li, H., Wang, J., Han, G., Zhao, M., Cessation of grazing stabilizes productivity through effects on species asynchrony and stability of shrub/semi-shrub plants in arid grasslands. Agric. Ecosyst. Environ., 348, 2023, 108411, 10.1016/j.agee.2023.108411.
Zhang, Q., Buyantuev, A., Fang, X., Han, P., Li, A., Li, F.Y., Liang, C., Liu, Q., Ma, Q., Niu, J., Shang, C., Yan, Y., Zhang, J., Ecology and sustainability of the inner mongolian grassland: looking back and moving forward. Landsc. Ecol. 35 (2020), 2413–2432, 10.1007/s10980-020-01083-9.
Zhang, H., Sun, Y., Zhao, C., Wang, B., Li, B., Wang, B., Review on typical behavior monitoring and physioloical condition ldentification methods for ruminant livestock. Transactions of the Chinese Society for Agricultural Machinery 54:03 (2023), 1–21.
Zhang, R., Wang, Z., Han, G., Schellenberg, M.P., Wu, Q., Gu, C., Grazing induced changes in plant diversity is a critical factor controlling grassland productivity in the desert steppe, northern China. Agric. Ecosyst. Environ. 265 (2018), 73–83, 10.1016/j.agee.2018.05.014.
Zhou, J., Effects of rotational grazing on digestion and metabolism of tan sheep in typical steppe of the loess plateau. 2023, LanZhou University, LanZhou Master thesis.
Zhou, W., Lu, J., Xiong, Z., Wang, W., Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition. ArXiv., 10.48550/arXiv, 2023, 2303.08356.
Zobel, G., Weary, D.M., Leslie, K., Chapinal, N., von Keyserlingk, M.A.G., Technical note: validation of data loggers for recording lying behavior in dairy goats. J. Dairy Sci. 98:2 (2015), 1082–1089, 10.3168/jds.2014-8635.
