[en] Monitoring the foraging behaviour of ruminants is a key task to improve their productivity and welfare. During the last decades, several monitoring approaches have been proposed based on different types of sensors such as pressure-based, accelerometers and microphones. Among them, microphones have been one of the most promising options because acoustic signals provide comprehensive information about the foraging behaviour. In this work, a fully end-to-end deep architecture is proposed in order to perform both detection and classification tasks of masticatory events in one step, relying only on raw acoustic signals. The main benefit of this novel approach is the substitution of handcrafted preprocessing and feature extraction phases for a pure deep learning approach, which has shown better performance in related fields. Furthermore, different data augmentation techniques have been evaluated to address the data shortness for models development, typical in this field. The results demonstrate that the proposed architecture achieves a F1 score value of 79.82, which represents an increment close to 18% with respect to other state-of-the-art algorithms. Moreover, the proposed data augmentation techniques provide further performance enhancements, emerging as interesting alternatives in this field.
Disciplines :
Computer science Animal production & animal husbandry
Author, co-author :
Ferrero, Mariano; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Vignolo, Leandro D. ; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Vanrell, Sebastián R.; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Martinez-Rau, Luciano S. ; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Chelotti, José ; Université de Liège - ULiège > Département GxABT > Animal Sciences (AS) ; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Galli, Julio R.; Instituto de Investigaciones en Ciencias Agrarias de Rosario, IICAR, Facultad de Ciencias Agrarias, UNR-CONICET, Parque J.F. Villarino, Zavalla, Argentina
Giovanini, Leonardo L.; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
Rufiner, H. Leonardo ; Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina ; Laboratorio de Cibernética, Facultad de Ingeniería, Univ. Nacional de Entre Ríos, Oro Verde, Argentina
Language :
English
Title :
A full end-to-end deep approach for detecting and classifying jaw movements from acoustic signals in grazing cattle
Publication date :
2023
Journal title :
Engineering Applications of Artificial Intelligence
ASaCTeI - Agencia Santafesina de Ciencia, Tecnología e Innovación Nvidia UNR - Universidad Nacional de Rosario CONICET - Consejo Nacional de Investigaciones Científicas y Técnicas UNL - Universidad Nacional del Litoral
Funding text :
This work has been funded by Universidad Nacional del Litoral, CAID, Argentina 50620190100080LI and 50620190100151LI , Universidad Nacional de Rosario, Argentina , projects 2013-AGR216, 2016-AGR266 and 80020180300053UR, Agencia Santafesina de Ciencia, Argentina , Tecnología e Innovación (ASACTEI), Argentina , project IO–2018—00082, Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Argentina , project 2017-PUE sinc(i). Authors would like to thank the dedication and perceptive help by Campo Experimental J. Villarino Dairy Farm staff for their assistance and support during the completion of this study. Authors also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan XP GPU used for this research.
Andriamandroso, A., Bindelle, J., Mercatoris, B., Lebeau, F., A review on the use of sensors to monitor cattle jaw movements and behavior when grazing. Biotechnol. Agron. Soc. Environ. 20 (2016), 273–286.
Andriamandroso, A., Lebeau, F., Bindelle, J., 2015. Changes in biting characteristics recorded using the inertial measurement unit of a smartphone reflect differences in sward attributes. In: 7th Conference on Precision Livestock Farming. pp. 283–289.
Arrieta, A.B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58 (2020), 82–115.
Bahmei, B., Birmingham, E., Arzanpour, S., CNN-RNN and data augmentation using deep convolutional generative adversarial network for environmental sound classification. IEEE Signal Process. Lett. 29 (2022), 682–686.
Balasso, P., Marchesini, G., Ughelini, N., Serva, L., Andrighetto, I., Machine learning to detect posture and behavior in dairy cows: Information from an accelerometer on the animal's left flank. Animals, 11(10), 2021, 2972.
Balch, C., Observations on the act of eating in cattle. Br. J. Nutr. 12:3 (1958), 330–345.
Calamari, L., Soriani, N., Panella, G., Petrera, F., Minuti, A., Trevisi, E., Rumination time around calving: An early signal to detect cows at greater risk of disease. J. Dairy Sci. 97:6 (2014), 3635–3647.
Chelotti, J.O., Vanrell, S.R., Galli, J.R., Giovanini, L.L., Rufiner, H.L., A pattern recognition approach for detecting and classifying jaw movements in grazing cattle. Comput. Electron. Agric. 145 (2018), 83–91.
Chelotti, J.O., Vanrell, S.R., Milone, D.H., Utsumi, S.A., Galli, J.R., Rufiner, H.L., Giovanini, L.L., A real-time algorithm for acoustic monitoring of ingestive behavior of grazing cattle. Comput. Electron. Agric. 127 (2016), 64–75.
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y., 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In: Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014). arXiv.
De Boever, J., Andries, J., De Brabander, D., Cottyn, B., Buysse, F., Chewing activity of ruminants as a measure of physical structure—A review of factors affecting it. Anim. Feed Sci. Technol. 27:4 (1990), 281–291.
Ding, L., Lv, Y., Jiang, R., Zhao, W., Li, Q., Yang, B., Yu, L., Ma, W., Gao, R., Yu, Q., Predicting the feed intake of cattle based on jaw movement using a triaxial accelerometer. Agriculture, 12(7), 2022, 899.
Fogarty, E.S., Swain, D.L., Cronin, G.M., Moraes, L.E., Trotter, M., Behaviour classification of extensively grazed sheep using machine learning. Comput. Electron. Agric., 169, 2020, 105175.
Frost, A.R., Schofield, C.P., Beaulah, S.A., Mottram, T.T., Lines, J.A., Wathes, C.M., A review of livestock monitoring and the need for integrated systems. Comput. Electron. Agric. 17:2 (1997), 139–159.
Giovanetti, V., Decandia, M., Molle, G., Acciaro, M., Mameli, M., Cabiddu, A., Cossu, R., Serra, M., Manca, C., Rassu, S., et al. Automatic classification system for grazing, ruminating and resting behaviour of dairy sheep using a tri-axial accelerometer. Livestock Sci. 196 (2017), 42–48.
Glorot, X., Bengio, Y., Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010, JMLR Workshop and Conference Proceedings, 249–256.
Hershey, S., Chaudhuri, S., Ellis, D.P., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., Seybold, B., et al. CNN architectures for large-scale audio classification. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, IEEE, 131–135.
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R., Improving neural networks by preventing co-adaptation of feature detectors. 2012.
Hoxhallari, K., Purcell, W., Neubauer, T., 2022. Precision Livestock Farming. In: 10th European Conference on Precision Livestock Farming.
Kamminga, J.W., Le, D.V., Meijers, J.P., Bisby, H., Meratnia, N., Havinga, P.J., Robust sensor-orientation-independent feature selection for animal activity recognition on collar tags. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2:1 (2018), 1–27.
Khamees, A.A., Hejazi, H.D., Alshurideh, M., Salloum, S.A., Classifying audio music genres using CNN and RNN. Hassanien, A.-E., Chang, K.-C., Mincong, T., (eds.) Advanced Machine Learning Technologies and Applications, 2021, Springer International Publishing, Cham, 315–323.
Kingma, D.P., Ba, J., Adam: A method for stochastic optimization. 2014.
Kokalis, C.-C.A., Tasakos, T., Kontargyri, V.T., Siolas, G., Gonos, I.F., Hydrophobicity classification of composite insulators based on convolutional neural networks. Eng. Appl. Artif. Intell., 91, 2020, 103613.
Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Commun. ACM 60:6 (2017), 84–90.
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. Proc. IEEE 86:11 (1998), 2278–2324.
Li, D., Liu, J., Yang, Z., Sun, L., Wang, Z., Speech emotion recognition using recurrent neural networks with directional self-attention. Expert Syst. Appl., 173, 2021, 114683.
Li, C., Tokgoz, K.K., Fukawa, M., Bartels, J., Ohashi, T., Takeda, K.-i., Ito, H., Data augmentation for inertial sensor data in CNNs for cattle behavior classification. IEEE Sens. Lett. 5:11 (2021), 1–4.
Li, G., Xiong, Y., Du, Q., Shi, Z., Gates, R.S., Classifying ingestive behavior of dairy cows via automatic sound recognition. Sensors, 21(15), 2021.
Lim, S.J., Jang, S.J., Lim, J.Y., Ko, J.H., Classification of snoring sound based on a recurrent neural network. Expert Syst. Appl. 123 (2019), 237–245.
Lu, R., Duan, Z., Zhang, C., 2018. Multi-Scale Recurrent Neural Network for Sound Event Detection. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. ICASSP, pp. 131–135.
Martinez-Rau, L.S., Chelotti, J.O., Vanrell, S.R., Galli, J.R., Utsumi, S.A., Planisich, A.M., Rufiner, H.L., Giovanini, L.L., A robust computational approach for jaw movement detection and classification in grazing cattle using acoustic signals. Comput. Electron. Agric., 192, 2022, 106569.
Matsui, K., Okubo, T., A method for quantification of jaw movements suitable for use on free-ranging cattle. Appl. Anim. Behav. Sci. 32:2–3 (1991), 107–116.
Meng, J., Wang, X., Wang, J., Teng, X., Xu, Y., A capsule network with pixel-based attention and BGRU for sound event detection. Digit. Signal Process., 123, 2022, 103434.
Mesaros, A., Heittola, T., Virtanen, T., Plumbley, M.D., Sound event detection: A tutorial. IEEE Signal Process. Mag. 38:5 (2021), 67–83.
Milone, D.H., Galli, J.R., Cangiano, C.A., Rufiner, H.L., Laca, E.A., Automatic recognition of ingestive sounds of cattle based on hidden Markov models. Comput. Electron. Agric. 87 (2012), 51–55.
Milone, D.H., Rufiner, H.L., Galli, J.R., Laca, E.A., Cangiano, C.A., Computational method for segmentation and classification of ingestive sounds in sheep. Comput. Electron. Agric. 65:2 (2009), 228–237.
Monteiro, A., Santos, S., Gonçalves, P., Precision agriculture for crop and livestock farming—Brief review. Animals, 11(8), 2021, 2345.
Nanni, L., Paci, M., Brahnam, S., Lumini, A., Comparison of different image data augmentation approaches. J. Imaging, 7(12), 2021, 254.
Navon, S., Mizrach, A., Hetzroni, A., Ungar, E.D., Automatic recognition of jaw movements in free-ranging cattle, goats and sheep, using acoustic monitoring. Biosyst. Eng. 114:4 (2013), 474–483.
Neethirajan, S., The role of sensors, big data and machine learning in modern animal farming. Sens. Bio-Sens. Res., 29, 2020, 100367.
Nydegger, F., Gyga, L., Egli, W., 2011. Automatic measurement of jaw movements in ruminants by means of a pressure sensor. In: International Conference on Agricultural Engineering. p. 27.
Oudshoorn, F.W., Cornou, C., Hellwing, A.L.F., Hansen, H.H., Munksgaard, L., Lund, P., Kristensen, T., Estimation of grass intake on pasture for dairy cows using tightly and loosely mounted di- and tri-axial accelerometers combined with bite count. Comput. Electron. Agric. 99 (2013), 227–235.
Papakipos, Z., Bitton, J., Augly: Data augmentations for robustness. 2022 arXiv preprint arXiv:2201.06494.
Paudyal, S., Maunsell, F.P., Richeson, J.T., Risco, C.A., Donovan, D.A., Pinedo, P.J., Rumination time and monitoring of health disorders during early lactation. Animal 12:7 (2018), 1484–1492.
Penning, P.D., A technique to record automatically some aspects of grazing and ruminating behaviour in sheep. Grass Forage Sci. 38:2 (1983), 89–96.
Petmezas, G., Cheimariotis, G.-A., Stefanopoulos, L., Rocha, B., Paiva, R.P., Katsaggelos, A.K., Maglaveras, N., Automated lung sound classification using a hybrid CNN-LSTM network and focal loss function. Sensors, 22(3), 2022, 1232.
Ramirez, A.E., Donati, E., Chousidis, C., A siren identification system using deep learning to aid hearing-impaired people. Eng. Appl. Artif. Intell., 114, 2022, 105000.
Riaboff, L., Shalloo, L., Smeaton, A., Couvreur, S., Madouasse, A., Keane, M., Predicting livestock behaviour using accelerometers: A systematic review of processing techniques for ruminant behaviour prediction from raw accelerometer data. Comput. Electron. Agric., 192, 2022, 106610.
Rombach, M., Südekum, K.-H., Münger, A., Schori, F., Herbage dry matter intake estimation of grazing dairy cows based on animal, behavioral, environmental, and feed variables. J. Dairy Sci. 102:4 (2019), 2985–2999.
Ruuska, S., Kajava, S., Mughal, M., Zehner, N., Mononen, J., Validation of a pressure sensor-based system for measuring eating, rumination and drinking behaviour of dairy cattle. Appl. Anim. Behav. Sci. 174 (2016), 19–23.
Schuster, M., Paliwal, K.K., Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45:11 (1997), 2673–2681.
Shorten, C., Khoshgoftaar, T.M., A survey on image data augmentation for deep learning. J. Big Data, 6(1), 2019.
Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition. 2014 arXiv preprint arXiv:1409.1556.
Sokolova, M., Lapalme, G., A systematic analysis of performance measures for classification tasks. Inf. Process. Manage. 45:4 (2009), 427–437.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z., 2016. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2818–2826.
Tani, Y., Yokota, Y., Yayota, M., Ohtani, S., Automatic recognition and classification of cattle chewing activity by an acoustic monitoring method with a single-axis acceleration sensor. Comput. Electron. Agric. 92 (2013), 54–65.
Ungar, E.D., Ravid, N., Zada, T., Ben-Moshe, E., Yonatan, R., Baram, H., Genizi, A., The implications of compound chew–bite jaw movements for bite rate in grazing cattle. Appl. Anim. Behav. Sci. 98:3–4 (2006), 183–195.
Vanrell, S.R., Chelotti, J.O., Bugnon, L.A., Rufiner, H.L., Milone, D.H., Laca, E.A., Galli, J.R., Audio recordings dataset of grazing jaw movements in dairy cattle. Data Brief, 30, 2020, 105623.
Werner, J., Leso, L., Umstatter, C., Niederhauser, J., Kennedy, E., Geoghegan, A., Shalloo, L., Schick, M., O'Brien, B., Evaluation of the RumiWatchSystem for measuring grazing behaviour of cows. J. Neurosci. Methods 300 (2018), 138–146.
