Working toward Solving Safety Issues in Human–Robot Collaboration: A Case Study for Recognising Collisions Using Machine Learning Algorithms

Justyna Patalas-Maliszewska, Adam Dudek, Grzegorz Pajak, Iwona Pajak

The monitoring and early avoidance of collisions in a workspace shared by collaborative robots (cobots) and human operators is crucial for assessing the quality of operations and tasks completed within manufacturing. A gap in the research has been observed regarding effective methods to automatically assess the safety of such collaboration, so that employees can work alongside robots, with trust. The main goal of the study is to build a new method for recognising collisions in workspaces shared by the cobot and human operator. For the purposes of the research, a research unit was built with two UR10e cobots and seven series of subsequent of the operator activities, specifically: (1) entering the cobot’s workspace facing forward, (2) turning around in the cobot’s workspace and (3) crouching in the cobot’s workspace, taken as video recordings from three cameras, totalling 484 images, were analysed. This innovative method involves, firstly, isolating the objects using a Convolutional Neutral Network (CNN), namely the Region-Based CNN (YOLOv8 Tiny) for recognising the objects (stage 1). Next, the Non-Maximum Suppression (NMS) algorithm was used for filtering the objects isolated in previous stage, the k-means clustering method and Simple Online Real-Time Tracking (SORT) approach were used for separating and tracking cobots and human operators (stage 2) and the Convolutional Neutral Network (CNN) was used to predict possible collisions (stage 3). The method developed yields 90% accuracy in recognising the object and 96.4% accuracy in predicting collisions accuracy, respectively. The results achieved indicate that understanding human behaviour working with cobots is the new challenge for modern production in the Industry 4.0 and 5.0 concept.