Arreta partekatzen Pepper robot sozialarekin

Abstract

For social robots to show social attention it is essential to be able to detect the object of interest, if any, during human-robot interactions. This work shows an attempt to locate and identify the object in the human’s visual focus of attention and integrate the system into the social robot Pepper. YOLOv8 was used to detect the person through the image and extract their face. In order to get the Point of Regard, a VGG16 convolutional network has been adapted and trained for the specific regression task,
showing acceptable results. The orientation of the Point of Regard with respect to the head is used to obtain a headmap that allows to extract the object of interest (and its identity) among the ones obtained by the ResNet101, using depth segmentation and the intersection of union strategy. This system has been transferred to the robot that develops a shared attention behavior by performing the proper head motion accompanied with a verbal response that makes the human aware of the situation. A video shows the result of the general behavior.