부산대학교 도서관

In the evolving world of robotics, the way humans and robots interact plays a crucial role in the overall system performance and user experience. Traditionally, robotic control for accomplishing tasks have been dominated by programming languages like ROBOFORTH or touchpad interfaces. However, these methods often create barriers to user interaction. This study explores the potential of hand gesture recognition, specifically focusing on the deep-learning MediaPipe library’s real-time hand-landmark detection capabilities. We present a system that combines MediaPipes abilities with controlling the ST Robotics R12 robot, allowing real-time hand gestures to be translated into robotic drawing actions and tasks. This integration offers a natural and interactive communication medium that could revolutionize how humans carry out tasks with robots. By leveraging MediaPipes computer vision and deep learning techniques, we successfully controlled the R12 robot through gesture-based interactions. The system effectively translated hand gestures into robotic drawing functions and specific tasks such as object relocation. However, future improvements should concentrate on enhancing precision, reevaluating reliance on ROBOFORTH and exploring MATLAB Simulink’s potential for advanced gesture-based control algorithms.