Vision-based Control of an Industrial Robot Arm

The object of this project is to design and build a stereoscopic vision system capable of tracking a fast-moving object and predicting its future positions, enabling a robot arm to interact with it and to perform a dexterous task, i.e., bouncing an unconstrained ball.
A colour-based object detection has been used and the position in 3D coordinates has been estimated with two different strategies: a geometrical triangulation and a feedforward neural network trained for this purpose; these strategies have been compared with some experiments, whose results suggested that a neural network might be a good alternative in some circum- stances in which to locate with precision the cameras relatively to the robot arm can be difficult.
A curve fitting algorithm, based on a simplified physical model neglecting air friction and Magnus force (the precision and the robustness of this model have been tested and the tests showed that the approximations introduced were acceptable for this application), predicted the future positions of the tracked object.
On the base of the predicted hitting position, a paddle moved by the robot arm is oriented and positioned according to a strategy meant to stabilize the bouncing by minimizing the H2 norm of the system, which can be interpreted as the gain of the system when the input is white noise.
The robot motion has been performed with different strategies, having different performances that have been tested and compared; one of these strategies involved the need of a custom controller, working on the top of the internal controller of the robot, and of the inverse kinematics of the arm, which was calculated using a neural network.