Optical Flow Teleop

Goal

In this practice the intention is to develop an optical flow algorithm to teleoperate the robot using the images obtained from a webcam.

Gallery

Note: If you haven’t, take a look at the user guide to understand how the installation is made, how to launch a RoboticsBackend and how to perform the exercises.

Robot API

• from HAL import HAL - to import the HAL library class. This class contains the functions that receives information from the webcam.
• from GUI import GUI - to import the GUI (Graphical User Interface) library class. This class contains the functions used to view the debugging information, like image widgets.
• HAL.getImage() - to get the image
• GUI.showImage() - allows you to view a debug image or with relevant information
• HAL.motors.sendV() - to set the linear speed
• HAL.motors.sendW() - to set the angular velocity

Videos

Theory

Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is 2D vector field where each vector is a displacement vector showing the movement of points from first frame to second. Consider the image below:

It shows a ball moving in 5 consecutive frames. The arrow shows its displacement vector

Optical flow works on several assumptions: 1. The pixel intensities of an object do not change between consecutive frames. 2. Neighbouring pixels have similar motion.

Example of optical flow motion estimation

Optical flow has many applications in areas like: - Structure from Motion - Video Compression - Video Stabilization

Contributors

• Contributors: Jose María Cañas, David Valladares
• Maintained by David Valladares

References

1. https://docs.opencv.org/3.4/d4/dee/tutorial_optical_flow.html
2. https://medium.com/@jijupax/connect-the-webcam-to-docker-on-mac-or-windows-51d894c44468