Road Junction

Goal

The goal of this practice is to implement the logic of a navigation algorithm for an automated vehicle. The vehicle must Stop at the T joint, where there is a stop sign, wait until there are no cars and pass once the road is clear.

Instructions

This is the preferred way for running the exercise.

Installing and Launching

Download Docker. Windows users should choose WSL 2 backend Docker installation if possible, as it has better performance than Hyper-V.

Pull the current distribution of RoboticsBackend:

  docker pull jderobot/robotics-backend:latest

In order to obtain optimal performance, Docker should be using multiple CPU cores. In case of Docker for Mac or Docker for Windows, the VM should be assigned a greater number of cores.
It is recommended to use the latest image. However, older distributions of RoboticsBackend can be found here.

How to perform the exercises?

Start a new docker container of the image and keep it running in the background:

docker run --rm -it -p 7164:7164 -p 2303:2303 -p 1905:1905 -p 8765:8765 -p 6080:6080 -p 1108:1108 -p 7163:7163 jderobot/robotics-backend

On the local machine navigate to 127.0.0.1:7164/ in the browser and choose the desired exercise.
Wait for the Connect button to turn green and display “Connected”. Click on the “Launch” button and wait for some time until an alert appears with the message Connection Established and button displays “Ready”.
The exercise can be used after the alert.

Enable GPU Acceleration

Follow the advanced launching instructions from here.

Where to insert the code?

In the launched webpage, type your code in the text editor,

from GUI import GUI
from HAL import HAL
# Enter sequential code!


while True:
    # Enter iterative code!

Using the Interface

Control Buttons: The control buttons enable the control of the interface. Play button sends the code written by User to the Robot. Stop button stops the code that is currently running on the Robot. Save button saves the code on the local machine. Load button loads the code from the local machine. Reset button resets the simulation (primarily, the position of the robot).
Brain and GUI Frequency: This input shows the running frequency of the iterative part of the code (under the while True:). A smaller value implies the code runs less number of times. A higher value implies the code runs a large number of times. The numerator is the one set as the Measured Frequency who is the one measured by the computer (a frequency of execution the computer is able to maintain despite the commanded one) and the input (denominator) is the Target Frequency which is the desired frequency by the student. The student should adjust the Target Frequency according to the Measured Frequency.
RTF (Real Time Factor): The RTF defines how much real time passes with each step of simulation time. A RTF of 1 implies that simulation time is passing at the same speed as real time. The lower the value the slower the simulation will run, which will vary depending on the computer.
Pseudo Console: This shows the error messages related to the student’s code that is sent. In order to print certain debugging information on this console. The student can use the print() command in the Editor.

Robot API

from HAL import HAL - to import the HAL(Hardware Abstraction Layer) library class. This class contains the functions that sends and receives information to and from the Hardware(Gazebo).
from GUI import GUI - to import the GUI(Graphical User Interface) library class. This class contains the functions used to view the debugging information, like image widgets.
HAL.getImage('left') - to get the left image
HAL.getImage('right') - to get the right image
HAL.getImage('center') - to get the central image
HAL.getV() - to get the linear velocity
HAL.getW() - to get the angular velocity
HAL.setV() - to set the linear velocity
HAL.setW() - to set the angular velocity
HAL.getPose3D() - to obtain the position of the robot
HAL.getYaw() - to obtain the orientation of the robot
HAL.setPose3D() - to set the position of the robot
HAL.getTemplate() - to get the image template of the stop sign
GUI.showImages(imageLeft, imageCentral, imageRight) - to visualize the images in the webpage

Videos

Theory

This exercise mostly revolves around simple Computer Vision and Control mechanisms. Let’s start with the Computer Vision tasks:

Image Segmentation

Image Segmentation is the process of partitioning a digital image into multiple segments. Segmentation is carried out to simplify the representation of information we receive from an input image. There are many ways to carry out image segmentation, but it all depends on the task at hand.

For extremely cluttered images, segmentation is carried out by the use of Convolutional Neural Networks. Some of the most useful Neural Networks developed for this task are: UNet, FPN and R-CNNs.

Feature Pyramid Network — Neural Networks

Some simpler methods involve the use of Clustering Algorithms like K-Means Clustering. The simplest of all methods is the use of Color Filters.

Color Filters involve the use of color spaces and their intensity values. By thresholding the input image by appropriate intensity values, we segment a specific color part of an image. For example, the Stop Sign in the exercise is a very specific red color which can be easily segmented. For more information about this topic, the Robotics Academy already has an exercise based on Color Filter.

Motion Detection

Motion detection is the process of detecting a change in the position of an object relative to its surroundings or a change in the surroundings relative to an object. Difference of Frames is the simplest of all motion detection algorithms. By taking the difference between the frame of reference and the current frame, motion can be very easily detected.

Control

We require the use of control mechanisms in order to make the Car follow the road and take turns.

PID is an excellent control algorithm which works by calculating the error and making adjustments to keep the error as minimum as possible. A very simple example would be, in case of a quadcopter that is required to maintain a particular height: If the height of the quadcopter is extremely far away from the required height(very high or very low), the quadcopter has to make large adjustments in it’s rotor speed. If the height is near the required value, the quadcopter has to make small adjustments. This video illustrates the concept very well.

Follow Line is an exercise based completely on the concept of PID.

The physical equation concerning rotational kinematics is v = ωr . This equation governs the relation between the radius of curvature, the magnitude of velocity and the magnitude of angular velocity. In order to increase the radius of curvature, we need to increase the velocity and decrease the magnitude of angular velocity. In order to decrease the radius of curvature, we need to decrease the velocity and increase the magnitude of angular velocity. This is the fundamental equation that allows our autonomous car to take effective turns.

Hints

Simple hints provided to help you solve the car_junction exercise.

Following the Road

Color Segmentation is the way to go to detect roads. Thresholding and then applying morphological operations can easily help us detect the roads in the image. The centroid of the generated binary blob can be easily followed by the car. However, there are tiny problems with this approach that can be addressed by using tricks(which the reader is expected to figure out on his/her own).

A really interesting implementation would be, by means of lane detection. Here is a really good implementation. But again, there are some problems in this approach as well. (Do make a video and share with us in case someone solves using this)

Stop Sign

The stop sign is of a very particular red color, which clearly stands out in the whole gazebo simulation. So, color filters are the best way to approach. Check out this video for more details.

Turning

Turning is a challenging part of this exercise. The parameters have to be tuned in a specific way to carry out the turn, and then the control should be passed to the follow road method in order to continue with the task. The physical equation given above can simplify the process of setting the parameters.

Illustrations

Contributors

Contributors: Irene Lope, Alberto Martín, Francisco Rivas, Francisco Pérez, Jose María Cañas, Nacho Arranz.
Maintained by Jessica Fernández, Vanessa Fernández.