GSoC 2021

JdeRobot is a software toolkit for developing robotics and computer vision applications. These domains include sensors (for instance, cameras), actuators, and intelligent software in between. It is mostly written in C++, Python and JavaScript languages and provides a distributed component-based programming environment where the application program is made up of a collection of several concurrent asynchronous components. They may run in different computers and are connected using ICE communication middleware or ROS messages. Components interoperate through explicit interfaces. Each one may have its own independent Graphical User Interface or none at all. It is ROS-friendly and full compatible with ROS-Melodic (and Gazebo9)

Selected students

In the year 2021, the students selected for the Google Summer of Code have been the following:

GSoC 2021 - Arkajyoti Basak

Arkajyoti Basak

Robotics Academy: new drone based exercises

Project #2

GSoC 2021 - Chandan Kumar

Chandan Kumar

Migrating industrial robot manipulation exercises to web server

Project #3

GSoC 2021 - Shashwat Dalakoti

Shashwat Dalakoti

Robotics-Academy: exercise using Deep Learning for Visual Detection

Project #4

GSoC 2021 - Suhas Gopal

Suhas Gopal

Shifting VisualCircuit to a web server

Project #7

GSoC 2021 - Utkarsh Mishra

Utkarsh Mishra

Autonomous Driving drone with Gazebo using Deep Learning techniques

Project #8

GSoC 2021 - Siddharth Saha

Siddharth Saha

Robotics Academy: multirobot version of the Amazon warehouse exercise in ROS2

Project #10

Ideas list

JdeRobot simplifies the access to hardware devices from the control program. Getting sensor measurements is as simple as calling a local function, and ordering motor commands as easy as calling another local function. The platform attaches those calls to the remote invocation on the components connected to the sensor or the actuator devices. They can be connected to real sensors and actuators or simulated ones, both locally or remotely using the network. Those functions build the API for the Hardware Abstraction Layer. The robotic application get the sensor readings and order the actuator commands using it to unfold its behavior. Several driver components have been developed to support different physical sensors, actuators and simulators. The drivers are used as components installed at will depending on your configuration. They are included in the official release.

This open source project needs further development in these topics:

Project #1: Robotics Academy: new computer vision exercises

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises using Python language where the student has to code the behavior of a given (either simulated or real) robot to fit some task related to robotics or computer vision. It uses standard middleware and libraries as ROS and openCV, so the student learn the state of the art tools.

Currently there are many robotics exercises but few Computer Vision ones. Just a Color Filter and a 3D reconstruction exercise. The goal of this project is to design and program several Computer Vision exercises in JdeRobot Academy which cover an image processing introductory course using real cameras. It may take the available OpenCVdemo component as starting point.

ROS2 use for these exercises will be explored.

  • Expected results: ROS-based exercises involving Computer Vision
  • Knowledge Prerequisite: Python programming skills, OpenCV library.
  • Mentors: Luis Roberto Morales (lr.morales.iglesias AT gmail.com)

Project #2: Robotics Academy: new drone based exercises

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises using Python language where the student has to code the behavior of a given (either simulated or real) robot to fit some task related to robotics or computer vision. It uses standard middleware and libraries as ROS and openCV, so the student learn the state of the art tools.

Currently there are 9 drone exercises available in Gazebo-9 and ROS-melodic. The goal of this project is to migrate the drone exercises from v2.1 (ROs node based) to v2.3 (Web/Docker based) of the Robotics Academy. If time permits, the project may explore the creation of the new drone based exercises as well such as drone inspection or drone package delivery, along with their requisite gazebo models.

You will work with Gazebo-11 and ROS-noetic. Also, ROS2 use for these exercises will be explored.

  • Expected results: ROS-based drones exercises
  • Knowledge Prerequisite: Python, OpenCV library and ROS.
  • Mentors: Pedro Arias (pedroariasperez96 AT gmail.com) and Nikhil Khedekar (nikhilvkhedekar AT gmail.com)

Project #3: Migrating industrial robot manipulation exercises to web server

Brief Explanation: Last year, the Robotics-Academy group made a lot of effort on shifting previous exercises from the desktop application to using a Docker image with all dependencies and Web-template. In this way, it can support the usage in multiple platforms and simplify the process of installing several dependencies.

There are currently three industrial robot exercise prototypes implemented with ROS Melodic, MoveIt, and Gazebo-9. The goal of this project is to migrate these desktop-version industrial robot exercises to an online web server. The example of Web-based exercises can be found here. Those exercises with v2.3 are the latest version Web-based exercises.

These prototypes are already workable but still not robust, especially during grasping, so improving the behavior of grasping simulation is another expectation for this project. Better motion planning performance will also benefit the usage of these exercises. Any performance improvement ideas can be a part of the project to get these exercises better ready to use, but the core of this project is still the migration.

  • Expected results: Web-based industrial robot exercises
  • Knowledge Prerequisites: Python, ROS and JavaScript
  • Mentors: Yijia Wu ( sixerwyj AT gmail.com ) and Puskhal Katara (katarapushkal AT gmail.com)

Project #4: Robotics-Academy: exercise using Deep Learning for Visual Detection

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises. Up until now, the students had to code in Python the behavior of a given (either simulated or real) robot to fit some task related to robotics or computer vision.

The idea for this project is to develop a new deep learning exercise within the Robotics-Academy context. More specifically, we will be building a template for visual detection exercises (e.g. human pose estimation or traffic sign recognition) that must be solved by providing a trained model for the particular task. In that way, instead of asking the user to code the solution in a web-based editor, he or she will have to upload a deep learning model that matches our defined set of inputs (e.g. single or multiple images) and outputs (e.g. 2D coordinates). In this project, we will be:

  • Updating our web interface for accepting models trained with PyTorch/Tensorflow as input.
  • Building new widgets for monitoring results for the particular exercise.
  • Coding the core application that will feed the trained model with input data and return the result.
  • Training a naïve model that allow us to show how the exercise can be solved.

In the following videos examples of one of our current web-based exercises and a visual detection task solved using deep learning are shown:

  • Expected results: a web-based exercise for solving a visual detection task using deep learning
  • Knowledge Prerequisite: Python, OpenCV, PyTorch/Tensorflow
  • Mentors: David Pascual ( d.pascualhe AT gmail.com ) and Pankhuri Vanjani (pankhurivanjani AT gmail.com)

Project #5: Robotics-Academy: exercise using Deep Learning for Visual Control

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises. Up until now, the students had to code in Python the behavior of a given (either simulated or real) robot to fit some task related to robotics or computer vision.

The idea for this project is to develop a new deep learning exercise for visual robotic control within the Robotics-Academy context. In that sense, we will be building a web-based interface that allows the user to input a trained model that matches our defined input (e.g. a camera installed on a drone) and outputs (e.g. linear speed and angular velocity). The controlled robot and its environment will be simulated using Gazebo. In this project, we will be:

  • Updating our web interface for accepting models trained with PyTorch/Tensorflow as input.
  • Building new widgets for monitoring results for the particular exercise.
  • Getting a simulated environment ready.
  • Coding the core application that will feed the trained model with input data and send back the results.
  • Training a naïve model that allow us to show how the exercise can be solved.

In the following videos examples of one of our current web-based exercises and a visual control task solved using deep learning are shown:

  • Expected results: a web-based exercise for robotic visual control using deep learning
  • Knowledge Prerequisite: Python, OpenCV, PyTorch/Tensorflow, Gazebo
  • Mentors: David Pascual ( d.pascualhe AT gmail.com ) and Pankhuri Vanjani (pankhurivanjani AT gmail.com)

Project #6: Robotics-Academy: Web template improvements to use state of the art frontend technologies.

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises. Up until now, the users had to install all the necessary dependencies in their own computers for each exercise they wanted to run. In the last year, the Robotics-Academy group has worked towards the use of a Docker image with all the dependencies already installed in it and the use of Web-templates.

Web-templates consist of the use of the web browser as the user interface. The browser connects with the Docker image application in order to run the simulation and to allow the users to send their code and see the results of the simulation. Nowadays, the Web-templates have been developed using plain HTML, Javascript and CSS and we believe that there is potential improvementes to the interfacte by using advanced frontend technologies such as React, Vue, Angular or Flutter.

Color filter Drone template

  • Expected results: Upgrade the current Web-templates to use a more advanced frontend technologies.
  • Knowledge Prerequisite: Python, JavaScript, HTML, CSS
  • Mentors: David Roldán (david.roldan AT urjc.es) and Jose María Cañas (josemaria.plaza AT urjc.es)

Project #7: Shifting VisualCircuit to a web server

Brief Explanation: VisualCircuit allows users to program robotic intelligence using a visual language which consists of blocks and wires, as electronic circuits. We want to shift this tool from a desktop application to an online web server so anyone can try it out without going through the process of installation. The main area of focus would be on the frontend. The current desktop release is based on NodeJS and Electron. We want to shift that to a web server using Django. So basically you will need to develop a UI similar to the one in the current release which synthesizes that circuit into a .vc file (JSON file) which is then read by a local machine using the Python backend (already developed). You can read further about the tool on the website. In this way, VisualCircuit will be cross-platform, attract new users and be very convenient to use.

Requirements:

  • A Dockerfile for installation of required dependencies on a local machine.
  • A Small UI wrapper around the current Python backend that can execute updated circuits on a local machine.
  • Shifting VisualCircuit to the web.
  • Users can create an account which provides online storage to save their design files.

Automated Build on Local Machine through the Website: Web server → Downloaded circuit file → Check for updated .vc file (WatchDog) → Execute updated file

  • Expected results: An online working release on VisualCircuit on a Web Server.
  • Knowledge Prerequisite: Python, Django and some basic JavaScript knowledge..
  • Mentors: Muhammad Taha (mtsg09 AT gmail.com) and JoséMaría Cañas (josemaria.plaza AT urjc.es)

Project #8: Autonomous Driving drone with Gazebo using Deep Learning techniques

Brief Explanation: Behavior Studio is a tool used to compare different autonomous driving architectures. It uses Gazebo, a well-known really powerful robotics simulator and ROS Noetic.

Currently we are capable of autonomously driving using deep learning or reinforcement learning techniques using a F1 robot on different circuits (check out the videos below).

In this project, we would like to expand the current state with a new scenario for a drone that can learn how to drive following a line over the environment. For this purpose, the drone will be using its vision capabilities and with the help of deep learning algorithms that we will be implementing using PyTorch, it should be capable of driving autonomously across the scenario under different conditions. We will be comparing different deep learning approaches based on CNN and LSTMs. Additionally, the current Behavior Studio environment should be upgraded to ROS2.

  • Expected results: New drone scenario with an autonomous driving drone using Gazebo and PyTorch.
  • Knowledge Prerequisites: Python and Deep Learning knowledge (specially PyTorch).
  • Mentors: Sergio Paniego Blanco (sergiopaniegoblanco AT gmail.com)

Project #9: Robotics-Academy: exercise on Localization using Particle Filters

Brief Explanation: JdeRobot’s Robotics-Academy is a framework for learning robotics and computer vision. It consists of a collection of robot programming exercises using Python language where the student has to code the behavior of a given (either simulated or real) robot to fit some task related to robotics or computer vision. It is ROS and Gazebo based.

Autonomous navigation and obstacle avoidance are the two most important problems in mobile robotics. Navigation consists of 3 fundamental subproblems: Self-Localization, Path-Planning and Map Generation. The idea for this project is to develop and update this exercise draft and get a ready to use exercise, using the current Web Template Design.

The goal of the exercise would be to implement the logic that would allow a robot to self locate in space by means of a laser or a camera sensor, in order to estimate its position in a given space (of which a map would be provided).

The first video shows the current web-template design and the second video describes particle filters for localization.

  • Expected results: exercise on particle filters
  • Knowledge Prerequisite: Python, Javascript and ROS
  • Mentors: Sakshay Mahna ( sakshum19 AT gmail.com ) and ()

Project #10: Robotics Academy: multirobot version of the Amazon warehouse exercise in ROS2

Brief Explanation: Swarm robots have gained the importance in the domain of Intelligent Systems with their applications being extended to warehouses. One such popular multi-robot systems could be found in Amazon warehouses, from where this project borrows its inspiration. The goal of this project is to build a multi-robot navigation exercise extending the already existing Amazon warehouse exercise (demonstrated in video) to a fleet of robots and migrating its infrastructure to ROS2.

ROS2 has put forward several improvements over ROS with changes in middleware and software architecture in many aspects. One of its target applications is multi-robot agents. In this project, we would focus on developing new exercises with ROS2 for multi robots (starting with 2 - 3 robots and then increasing complexity). The Navigation stack of ROS2 introduces better features to implement control of a swarm of robots. with a support for multiple robots in the same ROS domain, navigating independently. The idea is to use the ROS2 navigation stack for the control system and planning to create new robotics exercises for JdeRobot Academy. This will involve the use of functionalities such as collision avoidance between robots, global path planning, and Multi-robot coordination for developing application based exercises.

For more information on ROS2 based exercises, have a look at GSoC 2020 project and corresponding academy exercises 1 and 2.

  • Expected results: A new multirobot exercise in JdeRobot-Academy
  • Knowledge Prerequisite: C++, Python programming skills, experience with ROS.
  • Good to know:* ROS2
  • Mentors: Pankhuri Vanjani (pankhurivanjani AT gmail.com) and Shreyas Gokhale (shreyas6gokhale AT gmail.com).

Application instructions for GSoC-2021

We welcome students to contact relevant mentors before submitting their application into GSoC official website. If in doubt for which project(s) to contact, send a message to jderobot AT gmail.com We recommend browsing previous GSoC student pages to look for ready-to-use projects, and to get an idea of the expected amount of work for a valid GSoC proposal.

Requirements

  • Git experience
  • C++ and Python programming experience (depending on the project)

Programming tests

Project #1 #2 #3 #4 #5 #6 #7 #8 #9 #10
Academy (A) X X X X X X X X X X
C++ (B) X O X X X O X X X X
Python (C) X X X X X X X X X X
ROS2 (D) O O O O O - - - - X


Where:  
* Not applicable
X Mandatory
O Optative

Before accepting any proposal all candidates have to do these programming challenges:

Send us your information

AFTER doing the programming tests, fill this web form with your information and challenge results. Then you are invited to ask the project mentors about the project details. Maybe we will require more information from you like this:

  1. Contact details
    • Name and surname:
    • Country:
    • Email:
    • Public repository/ies:
    • Personal blog (optional):
    • Twitter/Identica/LinkedIn/others:
  2. Timeline
    • Now split your project idea in smaller tasks. Quantify the time you think each task needs. Finally, draw a tentative project plan (timeline) including the dates covering all period of GSoC. Don’t forget to include also the days in which you don’t plan to code, because of exams, holidays etc.
    • Do you understand this is a serious commitment, equivalent to a full-time paid summer internship or summer job?
    • Do you have any known time conflicts during the official coding period?
  3. Studies
    • What is your School and degree?
    • Would your application contribute to your ongoing studies/degree? If so, how?
  4. Programming background
    • Computing experience: operating systems you use on a daily basis, known programming languages, hardware, etc.
    • Robot or Computer Vision programming experience:
    • Other software programming:
  5. GSoC participation
    • Have you participated to GSoC before?
    • How many times, which year, which project?
    • Have you applied but were not selected? When?
    • Have you submitted/will you submit another proposal for GSoC 2017 to a different org?

Previous GSoC students

How to increase your chances of being selected in GSoC-2021

If you put yourself in the shoes of the mentor that should select the student, you’ll immediately realize that there are some behaviors that are usually rewarded. Here’s some examples.

  1. Be proactive: Mentors are more likely to select students that openly discuss the existing ideas and / or propose their own. It is a bad idea to just submit your idea only in the Google web site without discussing it, because it won’t be noticed.

  2. Demonstrate your skills: Consider that mentors are being contacted by several students that apply for the same project. A way to show that you are the best candidate, is to demonstrate that you are familiar with the software and you can code. How? Browse the bug tracker (issues in github of JdeRobot project), fix some bugs and propose your patch submitting your PullRequest, and/or ask mentors to challenge you! Moreover, bug fixes are a great way to get familiar with the code.

  3. Demonstrate your intention to stay: Students that are likely to disappear after GSoC are less likely to be selected. This is because there is no point in developing something that won’t be maintained. And moreover, one scope of GSoC is to bring new developers to the community.

RTFM

Read the relevant information about GSoC in the wiki / web pages before asking. Most FAQs have been answered already!