Project - Making Petoi Bittle respond to colors

How to program a robot to respond to color changes

Introduction

We have been writing a series of posts which explore the possibilities of the Petoi Camera module which is based on the MU3 Vision Sensor. In a previous post, we saw how the Petoi Bittle robot can be made to track a ball and move its posture as the ball moves. We have also discussed how to view a continuous video stream from the camera module.

Objective

The objective of this post is to program Bittle to respond to a specific color. To recognize a color, we need to program Bittle to identify a color seen by the MU3 Vision Sensor, and then respond to it by performing a gait. By the end of this post, you will learn how Bittle can respond to different colors.

Color Recognition

The details of how the MU3 Vision Sensor detects colors is described in Section 4.3 in the Advanced Technical Manual. Programmatically, one needs to specify the position (X-Y) and size (W-H) of the recognition area. The X-Y coordinates denote the center of the recognition area within the image. The W-H coordinates denote the width and height of the recognition area. The camera sensor will collect color information of every pixel in the specific area, and then classify into 8 different colors. The way the sensor classifies color depending on R/G/B values is described in Section 4.3.1 of the Advanced Technical Manual.

---

Editor’s note: For this holiday season, consider gifting yourself or a loved one the power of knowledge. You can become a paying subscriber, gift a subscription, or donate a subscription via the following buttons.

---

Programming the Camera Module

Specifically for Bittle, the code needs to be written in camera.h. The following snippet of code demonstrates how to program the MU3 Vision Sensor for color recognition.

      (*Mu).write(VISION_COLOR_RECOGNITION, kXValue, 50);                  // set detect region center x value(0~100)
       (*Mu).write(VISION_COLOR_RECOGNITION, kYValue, 50);                  // set detect region center y value(0~100)
       (*Mu).write(VISION_COLOR_RECOGNITION, kWidthValue, 20);               // set detect region center width value(0~100)
       (*Mu).write(VISION_COLOR_RECOGNITION, kHeightValue, 20);              // set detect region center height value(0~100)
       (*Mu).CameraSetAwb(kLockWhiteBalance);                               // lock AWB

In the above code snipped, Mu is a reference pointer to the MuVisionSensor. We intend to recognize the color in a 20x20 region in the middle of the image seen by the camera.

The color detected by the camera can be retrieved using the following code:

int color = (*Mu).GetValue(VISION_COLOR_RECOGNITION, kLabel);

This color can be compared with pre-defined colors to know the color detected by MU3. Once a color is detected, an action may be performed. Here is sample code:

int color = (*Mu).GetValue(VISION_COLOR_RECOGNITION, kLabel);
if (color != lastColorType) { 
   switch (color) {
   case MU_COLOR_RED:
       PTLF("red");
       break;
   lastColorType = color;
}

In the above piece of code, we detect the color Red, and write to the serial console indicating that “red” is detected. The variable lastColorType saves the last color detected so that we don’t perform the action when the same color is detected consecutive times.

Performing an action

Now, we can put things together by performing an action instead of simply printing the color on the serial console. The code would look like:

int colorBehavior(int color) {
   if (color == MU_COLOR_RED) {
       PTLF("Balance");
       tQueue->addTask('k', "balance");
       return 1;
   } else {
       PTLF("Sit");
       tQueue->addTask('k', "sit");
       tQueue->addTask('i', "");
       return 1;
   } 
   return 0;
}

if (millis() - lastCommandTime > 5000) { // Issue the next color related command only if 5 seconds has elasped since the last color related command was issued
    if (colorBehavior(color) == 1) {
        lastCommandTime = millis();
    }
}

In colorBehavior(), we ask Bittle to balance if the color seen is Red, else we ask Bittle to sit. Note that colorBehavior() is issued only once every 5 seconds. This is required to prevent a burst of commands issued to Bittle in case there are frequent color changes.

Bittle recognizes color

Let us put everything in action. We compile the code in Arduino and then upload the new firmware to Bittle. We should first check the serial console and test if the sequence of operations is indeed performed when Bittle sees Red. Once, we test it to work, we can attach the battery and power ON Bittle. The following video shows how Bittle recognizes the color Red.

If you want to try the code, it is available for free in my git. Please let me know if you get a chance to try it. Also, please don’t forget if you tried this code, or tried to program Bittle’s camera in some alternate way.