Introduction

Per the lab assignment handout, the system is supposed to deliver the following verbal output (both spoken and text) upon recognizing the respective objects: "You are wearing a horizontally striped shirt today" or "You're wearing a vertically striped shirt today" or "I see something red" or "This is blue like the sky".

In this implementation, I detect the 6 HSV colors: Red, Yellow, Green, Cyan, Blue, and Magenta. A separate sound was generated when each color is detected.

Project Details

Color detection was done in the HSV color space and not in the RGB color space.

The program was written in the Mathematica language.

The program must be run from inside Mathematica since Mathematica does not allow one to compile the program into an executable for running outside it.

The following is an outline of the algorithm.

The program starts by reading the image file. It then extracts the RGB channels, then convert these to HSV channels.

It then scans the image, and for each pixel, it performs the following check to determine the color of the pixel:

IF pixel hue value is within a certain degree range (depends on the color. I used the HSV color wheel)

AND the pixel Lightness Value > $70\%$ AND pixel saturation > $60\%$ THEN

count this pixel as of the color Hue

END IF

In the above, I only check for 6 different angle ranges which represents the 6 main HSV colors mentioned above. The above means that not all shades of colors will be detected.

The program performs many types of statistics on the image. I generates the Histograms and the PDF and the CDF (cumulative histogram) for each channels.

Initially I generated this data to be able to use it to detect the colors, but I found that using HSV results in a more robust detection of colors. I have tried many different images and found that detection using HSV as compared to using just RGB histogram results in more accurate detection of colors including more different colors and not just for RGB colors.

In the appendix I show a complete output of one such run in Mathematica.

The program comes with all the needed sound wav files to use for each color detected. The wav files are loaded at the start of the program, the then the correct wave file is played at the end. The sound was clear and can be easily recognized.

Results

Here I show a short outline of one run using one of the reference images (the red tomato image)

This shows the user prompt to enter the image file name

This below shows the start of the run.

Appendix

For the HSV color wheel, I used this reference: http://semmix.pl/color/models/hsvkolo.bmp

In addition, I used Adobe Photoshop student version to help with this work even though I was not able to use it for learning the HSV model well. But I used it to generate images with pure colors to test with. This was useful.