It started about 5/4 weeks before the Maker Faire, in a pub. Where the small group of members of the Brighton Pi sat round a table and thought what can we do for the Maker Faire. I had spoken about using some sort of computer vision on my autonomous RC car project sometime ago but didn't do anything more with it. So the others decided that I would use my Raspberry Pi to demo some computer vision, face detection, object and edge detection. "Ok" I said, "I should be able to put something together"
My Pi was still in the anti-static bag, with no other items to enable it's use in this project. So quickly ordered a box from pibow and quick look on Amazon and the Elinux.org SD card section. Picked the Integral 16GB SDHC Class 10 Ultima pro 20 Mb/s. Also ordered a Flexible keyboard from HK only a £5 and powered USB Hub (which I needed to modify). I had a Logitech C250 webcam, also an USB mouse.
While I was waiting for the delivery of the other items, also have some items coming from Hong Kong (fingers crossed they arrive in time). I used my Ubuntu (Linux) computer to enable me to start working on the code and see if this at all is possible to do this. Plugged the webcam into the computer, showed it was connected, loaded a program called Cheese (a webcam photo booth) to test the webcam. All working fine.
Cheese is available as a Debian package using the apt-get command in a terminal. As it was also going to run on a Raspberry and I knew a little Python, so that would be the language used to talk to Open CV. Open CV is written in C++ and available from http://opencv.org/
OpenCV is released under a BSD license and hence it’s free for both academic and commercial use.
It has C++, C, Python and Java interfaces and supports Windows, Linux, Android and Mac OS.
The image used for this project was the Raspbian available from http://www.raspberrypi.org/downloads which was a gamble as there was no support for the webcam I was using by default. The Arch Linux image did have the support required but due to my 250mA flexible keyboard I was unable to boot into that OS. It's too late to get another keyboard. Anyway I liked the keyboard as it was easy to transport so I stuck with the Raspbian image, just need to recompile the Linux Kernel with the required UVC driver. Found some details here (Linux UVC driver & tools) http://www.ideasonboard.org/uvc/#download I had started tweeting about my project and the kernel gods answered by adding USB webcam support to the Raspbian kernel, which I somehow picked up by doing a lot of reboots and apt-get update, apt-get upgrade, Ri-update commands. Also following a USB cable mod (cut the red wire) to stop the powered hub powering the Raspi the webcam was detected by my raspberry. Whoo hoo major hurdle overcome. The code was being worked on my Ubuntu box, which showed Grey-scale, Face Detection, Edge Detection, Gaussian Blur. These were running in loop which ran about 4.5 FPS. Now the Ubuntu box is 2GHZ CPU with 512Mb Ram so it going to be a lot slower on the Pi which runs at 700Mhz with 256Mb ram.
Moved the code over to the raspberry and also installed Genny (lightweight text editor) able to help with python indentation. First I installed OpenCV from the site above but found that the python bindings didn't work so I use the package python-opencv available by using the command sudo apt-get install python-opencv This enabled my python scripts to import cv gaining access to the Image processing commands within OpenCV.
Another major problem I overcame was that raspberry pi couldn't cope with taking the feed directly from the cam though the USB at 640x480 so I used a program called fswebcam which enable me to take frames from the cam, perform some basic image manipulation, save to a Jpeg file and reduce in size. This file was then placed upon a USB stick to save wear and tear on a SD card, because I was polling the cam every 10 secs with a 1sec delay upon capture saving the image. The python script was then loading/reading the image from the USB and processing the same image with a loop, which ran at about 0.5 fps. The webcam program was running at 7 fps. This caused a small delay in the update of the 4 images shown but did a good job with edge and face detection. As you can see in the pic below, Pi's Cpu was running at 90-100% and using about 25% of the ram available.
Maybe with a directly connected camera . So by passing the USB bottleneck and access to the GPU chip it could have been quicker. Also it wouldn't be common practice to run each image function Face and edge detection using Haar Cascade and Canny within a loop but it did show that the Raspberry Pi could be used in image processing applications.
Some late night debugging was done the Thursday night before the Maker Faire at the Build Brighton Hack-Space Set-up was done Friday night with a bit more debugging.
Thanks to the members of Build Brighton, Brighton Pi, Brighton Mini Maker Faire also a blog post by Fajita and various other blog posts which enabled this project to presented at Brighton Mini Maker Faire 2012
Result below:
