Several years ago, I was new to my specific group at work. I was working on a combination of image processing and a GUI for keeping track of specific objects in the camera’s view. I started on the GUI first, writing a server that received object locations and handed them off to the GUI which displayed them in a nifty way. I was using Qt and QML, which I have used in numerous personal projects, so I was pretty familiar and confident with what I was doing. In order to test/develop the GUI I threw a quick C++ program together to create red and blue OpenCV images and fire them over the network.

Right as I got close to getting the GUI far enough along that I was ready for real images, my supervisor came in and said that we had our project sponsors coming by to see how we were getting along. He wanted to show them my GUI work, and wanted it working with detecting faces in a live camera feed. Oh, and they were coming day after tomorrow.

What? I had never really done much image processing before (other than one class in college), and had NEVER used OpenCV before (other than generating red and blue images). Now I needed to throw something together in a day?

So I started doing some research. It turned out that Haar-cascade-based face detection was pretty easy in OpenCV, but it was too slow to do every frame in my case. So I went with initially detecting faces via Haar cascades, and then handed it off to a color tracker (based on Camshift). It wasn’t great, but it was acceptable for something I threw together in half a day. I spent the other half a day tying it together with the GUI.

With the Camshift-based color tracker, one needed to calibrate it for the color you want to track (and toss out unrelated colors). So the day of the demo, I set my stuff up where I wanted and played with the calibration until it detected my Italian face quite nicely. I saved those parameters so they would be used every time the detector/tracker software was run. Multiple people came by throughout the demo day– the setup was a hit! The initial face detection worked, and then people had fun trying to ditch the color tracker by trying to get it to track their hand, etc.

Then my supervisor came in with the sponsor in tow. He was black.

He stood in front of the camera with the other people playing with it, and it wouldn’t track him. He tried getting close: no cigar. Tilting his head… nada. I exchanged awkward glances with my coworkers and tapped the camera like some idiot. “Huh… looks like it’s not working anymore.” I quickly quit the program so he wouldn’t notice it tracking other people on my screen. I had no idea what to say. I had made a racist face detector.

He finally stood up and said “Huh… well cool,” and left. We’re still funded. Now that I know more, I would have used something like OpenTLD.