I wanted to create a program which can detect the fingers on a photo. So I made one. This was my homework for my Industrial image processing class in University.

This was the rock,paper,scissors problem because my Matlab app can find the fingers on the image. So it’s just a playful name for an interesting problem.

I used morphology for the segmentation of the fingers. I know that there are much better methods but I wanted to try out this interesting way.Maybe I’ll create another one with a different method.

Now, I’ll show you the basic process:

  • First of all we need to detect the skin on the image. The easiest way was to convert the original image (RGB) to a new color space called YCbCr. And I found a gorgeous algorithm to detect the skin pixels by Chai and Ngan. I marked the skin pixels with red color.
    skin_marked
  • Than we just create a binary picture what is represented with zeros and ones in a (n x m) matrix. The zeros are shown as black and the ones are the white on a picture. (This is why it’s called a binary image). So the skin pixels are the ones and the rest is just zeros. Than we need to fill in holes and “clear” the picture (get rid of the little white parts on the image).
    binary_image
  • And now….The morphology! (my favourite part):
    We have the binary image with the marked skin pixels. But we can see the whole hand and not just the fingers. What should we do?
    Vanish the palm and pay attention only for the fingers. With morphology it’s child’s play. With a simple (square) structuring element we use erode to delete the fingers. This is possible because the smaller parts of the image will shrink more faster than the bigger parts so the fingers disappear and we will have the palm only.Than we can restore the original size of the palm with dilatation.
    restore_palm
  • With a simple subtraction we have the fingers. Because if the subtract the original image (with the fingers) with the matrix we’ve just created we’ll get the fingers only. And we can count the coherent areas on the image so we can get the number of the fingers.
    scissors
    rock_and_paper

And we are done. 🙂

Of course you can get the source code from here: Rock,Paper,Scissors problem

Unfortunately detecting fingers on a picture with this method is a little bit slow. But it’s perfect for this kind of usage.  I measure the run time. The slowest part was when we converted from RGB to YCbCr. and I got an average 0.78 sec.

With good light conditions the detection rate was more than 75%.

So we can say that the results are not so bad and it is an interesting way to detect the fingers on a picture. 🙂

Stay tuned for my next post! 😉

Advertisements