Tuesday, January 4, 2011

Hand Gesture Recognition & My time at ACM-IITM Allahabad 2010

30 dec 2010 , It was a day of surprise when I came to know the result of IITM Student project contest.It was my first appearence at any of ACM conference however I learned a lot many stuff their.Future and current technologies , brilliant lectures by Prof. James Bown , Prof. David Benyon and many others.

This blog is a mix of my experience at IITM and how did I do it(Hand gesture recognition).It was a great opportunity for me to meet the people from all around the world who are working in a society with all the knowledge that they have for the benefit of the society.Luckily , I got a chance to meet Prof. James and Prof. Michael Herczeg.They both have given their expertise suggestion which will play very crucial role in furthur development in this field.Project that I've submitted at IITM is "Controlling Virtual Object through Hand Gesture (Click to watch it work.)" and the hard part of it is to identify the gestures.However it took 3 months of work to successfully recognize 7 different gesture and 1 extra month to improve it's accuracy which is working with an accuracy of about 90% (Willing to work on it to make it 100% accurate).

How Am I Doing It

Image processing is a very challanging field to work in.To identify the hand we need to recognize the features of the hand.
Basic ideas behind gesture recognition :-

  • Background Substraction
  • Contour Detection
  • Finding Convex hull
  • Calculating Defects

Some of the functions that will help in the above operations are cvAbsDiff , cvCvtColor , cvFindContours , cvContourConvexHull , cvConvexityDefects.I've worked on OpenCV2.1 on DevCpp however one can use any of the OpenCV version if they support the above functions.
cvContourConvexHull :- All the above functions will be easy to understand and work with the problem that one may face is to work with "cvContourConvexHull".It's arguments are

  • CvContour:- Obtained from cvFindContour
  • Storage :- CvMemStorage*

It returns a CvSeq* from which CvConvexityDefects can be obtained by a function CV_GET_SEQ_ELEM and typecasting it to (CVConvexityDefects*).

This blog is more about CvContourConvexHull which is the final step of the operation on frame captured from camera.Other things are very easy to understand from the willowgarage wiki.Hope you have enjoyed this blog.Currently I'm working on developing a library that will provide very basic functions to hand gesture recognition that will read and inform about the gesture directly and other things will be vary easy to do.Soon , I'll be back with new ideas and information.In case of any query feel free to contact & forget not to subscribe.


  1. @Santak ... Ya you got the right point. It was expected to be a lil slow.But ya finally it gave a quite an appriciable result. Tryin to make it more robust and accurate.

  2. After extraction of convexity defects how did you proceed with gesture recognition .were you able to execute recognition in real time

    1. Once you have extracted the convexity defect , you can have state representation of configuration of your hand. This configuration of hand is stored in the a list of variable ( Orientation , number of defects , Depth of defects etc ). After you have the configuration you can map it to some gesture.
      eg :- left orientation and 5 defects => gesture1 etc.