Greetings, Fellow Programmers out there in the wild, today i will discuss a little project that i have been working on it’s a finger counter project through this project we will go through isolating the hand and then counting it’s raised fingers using OpenCV.
First things first, Let’s see the unfinished version in action
I have to say that OpenCV is a cool library to have and work with, it may seem hard to grasp but it’s a useful experience you’ll have.
Now to the project itself, well i was working with this fellow “Mahmoud Arafa“, we divided this little project into two pieces the first one is the Processor, the second is the Detector, now let’s get into the meat and potatoes.
The First Thought:
When we started this project we imagined the following, we will have a manager class this manager class will take care of getting input from the camera and supply the input and output between the two parts the processing and the detector, but also in order to provide the ability to do operations that may cost a lot we decided that the processor and the detector shall work in parallel, so the workflow of the application goes like this, the manager gets a frame from the camera at a constant rate, let’s say it’s 20 frame per second and pushes that stream into a queue that will be fed to the processor which will do it’s magic and then stores it’s output in a queue for the detector to perform the last step which is extracting the information from the image so, yeah it may seems naive but consider that this is the first time of us to deal with OpenCV
So, this chunk of code is all about managing the whole project and getting the image ready for the detector, so we had a “WindowManager” which task is to handle the output thing windows and stuff like that it’s truly a naive chunk of code but we will get through it anyway.
Now, the second one is the “TrafficOfficer“, and i will explain first why we needed that, as i mentioned above we needed the two main parts of the application to work in parallel so we had to find a way to manage the working memory and object between the two threads actually it’s three including the main thread but here we go,
As you can see it’s full of queues and map to send variables through threads, with a few functions to retrieve these objects from the queues or the map.
I guess i must mention the container class of our project, this class contains the data that will be going here and there between threads and functions, it’s the “ImgToken“, this class contains the main data that out processing will need, the three main images the original, binary and the contour image, the other objects are just vectors of contours and defects and finger points that will be filled later.
Now the Processor for now it’s just a function that takes the original image and produce a binary one and it does so through 3 steps
– Blurring the image to eliminate the small noise
– Threshold the values of the image to the colors of the hand and produce binary image
– Using Erode and Dilate functions to fill holes that may be produced
Now the detector part, there’s about 5 steps for counting the hand that i used in this app
– Find Contours of the Binary image and store it in a vector
– Getting the biggest contour on the screen which should be the hand you can use other method that’s smarter than this.
– Using OpenCV convexHull Algorithm to get the points of the convex of the biggest contour.
– Now as we have generated the convex hull we should get the convexity defects, and those are the points that make the shape of the contour not a convex which should be by this point the inner part of the hand that’s between the fingers
– The last thing is to process this convexity defects points and get the points of Finger
Now without further talking the detector code
and finally, here is the repository.
Happy Coding, bye.