Time-based human hand movement recognition technique

The success of the project "Research and development of real-time human hand movement recognition techniques" by a young research group from the Faculty of Information Technology, Military Technical Academy, led by Dr. Tran Nguyen Ngoc, has opened up many effective application directions in practice.

The effectiveness of the application of the topic such as developing a system to support information lookup and remote device control, controlling robots to assist people with disabilities, integrating TV and music player controls...

Research direction with high applicability

According to Dr. Tran Nguyen Ngoc, the research direction of recognizing human hand movements in real time has been of interest to the computer vision research group at the Military Technical Academy since 2009.

From previous studies on image processing and artificial intelligence techniques such as facial recognition, age prediction, gender discrimination through image information, automatic pedestrian detection through the use of surveillance cameras to research and development requiring high synchronization and systematic capabilities such as license plate recognition systems, election counting systems using image processing technology, etc., members of the research team have accumulated a lot of experience to solve real-time application development problems.

By early 2010, major technology companies in the world such as Intel, Omron, Sony, Samsung, Microsoft... began to launch experimental products on the market that allow interaction with devices using natural human movements.

Also at this time, in the laboratories of research centers around the world such as Stanford University - USA, Hebrew University - Israel, MIT Institute - USA, Moscow State University - Russia as well as domestic research groups such as the Institute of Information Technology - Vietnam Academy of Science and Technology, Hanoi University of Science and Technology... there are many scientific publications related to the exploitation of modern sensor devices to solve the problem of natural interaction using gestures, voice, thoughts to control devices.

Realizing that this is a research direction with high applicability and suitable for their capacity, the young research group of the Faculty of Information Technology, Military Technical Academy registered to carry out this topic. The topic has identified the research goal of developing algorithms to detect and identify objects in images in real time, applied in human-machine interaction through human hand movements, thereby creating a research group specializing in computer vision and human-machine interaction," said Dr. Tran Nguyen Ngoc.

Opens up many possibilities

After a year of passionate research, the group has built an algorithm that allows detecting hand regions with a reliability of 94%; proposed an algorithm to extract 39 hand region features and recognize 36 gesture combinations of two hands with an average accuracy of 88.54% in different lighting conditions. The group has built an algorithm to recognize 8 groups of movements when the hands move according to conventional trajectories in space with an accuracy of 89.85%.

In addition, the team has built computer software that receives input signals from Microsoft's Kinect sensor, allowing computer control operations to be performed through the movements of two hands without touching the keyboard, mouse, screen or any other device; recognition speed reaches 0.4 seconds/movement combination (from a series of images of about 12 consecutive frames)...

“The project has proposed a recognition algorithm that meets the requirements for real-time processing and has an accuracy equivalent to published research results in the world (approximately 90%),” Dr. Tran Nguyen Ngoc excitedly shared.

Dr. Tran Nguyen Ngoc also said that the results of this research can continue to develop many practical applications. Therefore, in the near future, the group will complete the system to support information lookup and remote device control towards the application of looking up patient records (when doctors are not allowed to touch the control applications), controlling robots to assist the disabled, integrating TV and music player controls, etc. The group will continue to develop the topic in the direction of building a virtual reality interactive system, allowing the creation of support solutions in advertising and entertainment technology.

Dr. Ngoc also said: “Currently, the group is inviting more researchers in the field of embedded software to integrate algorithms on microchips or mobile devices such as smartphones and tablets. At that time, the research results can be integrated into other technology products such as televisions, projectors, autonomous robots…”./.

According to VNA