Vision-based human robot interaction
We propose a hand posture recognition system consists two phases that are hand detection and hand posture recognition.
In hand detection step, we employed Viola-Jones detector with proposed concept Internal Haar-like feature. The proposed hand detection works in real-time within frames captured from real complex environments and avoids unexpected effects of background. The proposed detector outperforms original Viola-Jones detector using traditional Haar-like feature.
In hand posture recognition step, we proposed a new hand representation based on a good generic descriptor that is kernel descriptor (KDES). When applying KDES into hand posture recognition, we proposed three improvements to make it more robust that are adaptive patch, normalization of gradient orientation in patches, and hand pyramid structure. The improvements make KDES invariant to scale change, patch-level feature invariant to rotation, and final hand representation suitable to hand structure. Based on these improvements, the proposed method obtains better results than original KDES and a state of the art method.

Demo of human hand posture recognition for service robot in library

Keywords Hand posture recognition, Visual based Human-machine interaction, Hand detection, Internal Haar-like feature, AdaBoost, Cascade of classifiers, Kernel descriptor
Download:
- Dataset of 21 hand postures.
Please send me an email.
My colleagues and my students working on this topics:
- PhDs Van-Toi Nguyen (the work presented here is his thesis work)
- Dr. Thanh-Hai Tran
Main publications
Van-Toi Nguyen, Thi-Lan Le, Thanh-Hai Tran, Rémy Mullot, Vincent Courboulay, Eric Castelli, Visual interpretation of hand gesture for human-machine interaction[pdf], ACPR, Doctoral consortium, 2015
NGUYEN Van Toi, TRAN Thi Thanh Hai, LE Thi Lan, MULLOT Remy, COURBOULAY Vincent, Using hand postures for interacting with assistant robot in library [pdf], the 1st International Workshop on Pattern Recognition for Multimedia Content Analysis (PR4MCA 2015) In conjunction with the 7th International Conference on Knowledge and System Engineering, Ho Chi Minh City, Vietnam - 2015, Best paper award of the workshop
Van-Toi Nguyen, Thi-Lan Le, Thanh-Hai Tran, Remy Mullot, Vincent Courboulay, A New Hand Representation Based on Kernels for Hand Posture Recognition, The Eleventh IEEE International Conference on Automatic Face and Gesture Recognition (FG 2015), Slovenia, 2015
Van-Toi Nguyen, Thi-Lan Le, Thanh-Hai Tran, Rémy Mullot, Vincent Courboulay, Hand posture recognition using Kernel Descriptor, [pdf], iHCI, France, 2014
Home page