Gesture KINECT capture system for virtual surgical robots manipulation

  • Juan Diego Hurtado-Chaves Ing. en Automática Industrial. Universidad del Cauca. Popayán
  • Alex Aldemar Nástar-Guacales Ing. en Automática Industrial. Universidad del Cauca. Popayán
  • Oscar Andrés Vivas-Albán Ph. D. en Robótica, Universidad del Cauca. Popayán
Keywords: Kinect device, natural interfaces, surgical robots, surgical simulator


This article  presents  a  gesture  capture  system  for manipulating two virtual surgical robots. Gesture capture uses Kinect  device  to  detect  the  movement  of  the  user ́s  hand  to move  the  surgical  robots,  and  his  right  knee  to  change  the needed  surgical  tools.  This  natural interface  capture  system is  proved  on  a  surgical  simulator  for  laparoscopic,  that  has  a holder endoscopic robot manipulated by a joystick, and two surgical  robots  that  are  guided  by  the  Kinect  device.  Kinematics model is used to transform Kinect captured signals to articular movements for the robots. System is tested on a robotic cholecystectomy, allowing a more natural manipulation to the user.


Download data is not yet available.


[1] D.B. Jones, Laparoscopic Surgery: Principles and Procedures. New York: Marcel Dekker, 2004.

[2] N. Katkhouda, Advanced Laparoscopic Surgery: Techniques and Tips. New York: Springer, 2010.

[3] J. Marescaux, and F. Rubino, “The Zeus robotic system: experimental and clinical applications,” Surgical Clinics of North America, vol. 83, pp. 1305-1345, 2003.

[4] G.H. Ballantyne, and F. Molle, “The Da Vinci telerobotic surgical system: the virtual operative field and telepresence surgery,” Surgical Clinics of North America, vol. 83, pp. 1293-1304, 2003.

[5] C. Vaessen, “Location of robotic surgical systems worldwide and in France,” Journal of Visceral Surgery, vol. 148, pp. 9-11, 2011.

[6] A. Liu, F. Tendick, K. Cleary, and C. Kaufmann, “A survey of surgical simulation: applications, Technology, and education,” Presence, vol. 12, no. 6, pp. 599-614, 2003.

[7] H. Delingette and N. Ayache, “Hepatic Surgery Simulation,” Communications of the ACM, vol. 48, pp. 31-36, no. 2, 2005.

[8] J. García, M. Arias y E. Valencia, “Diseño de prototipo de simulador para entrenamiento en cirugía laparoscópica,” Revista Ingeniería Biomédica, vol. 5, no. 9, pp. 13-19, 2011.

[9] R. Valero, Y.H. Ko, S. Chauhan, O. Schatloff, A. Sivaraman, R.F. Coelho, F. Ortega, K.J. Palmer, R. SanchezSalas, H. Davila, X. Cathelineau, and V.R. Patel, “Cirugía robótica: Historia e impacto en la enseñanza,” Actas Urológicas Españolas, vol. 35, no. 9, pp. 540-545, 2011.

[10] C. Lallas and J. Davis, “Robotic surgery training with commercially available simulation systems in 2011: a current review and practice pattern survey from the society of urologic robotic surgeons”, Journal of Endourology, vol. 26, no. 3, pp. 283-293. 2012.

[11] S. Salinas y A. Vivas, “Modelado, simulación y control del robot para cirugía laparoscópica Lapbot,” Revista Chilena de Ingeniería, vol. 17, pp. 317-328, 2009.

[12] V. Torres, C. Méndez, A. Vivas y S. Salinas, “Diseño y simulación en 3D de un robot porta endoscopio para operaciones de laparoscopia,” en V Seminario Internacional de Ingeniería Electrónica, Bucaramanga, Colombia, 2011.

[13] D. Guzmán, “Herramienta software para la práctica y experimentación de la robótica quirúrgica,” Tesis Ingeniería en Automática Industrial, Universidad del Cauca, Popayán, Colombia, 2013.

[14] N. Petersen, “Continuous natural user interface: Reducing the gap between real and digital world,” presented at 8th IEEE International Symposium of Mixed and Augmented Reality, Orlando, FL, pp. 23-26, 2009.

[15] Y, Li, “Hand gesture recognition using Kinect,” presented at 3rd International Conference on Software Engineering and Software Science, Beijing, China, 2012.

[16] S. Lang, “Sign language Recognition whit Kinect,” Tesis de licenciatura, Freie Univesität Berlin, Berlin, Germany, 2011.

[17] K. Jahrmann, “3D Reconstruction with the Kinect-Camera,” Tesis de licenciatura, Technischen Universität Wien, Wien, Austria, 2013.

[18] E. Trilles, “Desarrollo de interfaces de usuarios naturales con Kinect,” Tesis Ingeniería Informática, Universidad Politécnica de Valencia, Valencia, España, 2012.

[19] S. Pfeiffer, “Guiado gestual de un robot humanoide mediante un sensor Kinect,” Tesis Ingeniería Técnica en Informática de Sistemas, Universidad Politécnica de Cataluña, Barcelona, España, 2011.

[20] G. Du and P. Zhang, “Markerless human–robot interface for dual robot manipulators using Kinect sensor,” Robotics and Computer-Integrated Manufacturing, vol. 30, no. 2, pp. 150-159, 2014.

[21] Y. Chang, B. Lange, M. Zhang, S. Koenig, P. Requejo, N. Somboon, A. Sawchuk and A. Rizzo, “Towards pervasive physical rehabilitation using Microsoft Kinect,” presented at Pervasive Computing Technologies for Healthcare (PervasiveHealth), San Diego, CA, 2012.

[22] Y. Chang, W. Han and Y. Tsai, “A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy,” Research in Developmental Disabilities, vol. 34, no. 11, pp. 3654-3659, 2013.

[23] R. Wen, W. Tay, B. Nguyen, C. Chng and C. Chui, “Hand gesture guided robot-assisted surgery based on a direct augmented reality interface,” Computer Methods and Programs in Biomedicine, vol. 116, no. 2, pp. 68-80, 2014.

[24] S. Dargar, A. Nunno and G. Sankaranarayanan, “Microsoft Kinect based head tracking for Life Size Collaborative Surgical Simulation Environments (LS-CollaSSLE),” Studies in Health Technology and Informatics, vol. 184, pp. 109-113, 2013.

[25] M. Jacob and J. Wachs, “Context-based hand gesture recognition for the operating room,” Pattern Recognition Letters, vol. 35, pp. 196-203, 2014.

[26] W. Souba, F. Mitchell, G. Jurkovich, L. Kaiser, W. Pearce, J. Pemberton, and N. Soper, ACS Surgery: Principles and Practice, 6th edition. Hamilton: Decker Publishing, 2007.

[27] D. Litwin, and M. Cahan, “Laparoscopic cholecystectomy,” Surgical Clinics of North America, vol. 88, pp. 1295-1313, 2008.
How to Cite
Hurtado-Chaves, J., Nástar-Guacales, A., & Vivas-Albán, O. (2015). Gesture KINECT capture system for virtual surgical robots manipulation. ITECKNE, 12(1), 17-24.
Research and Innovation Articles