Syed Fawaz Ahmed | syedfawaz2002@gmail.com

Teaching Robots with Gestures

Guide Manoj Neelakanthan
Sponsor Self Sponsered
 
Keywords automation;human gestures;throughput;TPU (Teach Pendant Unit)

Automation plays an important role in increasing productivity and maintaining quality in the industrial environment. Industrial robots enable scalability of throughput and ensure strict quality which is very tough to attain through manual labour.


Large automobile companies outsource the production of smaller parts to a few small and medium sized industries. Such industries invest in industrial robots for the manufacturing and finishing of auto parts. While commissioning the robots, ABB programs and configures the robot for a particular job. Any changes in the job or work piece are reverted to ABB. This project explored direct manipulation techniques to configure the robot; instead, of joysticks or push-buttons, human gestures were used to communicate with the robot.


This project comprised two main parts—configuring the robot; defining the paint paths. Microsoft Kinect camera was used to track different points on a body. The movement of hand joints was mapped on the axes of the robot. The angle of movement of a particular hand-joint is signaled to the robot and the robot responds to this. In this way, the robot could be configured to a large extent and the fine tuning could be done by a TPU (Teach Pendant Unit). Motion tracking was used to capture the paint path taken by an expert painter. The 3D points which constituted the path were fed to the robot to replicate the expert painter’s actions. In this way, paint-efficient and energy-efficient paths could be made.


This project explored how machine learning could happen through natural user interfaces such as human gestures. It aimed to bring about awareness and envision the gesture-based interaction in a control room environment. The project included within its ambit, a study on gaps, approaches, challenges, and trends in gesture-based interactions for users in industries.
Communication Design
ED (6)
AFD (22)
GD (28)
FVC (13)
IID (14)
PHD (6)
I.T. Integrated Design
NMD (12)
DDE (11)
Interdisciplinary Design
SDM (11)
DRE (15)
Industrial Design
CGD (7)
FID (13)
TAD (10)
TGD (8)
PD (28)
Textile Design
ADM (10)
LAD (13)
TD (23)