Facial recognition using an NXT and an iPhone


This is a robot that uses Face Recognition  in order to follow around a human. It uses an iPhone in conjunction with an NXT. Take a look!

You can download the Xcode Project and ROBOTC code here:

How it works

The iOS code uses iOS 5’s face detection algorithm to find the position of the face within the video frame.  I then needed a way to communicate with the NXT robot and steer it.  Since I didn’t want to go through the trouble of communicating through bluetooth with it (and I don’t know how to do it!), I chose to communicate with the NXT using the Light Sensor that comes with the NXT.

If I want the robot to go to the left, I dim the lower portion of the iPhone screen and if I want it to go to the right I increase its intensity.  Also, when the phone does not see a face, I turn the lower portion of the screen black.  This tells the robot that it needs to not move forward and spin in-place until it finds a face.

In the ROBOTC code, I also make use of the sound sensor to start and stop the robot.  A loud sound is used to toggle between start and stop.

The ROBOTC and iOS code is very simple.


(Code subject to change. Download the latest version of the code!)

#pragma config(Sensor, S1,     lightSensor,         sensorLightInactive)
#pragma config(Sensor, S2,     soundSensor,         sensorSoundDB)
#pragma config(Motor,  motorA,          mA,            tmotorNormal, PIDControl, encoder)

task main()
wait1Msec(50);                         // The program waits 50 milliseconds to initialize the light sensor.
float x;
while (1)
x = SensorValue[lightSensor];

float minLight, maxLight, d, a, c, v, alpha = 0.01, stopGo=0.0;
int l, sound, startMotors = 0, lostFace, faceFound = 0;

a = 0.60;
minLight = 9;
maxLight = 34;
lostFace = 5;

c = (minLight+maxLight)/2.0;

while (1) {

sound = SensorValue[soundSensor];
if(sound > 85) {
startMotors %= 2;

l = SensorValue[lightSensor];
d = a*(l-c);

faceFound = (l > lostFace) ? 1:0;

stopGo = alpha*faceFound + (1-alpha)*stopGo;

motor[motorB] = (-d+v*stopGo)*startMotors;
motor[motorC] = (d+v*stopGo)*startMotors;

Written by ramin

January 9th, 2012 at 8:58 am

Posted in Cool projects,NXT