In this paper, a pose-variant face recognition system is presented for study of human-robot interaction design. An iterative fitting algorithm is proposed to extract feature-point positions based on active appearance model (AAM). Comparing with the traditional Lucas-Kanade algorithm, the proposed iterative algorithm improves the capability of correct convergence as a larger variation of head posture occurs. After obtaining the location of feature points, the dimension of texture model is reduced and then sent to a back propagation neural network (BPNN) to recognize family-members. The proposed pose-variant face recognition system has been implemented on an embedded image processing system and integrated with a pet robot. Experimental results show that the robot can interact with a person in a responding manner. Tested with the database from UMIST and the database built in the lab, the proposed method achieved average recognition rates of 91.0% and 95.6% respectively.