The capability of robotic emotion recognition is an important factor for human-robot interaction. In order to facilitate a robot to function in daily live environments, a emotion recognition system needs to accommodate itself to various persons. In this paper, an emotion recognition system that can adapt to new facial data is proposed. The main idea of the proposed learning algorithm is to adjust parameters of SVM hyperplane for learning emotional expressions of a new face. After mapping the input space to Gaussian-kernel space, support vector pursuit learning (SVPL) is applied to retrain the hyperplane in the new feature space. To expedite the retraining procedure, only samples classified incorrectly in previous iteration are combined with critical historical sets to restrain a new SVM classifier. After adjusting hyperplane parameters, the new classifier will recognize previous erroneous facial data. Experimental results show that the proposed system recognize new facial data with high correction rates after fast retraining the hyperplane. Moreover, the proposed method also keeps satisfactory recognition rate of old facial samples.