Human activity recognition is a popular topic in the field of computer vision. While most analysis algorithms take into consideration of the whole human body, the movements of merely the head and limbs are often informative enough in many practical applications. In this paper, a novel approach is proposed to track these extruding parts of a human body in consecutive images. Accordingly, a simplified torso-less pattern of gesture is proposed to represent human activities, and with its effectiveness verified subjectively by some experimental results. Such a representation can not only ensure the privacy of the person being watched, but is also suitable for real-time surveillance based on bandwidth-limited communication since only a very small amount of data are used compared to conventional approaches.