In recent years, the visual object tracking has drawn increasing interests. There are many applications, e.g., video surveillance in airports, schools, hospitals and traffic. The object surveillance may provide crucial information about the behavior, interaction, and relationship between objects of interest. This paper addresses issues in object tracking where videos contain complex scenarios. We propose an adaptive particle filters tracking scheme with exquisite resampling (AERPF), which improves prediction, importance sampling and resampling. In prediction step, an adaptive strategy for search region and particle number is addressed for object disappearing or obstacle disturbance, which can obtain results more effectively. In addition, in importance sampling, we use optical flow to refine the particle weights using the dynamical object motion information, which results the better accuracy of object location updating. Moreover, exquisite resampling (ER) algorithm can be applied for reflecting more the posterior probability density function of true state. The proposed method can be applied for object tracking both on fixed and active camera, handling partial occlusion and full occlusion problem properly. As a result, it outperforms other existing methods.