This paper presents a novel design of a robust visual tracking control system, which consists of a visual tracking controller and a visual state estimator. This system facilitates human-robot interaction of a unicycle-modeled mobile robot equipped with a tilt camera. Based on a novel dual-Jacobian visual interaction model, a dynamic motion target can be tracked using a single visual tracking controller without target's 3D velocity information. The visual state estimator aims to estimate the optimal system state and target image velocity, which is used later by the visual tracking controller. To achieve this, a self-tuning Kalman filter is proposed to estimate interesting parameters online in real-time. Further, because the proposed method is fully working in image space, the computational complexity and the sensor/camera modeling errors can be reduced. Experimental results validate the effectiveness of the proposed method, in terms of tracking performance, system convergence, and robustness.