This paper presents a vision-based landing system concept which could be implemented in a small UAV (Unmanned Aerial Vehicle). Vision-based object detection provides object position information of objects such as pedestrian or road, and those detection methods can also provide precise position during the landing stage of aircraft. Besides, the take off weight of a small UAV should be light in order to increase endurance. Vision-based sensor is much low-cost compared with other sensors such as RADAR (Radio Detection And Ranging), LiDAR (Light Detection And Ranging) or DGPS (differential GPS) module. However, vision-based object detection methods have several challenges such as weather conditions, low illumination capability, and high false detection rate in complicated environment. Accordingly, this paper presents a practical method that can conquers the above challenges for vision-based landing system for UAV. In addition to desktop simulation, we also realize the proposed method on a portable device.