This paper presents a novel algorithm of image-based motion detection for lateral driving assistance. We propose a feature-based motion detection algorithm, which combines optical-flow estimation and road plane segmentation using FOE concept. The developed algorithm can segment moving objects from a changing background efficiently. Both cars and motorcycles on the lateral side of a driving vehicle can be detected. We have realized the system in a stand-alone image processing platform, which can operate independently on a vehicle using a CMOS image sensor and embedded processor. Experimental results show that the proposed methods are effective to segment moving vehicles from a moving background for lateral safety assistance.