This paper presents an effective vehicle and motorcycle detection system in the blind spot area in the daytime and nighttime scenes. The proposed method identifies vehicle and motorcycle by detecting the shadow and the edge features in the daytime, and the vehicle and motorcycle could be detected through locating the headlights at nighttime. First, shadow segmentation is performed to briefly locate the position of the vehicle. Then, the vertical and horizontal edges are utilized to verify the existence of the vehicle. After that, tracking procedure is operated to track the same vehicle in the consecutive frames. Finally, the driving behavior is judged by the trajectory. Second, the lamps in the nighttime are extracted based on automatic histogram thresholding, and are verified by spatial and temporal features to against the reflection of the pavement. The proposed real-time vision-based Blind Spot Safety-Assistance System has implemented and evaluated on a TI DM6437 platformto performthe vehicle detection on real highway, expressways, and urban roadways, and works well on sunny, cloudy, and rainy conditions in daytime and night time. Experimental results demonstrate that the proposed vehicle detection approach is effective and feasible in various environments.