This paper presents a novel algorithm for background extraction and its application to vision-based traffic monitoring. A modified histogram algorithm is proposed to obtain reliable pixel intensity of background image. After background removal, moving objects can be segmented from the current image via a robust threshold operation. The threshold value is assigned through a measure of illumination variation. We applied the proposed method to a vision-based traffic monitoring system to segment moving vehicles from traffic image sequences. Given degraded on-line traffic images from compressed image transmission, vehicles are successfully segmented from the image frame. We employed a detection window, which behaves like loop detectors, to count the vehicles at a multi-lane intersection. Experimental results demonstrate that traffic flow can be obtained in real time.