The intelligent transportation systems (ITS) aim to provide services to transport and traffic management and supply more information and safer to various users. The visual-based systems are the most popular solutions for ITS due to their highly maintainable, flexible, and intuitive features. This paper uses a background extraction algorithm to extract initial color backgrounds from surveillance video based on an entropy-analysis concept. The moving objects can then be segmented quickly and correctly by a robust object segmentation algorithm. The segmented object can be used to analyze the trajectory and to provide the collision between pedestrian and moving vehicle. A license plate detection algorithm is also provided to detect the license plate in this study.