In today's world, it becomes critical for a self-driving car to detect the vehicles irrespective of it being a day or night. We propose a real-time vehicle detection using a sequence of night-time thermal images. Moreover, the thermal images have the capability of retaining even the minuscule vehicle details in a dim environment. For an efcient vehicle detection, the thermal image dataset collected during the dusk and night is used for training purposes. Subsequently, the contrast enhancement and sharpening of these images are performed using the Thermal Feature Enhancement (TFE). Then the concatenated images are supplied as the input to allow the model to learn more efectively. Besides, we also propose an improved convolution network model entitled as the Thermal Image Only Looked Once (TOLO) model for vehicle detection. Additionally, we propose a method called as Low Probability Candidate Filter (LPCF) to compensate the probability of not-easy-to-detect vehicles. Our proposed method produces better results for the F1-measure in comparison with existing methods.