Automatically detecting the defects on the randomly textured surfaces for industrial purpose is a demanding procedure due to the ambiguity between defects and textures, lack of defect-labeled data and the must-have extreme accuracy. In this paper we proposed a procedure as the beginning of automating the defect detection on woods with randomly textured surfaces by employing 3 different architectures of convolutional neural networks. The deep convolutional neural network resulted in 99.80% accuracy, discriminating among normal wood and the other 4 types of defects images. The models were evaluated and understood by visualizing the saliency maps. The results from our work implies that other industrial images with defects on randomly textured surfaces may apply the similar procedures to accelerate the automating of defect detection and progressing of industry 4.0.