A full stage bank serial number (SN) recognition system is proposed in this paper. We introduce Block-wise Prediction Networks (BPN) to treat the localization of an SN as block-wise binary classification, which can be considered as a coarse version of dense/pixel-wise prediction used in semantic segmentation. The benefits include short execution time, which is equal to 85.22 ms in CPU, and the use of global features instead of local features to improve the segmentation. Our system then separates the localized Region of Interest (RoI) into individual characters, and feeds them into softmax CNN classifier. Experimental results show that the proposed method can achieve 99.92% and 99.24% accuracy for character and SN of Renminbi (RMB), respectively, tested with 2,368 two sides images of 1,184 RMB bills.