This paper proposes a novel RHF-Net (Recursive Hybrid Fusion pyramid network) to solve the problem of small object detection on real-time embedded devices. Though the object detection accuracy rate is improved by a large margin with SoTA (State-of-The-Art) models, e.g., SSD, YOLO, RetinaNet, and RefineDet, they are still problematic for small object detection and inefficient on embedded systems. One novelty of the RHF-Net is a bidirectional fusion module) that allows to fuse feature maps with both the top-down and bottom-up directions to generate flexible FPs for small object detection. This module can be easily integrated to any feature pyramid based object detection model. Another novelty of this net is a recursive concatenation and reshaping module which can recursively concatenate not only high-level semantic features from deep layers but also reshape spatially richer features from shallower layers to prevent small objects from disappearing. RHF-Net net adopts computationally low-cost and feature preserving operations in the fusion, thus it is efficient and accurate even on embedded devices. The superiority of RHF-Net is investigated on the COCO benchmark and UAVDT dataset in terms of mAP and FPS.