This paper presents a scheme to detect building regions, followed by a reconstruction procedure. Airborne LIDAR data and aerial imagery are integrated in the proposed scheme. In light of the different buildings, we target the ones with straight and curvilinear boundaries. In the detection stage, a region-based segmentation and object-based classification are integrated. In the building reconstruction, we perform an edge detection to obtain the initial building lines from the rasterized LIDAR data. The accurate arcs and straight lines are then obtained in the image space. By employing the roof analysis, we determine the three dimensional building structure lines. Finally, the Split-Merge-Shape method is applied to generate the building models. Experimental results indicate that the success rate of the building detection reaches 91%. Among the successfully detected buildings, 90% of the buildings are fully or partially reconstructed. The planimetric accuracy of the building boundaries is better than 0.8m, while the shaping error of reconstructed roofs in height is 0.14 m.