Spherical panoramic image processing has received renewed interest in the fields of photogrammetry and computer vision. The difficulty in spherical matching is largely due to inevitable image distortions introduced from the equirect-angular projection. In this paper, we present an effective strategy for tackling the problem of distortion to improve the performance of spherical image matching. The effectiveness of the rectified matching is evaluated with simulated data and compared with state-of-art methods. In addition, experiments with respect to matching between omnidirectional and planar images, establishing 2D-to-3D correspondence with lidar, and the pose estimation of a spherical image sequence are conducted. The results verify the utility of the proposed method, which provides stable and evenly distributed corresponding points, and it is suitable for integration with conventional techniques for further 3D exploitation of imagery.