This paper presents a design of simultaneous localization and mapping (SLAM) for an omni-directional mobile robot using an omni-directional camera. A method is proposed to realize visual SLAM of the omni-directional mobile robot based on extended Kalman filter (EKF). Taking advantage of the 360° view of omni-directional images, visual reference scan approach is adopted in the SLAM design. Features of previously visited places can be used repeatedly to reduce the complexity of EKF calculation. Practical experiments of the proposed self-localization and control algorithms have been carried out by using a self-constructed omni-directional mobile robot. The localization error between the start point and target point is less than 0.15m and 1° after traveling more than 40 meters in an indoor environment.