There is strong recent interest in exploiting the millimeter wave (mmWave) frequencies for cellular communication. For signal transmission studies, one needs a channel model that can simulate the spatial variation of mmWave channel impulse responses (CIRs) in treated operating environments. We consider reconstruction of spatially varying CIRs by interpolation over a set of existing CIR snapshots for some randomly scattered geographical locations. Such interpolation, however, is not trivial work, as we need to handle drifting of path delays over different spatial locations as well as deal with appearing and disappearing of paths. We propose a way to address these issues, and we present some numerical results for a typical urban area.