In this article, we present a substrate-integrated waveguide-based band-pass filter with wide stop-band rejection property. This band-pass filter is made up of two substrate-integrated waveguide cavities, which couple to each other by an aperture on their common waveguide side wall. By using the empirical formula  for obtaining the equivalent rectangular waveguide dimensions, the substrate integrated waveguide structure could then be properly approximated by a closed rectangular waveguide. We employed the rigorous mode-matching method to analyze the electromagnetic boundary-value problem of the structure containing multiple discontinuities. In comparison with the conventional end-to-end coupled cavity filters, we found that our new side-wall-coupled cavity filter can suppress the second resonance frequency and therefore, provide a considerably wide rejection band. It is interesting to note that the suppression of the second resonance frequency can be achieved by properly shifting the second cavity in lateral direction. To explore its physical insight, the resonant modes in the substrate integrated waveguide cavities are carefully inspected for realizing the physical picture of wave process taking place in the two cavities system. In addition to the theoretical analysis, we have fabricated the filters and measured their scattering parameters. The good agreement between the theoretical and measured results validates the accuracy of numerical computation, and also shows the promising performance of the proposed filters.