In this work we investigate the hybrid analog and digital combining applicable to both multi-user single-input multiple-output (SIMO) and single user multiple-input multiple-output (MIMO) systems via modeling equivalence. We consider the mean square error (MSE) of the estimated signal after the combiner, and indicate that the optimal MSE can be expressed as the function of the analog combining matrix. Then by exploiting the proposed MSE decomposition, we propose two hybrid combiner designs which are able to optimize the analog combining matrix with low complexity. We evaluate the proposed designs using simulations. Results show that the proposed designs can provide near-optimal performance and outperform the existing designs in MSE, sum-rate, or complexity.