This work derives an optimal structure of microphone array for sound source localization under the conditions of fixed microphone numbers limited distance between each microphone pair and different signal-to-noise ratios (SNR). The location estimation algorithm is based on the time difference of arrival (TDOA) of each microphone pair. The estimation accuracy is influenced by the TDOA errors and is dependent on the microphone array geometry. This work proposes a geometry optimization formulation using the relationship between the singular values of the estimation matrix and the estimation error. Under the far field condition, an analytical solution can be derived to minimize the Cramer-Rao Lower Bound (CRLB). In near field, the result leads to a set of nonlinear equations and is source location dependent. With additional constraints, several numerical solutions are discussed.