This work experimentally demonstrates a 60-GHz direct-detection polarization division multiplexing and orthogonal frequency division multiplexing (PDM-OFDM) system that does not require polarization tracking. The frequency offset of un-modulated optical carriers and a novel training symbol design enable the estimation of optical polarization mixing and wireless multi-input-multi-output (MIMO) channels for signal demodulation. We also adopted an empty subcarrier to mediate noise enhancement. Simulation and experiment results both demonstrate the feasibility of this approach. With pre-compensation, we achieved a BER of 1 × 10-3 under all polarization states over 10-km fiber and 3-m wireless MIMO transmission. The maximal allowable transmission data rate of up to 52 Gb at the worst case of states of polarization can be achieved.