For radio-based indoor localization, the approaches founded on the radio fingerprint concept are efficient duo to low cost and the ability to handle occlusion effects. However, the approaches require a lot of human labor to label training data for radio map (fingerprint) construction. To address this issue, in this paper, we proposed an unsupervised framework to learn a Wi-Fi radio map in an indoor environment. Unlike conventional approaches that depend on a simulated radio map or a prior radio propagation model to reduce human efforts, our method uses Wi-Fi and IMU signals collecting by crowdsourcing to build a robust radio map automatically. More concretely, four types of constraints are fused by the proposed radio map optimization procedure. They include the alignment of Wi-Fi landmarks, the displacement constraint, the manifold-based smooth constraint, and the inter-trajectory constraints. Our experiment results also show the effectiveness of the unsupervised radio map.