Device-to-Device (D2D) communication in cellular network is defined as the direct communication between mobile users involving neither eNB nor core networks. D2D improves spectrum efficiency and saves transmission power. In this paper, we propose a mode selection scheme that consists of Cellular Mode, D2D Mode and Relay Mode. We organize the D2D pair through proximity discovery, and UEs make all decisions of D2D grouping. Our method can reduce eNB's load of network control without causing heavy computation to UEs. By applying three modes in the network, it can minimize the drawback of D2D communication underlay and maximize the spectrum efficiency and system throughput over the LTE cellular networks. We use MATLAB simulator to evaluate our proposed scheme, the results show that our method not only upgrades the system throughput but also reduces the impact to cellular users caused by adding D2D communication to the network.