Small cells have been proposed to improve the signal quality of cell edge users and enhance the serving coverage. But a large number of small cells will cause severe interference and high energy consumption. Hence, green communication is an important issue. The main contribution of this paper is to provide an interference-aware small cell on/off mechanism in a downlink heterogeneous network system to reduce inter-cell interference in the sense of energy saving. Two criteria for the small cell on/off mechanism are proposed; one is based on the generated interference (GI) and the other is based on the signal to generated interference ratio (SGIR). In addition, two selection rules are provided; one is the exhaustive search (ES) method and the other is the one time (OT) method. In hyper dense small cells deployment scenario, our interference-aware small cell on/off mechanism can efficiently reduce inter-cell interference to maximize the total network energy efficiency. Compare to the traditional UE-cell association approach, our method provide 47.38% improvement at λb/λu = 0.4 and 61.44% improvement at λb/λu = 0.6.