Automatic location semantics prediction for living analytics based on smartphone data has attracted extensive attention in just recent years. Basically, this task can be formulated as a multi-class classification problem, where different location/places are regarded as different labels. Previous studies were mostly based on common classification techniques directly, neglecting the critical challenging issue of class imbalance in such a problem (e.g., people go to offices much more often than they go to cinemas). It is also noteworthy that in contrast to common multi-class problems where the classes can be treated independently and interchangeably, the places for labeling usually have important correlations, which should be taken account in the classification/labeling process. Moreover, several activities may occur in the same place and thus the same place label might convey different semantics. In this paper, we address the above issues for location semantics prediction by proposing the FS-Mining (Frame-based Semantics Mining) approach. We treat the raw sensor data in the smartphone as a sequence of short and non-overlapping frames, based on which the user behavior at each place can be characterized and the place semantics can be modeled. To deal with the issues of label relation and class imbalance, a multi-level classification model with class-split and class-merge mechanisms was also developed. An ensemble strategy was also employed to further improve the performance. Experiments on the dataset of Nokia Mobile Data Challenge  demonstrate promising performances for the FS-Mining approach.