The rapid evolution of digital equipments has led to the explosive proliferation of multimedia data in education, entertainment, sport and various applications. The development of automatic or semi-automatic systems and tools for digital content analysis and understanding becomes compelling. As important multimedia content, sports video has been attracting increasing attention due to commercial benefits, entertaining functionalities and audience requirements. Much research on shot classification, highlight extraction and event detection in sports video has been done to provide interactive video viewing systems for quick browsing, indexing and summarization. More keenly than ever, the audience desires professional insights into the games. The coach and players demand automatic tactics analysis and performance evaluation with the aid of multimedia information retrieval technologies. Therefore, sports video analysis is certainly a research issue worth investigation. In this paper, we review current research and give an insight into sports video analysis.