One of the most important requirements to bring the interactive video-on-demand services to residential consumer use is the scalability of the video server. In this paper, the design issues for a scalable video server is described and, according to these issues, the system architecture of the scalable video servers are proposed. The system uses ATM switching technology to scale up the dimension of the service. There are four major components within the proposed system, namely the Server Headend Control (SHC) unit, the stream router, the Video Pumping Unit (VPU) and distributed storage devices. The functions of each components will be described and the key components in the system will be the stream router and the VPU. Operations and performance limitations on the video pumping unit are given and the result shows that a single storage device can serve only a limited number of video streams. Finally, the paper focuses on the stream router as it is used to connect multiple VPUs. Experimental results show that the proposed stream router can produce guaranteed throughput and bounded delay variance for video applications.