More and more streaming protocols are developed for the multimedia applications. However, many streaming protocols only consider the network stability, but not the characteristics of streaming applications. In order to cooperate with H.264/MPEG-4 AVC scalable extension which can achieve fine granularity of scalability at bit level to the time-vary heterogeneous networks, we design a TCP-friendly congestion control algorithm based on the bandwidth estimation to smoothly change sending rate to avoid unnecessary oscillations so that the subscription decision of SVC layers can be made to better utilize the network resource. In case of the unavoidable network congestion, we unsubscribe scalable video layers according to the packet lost rate and the recently received throughput instead of only dropping one layer at a time to rapidly accommodate the streaming service to the channels and avoid persecuting the other flows at the same bottleneck. In addition, the probing packets for estimating the available bandwidth are encapsulated with RTP/RTCP. The simulations show that the proposed congestion control algorithm for real-time applications efficiently utilizes network bandwidth without hampering the performance of the existing TCP applications.