Bridges are prone to suffer from multiple hazards such as earthquake, wind, or floods for the special structural characteristic. To guarantee the stability of bridge structure, how to precisely evaluate the scour depth of bridge foundation has become an important issue recently as most of the unexpected damage or collapse of bridges are caused by hydraulic issue. In this paper, a vibration-based bridge health monitoring system considering the response of superstructure only is proposed to rapidly evaluate the embedded depth of bridge column. To clarify the complex fluid-solid coupling phenomenon, the effect of embedded depth and water level was first verified through a series of static experiment. A finite element model with confinement simulated by soil spring was then established to illustrate the relationship between the fundamental frequency and the embedded depth. With the proposed algorithm, the health condition of the bridge can be inferred by processing the ambient vibration response of the superstructure. To implement the proposed algorithm, a SHM prototype system monitoring the environmental factors such as temperature, water level, and inclination was developed to support on-line processing. The performance of the proposed system was verified by a series of dynamic bridge scour experiment conducted in laboratory flume and compared with the reading from water-proof camera. The result has shown that by using the proposed vibration-based bridge health monitoring system, the embedded depth of bridge column during complex scour process can be reliably reflected.