Switched systems theory is used to analyze the stability of vision-based observers for 3D localization of objects in a scene. Observers or filters that are exponentially stable under persistent observability may have unbounded error growth under intermittent sensing loss, even while providing seemingly accurate state estimates. Therefore, conditions are developed based on an average dwell time criteria to guarantee state error convergence with a known decay rate. In cases where sensing is controllable, these conditions relax path constraints for visual servoing applications. The conditions are developed in a general form, applicable to any exponentially stable observer, and utilize object motion knowledge to maximize the allowable time spent in stabilizable, but unobservable, periods.