In this paper, we developed a fingertip finding algorithm working with a regular diffuser. The proposed algorithm works on images captured by infra-red cameras, settled on one side of the diffuser, observing human gestures taken place on the other side. With diffusion characteristics of the diffuser, we can separate finger-touch from palm-hover events when the user interacts with the diffuser. This paper contributes on: Firstly, the technique works with a regular diffuser, infra-red camera coupled with an infra-red illuminator, which is easy to deploy and cost effective. Secondly, the proposed algorithm is designed to be robust for casually illuminated surface. Lastly, with diffusion characteristics of the diffuser, we can detect finger-touch and palm-hover events, which is useful for natural user interface design. We have deployed the algorithm on a rear-projection multi-resolution tabletop, called I-M-Top. A video retrieval application using the two events on design of UIs is implemented to show its intuitiveness on the tabletop system.