Visual acuity (VA) measurement is for a subject to test his/her acuteness of vision. Several kinds of automatic VA test are gradually developed and used in recent years. Without experimenter, the traditional way for a subject to speak out or wave a hand in response to the direction of optotype is then replaced mostly by the contact based response such as pushing buttons or keyboards on a device nowadays. However, the contact based response is not intuitive as speaking or waving hands, and it may distract subjects from concentrating on the test. To overcome these problems, we propose a hand motion recognition based visual acuity (HMRVA) measurement which keeps the advantage of automatic VA measurement, and also allows subject to respond in an intuitive contactless way. A velocity based hand motion recognition (V-HMR) algorithm is used to classify hand motion data collected by a sensing device into one of the four directions of optotypes. Based on the V-HMR scheme, a maximum likelihood based visual acuity (ML-VA) estimation algorithm is developed for VA estimation and is implemented on a tablet. According to the experimental results, we can conclude that the proposed HMRVA system achieve our goals to provide accurate and efficient automatic VA tests.