To effectively mitigate traffic congestion, many densely populated and well-developed cities worldwide decided to charge in-town traffic a congestion fee and the effectiveness of the congestion charge policy has been proven. However, a successful congestion charge scheme mainly depends on a clear understanding of urban travelers' responses. To do so, this study aims to model behaviors of travelers, including car drivers and motorcyclists, in response to in-town congestion charge in Taipei City and to propose feasible in-town congestion charge scheme accordingly. Four possible choices of travelers are defined as: 1) Pay for the charge, 2) Shift to off-peak hours/Cancel the trip, 3) Shift to public transportation, and 4) Shift to other private modes. Due to the possible correlation among alternatives and the potential heterogeneity among travelers, Multinomial Logit model (MNL), Nested Logit model (NL) and Mixed nested Logit model (MXNL) are estimated and compared based on a large-scale post-mail questionnaire survey. A total of 5,906 valid questionnaires were returned, including 3,450 car drivers and 2,536 motorcyclists. Among them, a total of 355 drivers and 314 motorcyclists who have morning peak-hour commuting experiences in Taipei City were selected. The estimation results show the existence of correlation among alternatives and heterogeneity for car drivers. Additionally, motorcyclists are much more sensitive to the charge than car drivers. The proportion of drivers and motorcyclists who are discouraged by various congestion charges are also predicted based on the estimated model. Suggestions for implementation of congestion charge are then proposed accordingly.