The coefficient values and number representations of digital FIR filters have significant impacts on the complexity of their VLSI realizations and thus on the system cost and performance. So, making a good tradeoff between implementation costs and quantization errors is essential for designing optimal FIR filters. This paper presents our complexity-aware quantization framework of FIR filters, which allows the explicit tradeoffs between the hardware complexity and quantization error to facilitate FIR filter design exploration. A new common subexpression sharing method and systematic bit-serialization are also proposed for lightweight VLSI implementations. In our experiments, the proposed framework saves 4951 additions of the filters with 2's complement coefficients and 1020 of those with conventional signed-digit representations for comparable quantization errors. Moreover, the bit-serialization can reduce 3335 silicon area for less timing-critical applications.