The OH bond of methanol, ethanol and t-butyl alcohol becomes more anharmonic upon hydrogen bonding and the infrared intensity ratio between the overtone and the fundamental transition of the bridging OH stretching mode decreases drastically. FTIR spectroscopy of supersonic slit jet expansions allows to quantify these effects for isolated alcohol dimers, enabling a direct comparison to anharmonic vibrational predictions. The diagonal anharmonicity increase amounts to 15-18%, growing with increasing alkyl substitution. The overtone/fundamental IR intensity ratio, which is on the order of 0.1 or more for isolated alcohols, drops to 0.004-0.001 in the hydrogen-bonded OH group, making overtone detection very challenging. Again, alkyl substitution enhances the intensity suppression. Vibrational second order perturbation theory appears to capture these effects in a semiquantitative way. Harmonic quantum chemistry predictions for the hydrogen bond-induced OH stretching frequency shift (the widely used infrared signature of hydrogen bonding) are insufficient, and diagonal anharmonicity corrections from experiment make the agreement between theory and experiment worse. Inclusion of anharmonic cross terms between hydrogen bond modes and the OH stretching mode is thus essential, as is a high level electronic structure theory. The isolated molecule results are compared to matrix isolation data, complementing earlier studies in N2 and Ar by the more weakly interacting Ne and p-H2 matrices. Matrix effects on the hydrogen bond donor vibration are quantified.