Ac adapter buzzing noise

The buzzing noise that you are hearing is normal to that point that you should be able to hear it when the brick is next to your ear. If you are about a foot away from the ac adapter and you can still hear it then it is nearing end of life or already faulty.

Without a load on the device the hum should lessen due to circuits that reduce power output that allow the ac adapter to "idle" while waiting to be plugged in and will also allow the brick to cool down. Once plugged in the output circuit is activated and the louder hum will return as well as the ac adapter becoming warm up again.

If you are interested the rectifier circuit in the ac adapter will filter half of the ac adapter wave and then a voltage regulator will clip the top wave to generate an even source of DC power. Using these circuits in such close proximity of the magnetic field of a transformer can contribute to a high pitched hum from the device that is not very loud and is typically dampened by the casing for the device. As parts in the circuit degrade they can become more susceptible to the magnetism and heat strains that are put on them which can result in small fluctuations in power and voltage provided as well as a louder hum. There is nothing you can do about this degradation as the longevity and hum from the device is dependent on the design of the device, the quality of the parts used, and how well the heat generated is dealt with.