In telecommunications, a spectral mask is a mathematically-defined set of lines applied to the levels of radio (or optical) transmissions. The spectral mask is generally intended to reduce interference by limiting excessive radiation at frequencies beyond the necessary bandwidth. Attenuation of these spurious emissions is usually done with a pass filter, tuned to allow through the correct center frequency of the carrier wave, as well as all necessary sidebands.

The spectral mask is usually one of the things defined in a bandplan for each particular band. It is essential in assuring that a transmission stays within its channel. An FM radio station, for example, must attenuate everything beyond ±75kHz from the center frequency by a few decibels, and anything beyond ±100kHz (the channel boundary) by much more. Emissions on further adjacent channels must be reduced to almost zero.

FM broadcast subcarriers are normally required to stay under 75kHz (and up to 100kHz if reduced) to comply with the mask. The introduction of in-band on-channel (IBOC) digital radio in the U.S. has been slowed down by issues concerning the subcarriers it uses – and the corresponding increase in the amount of energy in the sidebands – overstepping the bounds of the spectral mask set forth for FM by the NRSC and enforced by the FCC.

Other types of modulation have different spectral masks for the same purpose. Many digital modulation methods such as COFDM use the spectrum very efficiently, allowing for a very tight spectral mask. This allows placement of broadcast stations or other transmissions on channels right next to each other without interference, allowing for an increase in a band's total capacity. Conversely, it is allowing the U.S. to eliminate TV channels 52 to 69, freeing up 108MHz (from approximately 700 to 800MHz) for emergency services and to be auctioned off to the highest bidder, while still retaining (although moving) all existing TV stations.

See also: modulation envelope