The ancient and mysterious city of Carcosa was first named in the Ambrose Bierce short story, An Inhabitant of Carcosa . In Bierce's story, the city is barely described, and is viewed only in hindsight (after its destruction) by a character who once lived there.

Later, the city was used more extensively in Robert W. Chambers' book of horror short stories published in 1895, titled The King in Yellow. Chambers had read Bierce's work, and had also borrowed a few other names (including, for instance, Hastur) from Bierce's work.

In Chamber's stories, and within the apocryphal play (also titled "The King in Yellow") which is mentioned several times within them, the city is a mysterious, ancient, and possibly cursed place. The most precise description of its location given is that it said to be located on the shores of Lake Hali in the Hyades. The descriptions given of it, however, make it clear that it must be located on another planet, or possibly even in another universe.

For instance:

Along the shore the cloud waves break,
The twin suns sink behind the lake,
The shadows lengthen
   In Carcosa.
 
Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is
   Lost Carcosa.
 
Songs that the Hyades shall sing,
Where flap the tatters of the King,
Must die unheard in
   Dim Carcosa.
 
Song of my soul, my voice is dead,
Die though, unsung, as tears unshed
Shall dry and die in
   Lost Carcosa.
 
   Cassilda's Song in "The King in Yellow" Act 1, Scene 2.

Later writers, including H. P. Lovecraft and his many admirers, became great fans of Chambers' work, and incorporated the name of Carcosa into their own stories, set in the Cthulhu Mythos. For instance, in some stories, Carcosa is the residence of the Great Old One Hastur. Occasionally, Hastur will alter reality and merge parts of Earth into Carcosa; usually bringing along unwilling people as well.

In the short story "More Light," in which James Blish presented his version of a complete text of the play "The King in Yellow," Carcosa was described as having four singularities: that it appeared overnight, that no one could tell whether it sat upon the waters of Lake Hali or beyond them on the unseen farther shore, that the rising moon appeared to be in front of the city's towers rather than behind them, and that one knew the city's name to be Carcosa the moment one looked upon it. In Blish's version, Carcosa was created as a city of exile for the King in Yellow, because he was not king in Aldebaran.

Marion Zimmer Bradley also used the name Carcosa for a city on her fictional planet Darkover. According to her, this usage and the appearance of other distinctive names from Chambers' work dated from her own youthful fascination with "The King in Yellow" and her ambitions to produce her own reconstruction of the play on the basis of the fragments in Chambers' works. Only later did she transform those early fantasy writings into science fiction by relocating them from a parallel earth to a distant world under a red sun.