The title Earl of Elgin was created in 1633 in the Peerage of Scotland for Thomas Bruce. His son, Robert, succeeded to that title, and was also created Earl of Ailesbury in the Peerage of England. The two Earldoms continued united until the death of the fourth Earl of Elgin, when the Ailesbury title became extinct, and the Elgin title passed to the Earl of Kincardine. Thereafter, the Earldoms of Elgin and Kincardine have remained united. The most famous Earl was the 7th Earl, who recovered the so-called Elgin Marbles from the Parthenon. In Dublin there are roads that come from the Earl's titles. These are Elgin Road and Ailesbury Road.

Other associated titles are: Lord Bruce of Kinloss (created 1633), Lord Bruce of Torry (1647) and Baron Elgin (1849). The first two are in the Peerage of Scotland; the last is in the Peerage of England.

The family seat is Broomhall House , three miles south-west of Dunfermline, Scotland.

Earls of Elgin (1633)

• Thomas Bruce, 1st Earl of Elgin (1599-1663)
• Robert Bruce, 2nd Earl of Elgin, 1st Earl of Ailesbury (1627-1685)
• Thomas Bruce, 3rd Earl of Elgin, 2nd Earl of Ailesbury (1656-1741)
• Charles Bruce, 4th Earl of Elgin, 3rd Earl of Ailesbury (1682-1747)
• Charles Bruce, 5th Earl of Elgin and 9th Earl of Kincardine (1732-1771)
• William Robert Bruce, 6th Earl of Elgin and 10th Earl of Kincardine (1764-1771)
• Thomas Bruce, 7th Earl of Elgin, 11th Earl of Kincardine (1766-1841)
• James Bruce, 8th Earl of Elgin, 12th Earl of Kincardine (1811-1863)
• Victor Alexander Bruce, 9th Earl of Elgin, 13th Earl of Kincardine (1849-1917)
• Edward James Bruce, 10th Earl of Elgin, 14th Earl of Kincardine (1881-1968)
• Andrew Douglas Alexander Thomas Bruce, 11th Earl of Elgin, 15th Earl of Kincardine (b. 1924)