Alleghany Corporation was incorporated by the railroad entrepreneurs Oris and Mantis Van Sweringen as a holding company for their interests.

After their bankruptcy in the Great Depression, control of the company fell into the hands of Robert Ralph Young and Allan Price Kirby. Young used the company as a vehicle for his vendetta against the J.P. Morgan banking interests, who had financed the Van Sweringens, and managed to defeat them and the Vanderbilt interests in a 1954 proxy fight for the New York Central Railroad. The failing New York Central was in worse shape than Young had bargained for, and he committed suicide shortly after being forced to suspend the dividend in January 1958.

The bankruptcy of the Penn Central railroad mostly ended Alleghany's involvement in the railroad business, but it has continued to this day with other investments. Allan Kirby's son, Fred M. Kirby 2nd, is chairman of the board and a sometime member of the Forbes 400 list of richest Americans.

The company's residual railroad investments led to president and CEO John J. Burns serving on the board of Burlington Northern Santa Fe Corporation from 1995 to 2004.