He was born at De Ruyter, New York, educated at an academy at Rome, New York, and at the age of 17 he became a bank clerk. In 1853 he removed to Chicago, served for three years as bookkeeper, and in 1858 entered tile banking house of the Merchants Loan and Trust Company, of which he was cashier in 1861-1868. Afterwards be became successively assistant cashier, vice-president and president of the First National Bank of Chicago, one of the strongest financial institutions in the Middle West.
He was chosen in 1892 president of the board of directors of the World's Columbian Exposition, the successful financing of which was due more to him than to any other man. In politics he was originally a Republican, and was a delegate to the national convention of the party in 1880, and chairman of its finance committee.
In 1884, however, he supported Grover Cleveland for the presidency, and came to be looked upon as a Democrat. In 1892 President Cleveland, after his second election, offered Gage the post of secretary of the treasury, but the offer was declined. In the free-silver campaign of 1896 Gage labored effectively for the election of William McKinley, and from March 1897 until January 1902 he was Secretary of the Treasury in the cabinets successively of Presidents McKinley and Theodore Roosevelt.
As Secretary of the Treasury, Gage was influential in securing passage of the Gold Standard Act of March 14, 1900, which reestablished a currency backed solely by gold. Since this limited the amount of currency in circulation, it initiated a period, continuing until 1912, in which the Secretary of Treasury was obliged to interact in the money market by introducing into circulation the Treasury surplus. The inability of the Treasury to respond to the needs of the market and the need for a currency which would expand and contract with the needs of the nation, led to the creation of the Federal Reserve in 1913 to regulate the money market.
Gage resigned in 1902 to become a banker in New York. From April 1902 until 1906 he was president of the United States Trust Company in New York City. His administration of the treasury department, through a more than ordinarily trying period, was marked by a conservative policy, looking toward the strengthening of the gold standard, the securing of greater flexibility in the currency, and a more perfect adjustment of the relations between the government and the National banks.