Finding “first” mentions of significant, people, places and terms is always a delight for the rare newspaper collector, and with the internet–and the time required–many fascinating items can be found. 
The term “Columbia” as a reference to America, very commonly used through the 19th century in both print & image, was first used in the London publication “Gentleman’s Magazine” in 1738. Because the printing of Parliamentary debates was illegal in England, they appeared under the thinly veiled heading of “Debates in the Senate of Lilliput” or similar heading, with names & places often fictitious or taken from Johnathan Swift’s famous work, which was the literary sensation at that time. The term Columbia was coined by the famed Samuel Johnson, a regular contributor to the “parliamentary” reports found in “The Gentleman’s Magazine“.
In the June issue of 1738 , the debates from Parliament note: “…It is observable that their conquests and acquisitions in Columbia (which is the Lilliputian name for the country that answers our America,) have very little contributed to the power of those nations…”.
A significant “first use” of a very popular poetic name for the United States of America.