The District of Columbia is a federal district that takes up exactly same area that the city of Washington is in. Therefore we could refer to either DC or Washington and be referring to the same place. The city’s not in a state, but it is in the country, and so it has to be in something, so it’s in a district that we named Columbia.
But why Columbia, you ask? Columbia was one of the old poetic names for our nation, kind of like we use „America“ or „the States“ or „Uncle Sam“ today. It references Christopher Columbus and a woman’s form vaguely akin to the French Marianne. It fell out of popularity about 100 years ago, but we used to have pro-American songs like „Hail Columbia“ (which, incidentally, is still the Vice President’s anthem as opposed to „Hail to the Chief“). CBS used to be called „Columbia Broadcasting System“.
But why specifically did we name the district Columbia? It’s partially because in 1791, the year they began planning the new Federal City and had to come up with a name for the district, it was one year short of the 300th anniversary of Christopher Columbus’s trip the the new world, and there were still some people grumpy that we hadn’t actually named our new nation Columbia instead of something unwieldy like „The United States of America.“ So we named the new Federal area „The Territory of Columbia.“ The first stone of the White House, which was the first official Federal building, was placed the day after the 300th anniversary of Christopher Columbus’s voyage, so you can see how these things dovetailed nicely.
There was overwhelming support for naming the city itself Washington, which they did. George Washington always just called it the „Federal City“ because he was a modest dude, and his name for it was used on occasion for many years afterwards.
Finally, we call it Washington, D.C because we don’t want to confuse it with the state, but we’re talking about the city, not the federal territory that the city takes up the entirety of.