The Federal Reserve Bank of Atlanta is one of 12 Fed branches in the country, and has a free fascinating museum inside. Open weekdays from 9 a.m. to 4 p.m., the museum covers the development of American banks, the transition from bartering to standard currency, and counterfeit bills. You can even see money being printed and sorted.
You have to go through a metal detector and lock your things in luggage storage, but photos aren't allowed anyways. Grab a free bag of shredded money before you go.