Pont Neuf is the oldest bridge in Paris, contrary to it's name. (Literally, it translates to New Bridge. Neuf in modern French means nine, but in older French it also means new. Nouveau is the translation for new today). During the day, my favourite bridges in Paris are the Pont Des Arts (Lock Bridge) or the Pont Alexandre III, but at night Pont Neuf really comes into it's own.
This was taken one evening in my first week in Paris this past Summer. We'd come off a cruise of the Seine and settled on a bank for a few hours, with wine that was cheaper than most water back home and still tasted amazing.
Considering this experience cost me all of 49 cents (the wine), I think you'd be mad not to do it at LEAST once while you're in Paris.