If you want to go shopping with the locals there is really only one way to do it, the laiki agora. The laiki agora, or just laiki as most people call it, translates to the people's market. In the olden days this was a weekly market where all of the local farmers would have the chance to come into the cities and villages to sell their wares. Today the tradition lives on, however many of the sellers may not be farmers themselves, and some crops are imported from other parts of Greece, or even from other parts of the world. This is still the place where many people get their weekly fruits and vegetables.
The products are generally sold by weight, and bargaining isn't always accepted (particularly if you’re buying less than a kilogram). Most of the vendors speak English, and sampling before you buy is often encouraged. The laiki can get very loud and a little chaotic, so some might be overwhelmed, but getting there early often leads to less crowds.
To find a laiki is a bit of a challenge for foreigners since they only happen once a week at fairly random locations. Asking a local can sometimes get results, but most people only know of the laiki where they shop, so hotel employees may not know of a laiki in the area of the hotel. Downtown Athens has 3 great laikis that I use, all of which take place on Fridays from about 8am till 2pm. Kolonaki has one about halfway up the hill, and Pangrati has two one near the Divani Caravel hotel and one at the end of Spirou Mercouri.