Victoria's historic downtown is a beautiful place to spend an afternoon. Next to the picturesque harbour, it's crammed with great places to eat and drink as well as great shops.
My favourite downtown streets for shopping are Broad, lower Fort, Government Street and lower Johnson and Store Street. These are all in Old Town and have many unique shops - Footloose Shoes, Hughes, Outlooks for Men, Roberta's Hats, Capital Iron, Ocean River Sports, Sitka, kitchen shops, great clothing shops on Johnson, etc, as well as good "chains" like Roots, The Bay, Mountain Equipment Co-op, Lululemon, and so on. There is also an entire shopping mall behind The Bay. Plus you can stop and have great coffee, food, and drink and enjoy the scene.
Now, if you want to shop here, heed this local's advice: there are some of what people like to call "cruise ship shops" at the lower end of Government Street near the harbour and Empress Hotel. This is not real Victoria shopping.