This has to be one of the highlights of San Francisco, the San Francisco Botanical Gardens.
Their website: http://www.sfbotanicalgarden.org/
What makes the gardens unique is they have plant collections based on various climate conditions including the Cloud Forrest, Mediterranean, Temperate, and Montane Tropic climates. My favorite is the Succulent garden, which has an amazing collection of cactus and succulents.
The walking is wonderful and you can even visit a redwood forest in the gardens.
And next door is the Japanese Tea Garden, the California Academy of Science, as well as the de Young Museum.