First thing you should do upon your first visit to Capri Island is take a tour boat surrounding the island. It may seem a bit touristy but Capri has been an island that has attracted tourist-types for 2000 years, one of which was the tyrannical Tiberius.
The boat tour, depending upon the tides, can take you around the entire island or only half. It is a relaxing way to see the island you are about to explore wether it be on foot, by convertible taxi or 2 Euro bus ride.
During the tour you will see the desired red coral that grows upon the rocks within the lagoons, the famous Fragolini rocks, the various luxury homes that are perched upon the limescale cliffs and various mythically linked caves & arches.