We analyse the behaviour of a bistable vertical cavity semiconductor laser under the influence of optical feedback. Without feedback, this laser emits either vertically or horizontally polarised light with a region where polarisation bistability is observed. In this paper, we show that weak optical feedback can induce polarisation switching regardless of the polarisation of the re-injected light. These switches are quite similar to those observed in the current noise induced case. However, remarkable differences appear in the probability distribution of switching times depending on the polarisation state of the re-injected light. Experimental results detailing these differences are presented and the possible contributing mechanisms discussed.