Mango

According to the book Wicked Plants [link], the outside of the mango fruit can cause irritation, and people who have had severe poison ivy outbreaks could be sensitive to the rind.