First a little about PEI, it's one of our three Maritime provinces and is the smallest in the nation in land area and in population. In September 1864, Prince Edward Island hosted the Charlottetown Conference, which was the first meeting in the process leading to the Quebec Resolutions and the creation of Canada in 1867, though PEI did not end up joining right away. It also, as many of us know, it very popular for it's potato production. Producing 25% of Canada's potatoes.
Basically the soil is red because it has a lot of rust in it. The soil has a high iron oxide content, which rusts on exposure to air. Prince Edward Island was formed a long time ago on sedimentary bedrock of soft, red sandstone, which produces rich, red soil.
It also erodes very easily and PEI often ends up with a lot of red snow from wind storms in the winter season!