The wikipedia article on Women in Shinto offers a few potential explanations (none of them specific to Okinoshima):
Some historians suggest that the practice may have originated from folk tales about women who were turned to stone or brought on natural disasters as they approached sacred sites on mountains, or owing to the choice of religious ascetics that rejected interactions with women, and commonly lived high in the mountains. Others suggest the prohibition is influenced by Buddhist doctrine against sexual relationships between monks and nuns. [...] Some shrines also ask recent mothers not to pass through torii gates to enter shrines for 72 days after childbirth.
Women's menstrual blood is a taboo in Shinto, thought to be influenced by the popularity of the Buddhist Blood Pond Sutra (血盆経 Ketsubonkyô). This doctrine preached that women were condemned to a blood pond hell for the sin of pollution through menstrual blood; only the prayer could spare them. Though Buddhist in origin, Shinto facilities emulated this practice in their teaching, encourage women, and men who had contact with menstrual women, to avoid shrines.
"Okinoshima Seen from Shintō", a paper by Norman HAVENS (Associate Professor at Kokugakuin University) available on the Okinoshima Heritage website, expand on the menstrual blood-related suggestion and offers a second possibility (h/t and thanks to Takahiro Waki's for the comment that led me to the doc):
The ordinary reasons cited for forbidding women from visiting Okinoshima are that (first) women’s visit to the island would make the goddess feel “jealous,” and (second) that women experience menses. While the origin of the former explanation is unknown, the latter objection can be compared to a similar taboo at the Grand Shrine of Ise. During pilgrimage to the Grand Shrine of Ise, women who experience their menstrual period were prohibited from entering the sacred borders. In the legendary folk records titled “Okagemairi Bunsei jin’iki” (“records of the miracles of the kami at the occasion of the mass pilgrimages to Ise of the Bunsei era), one account saying that a woman on her pilgraimage to Ise became ill just after crossing the river Miyagawa. Upon investigation, it turned out that the women had experienced her menstrual period. As soon as she recrossed the river and departed the sacred precincts, she regained her strength. While this work is a collection of anecdotal tales stressing the miracle of the Ise gods, it shows that women’s menstruation was a taboo event. Even at Ise, however, women who were not in their monthly cycles could visit the shrines in the same way as men. The taboo of blood is clearly an issue in modern Shintō (and Buddhism), but in that case the prohibition of blood at the Grand Shrine of Ise should be considered a general abomination of blood, rather than a specific taboo against women
Being a solitary island in the distant ocean, the ancient people who visited Okinoshima likely had to stay on the island for relatively long periods, in which case, the prohibition could be interpreted as meaning that women were prohibited from entering the island because they were likely to have a menstrual period during their stay.