According to the website ibizaisla.es,
The flag of Ibiza features the flag of the Crown of Aragon with four
castles over the sea, in allusion to the four quartons the island
was divided into after its conquest.
Quartó (plural quartons) was the Catalan name of each of the four quarters the island was divided into after the conquest of Ibiza and Formentera by the Aragonese in 1235 (Catalan Wikipedia article about the quartons here).
Interestingly enough, a couple of PDF documents (1 and 2) at the Institute of Ibicene Studies (Institut d'Estudis Eivissencs in Catalan) mention that according to official documents from the time of the flag's creation in 1979, its basis had been an alleged flag of the ancient University of Ibiza (no mention was made of why it had four castles). According to the said institute, there's no trace of any flag of the ancient University of Ibiza either, so in 2007 there was a proposal for changing the flag for another, more coherent one from a heraldic point of view, but was ultimately cancelled.