Traditionally, stars are associated with the divine. The origin of constellations, for example, are told with some frequency by poets, especially those who are relating myths. Theoi has a large collection of them.
While this probably endured at least in some form in traditional religion, various natural philosophers had other ideas about the composition of stars.
One of the more popular positions was that developed by Aristotle, who claimed that the stars are made up of "aether" (αἰθήρ), though that must be distinguished from Plato's usage: Quoting Wikipedia:
However, in his Book On the Heavens he introduced a new "first" element to the system of the classical elements of Ionian philosophy. He noted that the four terrestrial classical elements were subject to change and naturally moved linearly. The first element however, located in the celestial regions and heavenly bodies, moved circularly and had none of the qualities the terrestrial classical elements had. It was neither hot nor cold, neither wet nor dry. With this addition the system of elements was extended to five and later commentators started referring to the new first one as the fifth and also called it aether, a word that Aristotle had used in On the Heavens and the Meteorology.
Aether differed from the four terrestrial elements; it was incapable of motion of quality or motion of quantity. Aether was only capable of local motion. Aether naturally moved in circles, and had no contrary, or unnatural, motion. Aristotle also stated that celestial spheres made of aether held the stars and planets. The idea of aethereal spheres moving with natural circular motion led to Aristotle's explanation of the observed orbits of stars and planets in perfectly circular motion.
It's not just the heavens that are made of aether, but the stars they hold as well, as Aristotle claims in On the Heavens:
We have next to speak of the stars, as they are called, of their composition, shape, and movements. It would be most natural and consequent upon what has been said that each of the stars should be composed of that substance in which their path lies, since, as we said, there is an element whose natural movement is circular.
Interestingly enough, I think an earlier philosopher, Anaxagoras of Clazomenae, who preceded Aristotle's work by a century, was closer to the truth:
He was the first to give a correct explanation of eclipses, and was both famous and notorious for his scientific theories, including the claims that the sun is a mass of red-hot metal, that the moon is earthy, and that the stars are fiery stones.
