The question is a bit confusing. The way I read it, you're asking why something expensive has enough demand to sustain a profitable trade ("How did the high price of spices allow such high demand?").
The answer is that it wasn't that expensive. A pound of spices might cost several days' worth of wages for an average craftsman, but a pound of pepper is a lot of pepper. It was a luxury item, to be sure, but it wasn't completely out of reach for the average middle class professional. Furthermore, that is not at all a prohibitive sum for the wealthy, which includes the ranks of not only the aristocracy, but also the growing class of merchants and higher level professionals and other large land owners.
And no, Western Europe was not "mostly serfs" during this period.
The claim that spices were worth their weight in gold (quite an exaggeration to tell the truth, for all but saffron) also shows why the spices had a market. Gold and silver were expensive, and yet there was clearly a market for it. The Spanish shipped in shiploads of it from the New World, for instance. Spices, which were cheaper and yet consumables, would logically have a bigger market as long as Europeans have a demand for them as luxury goods.
Of course, highly priced luxury items by their nature are not traded in high volumes. Neither was spices: the supply, in Europe, was severely limited. Much later when the VOC gained its spice trade monopoly, the Dutch began shipping around 270 tons of cinnamon per year, which was at the time a vast quantity that outstripped local supplies. Contrast that with modern cinnamon production of ~35,000 tons. And as more and more trade routes were established and transplanted spices started to be growing in more places, supply enlarged and the value of spices fell.
Note that limited supplies in the late Medieval period didn't affect the profitability of the spice trade. The spice trade was "such a profitable venture" precisely because "spices are quite expensive when they reach their terminal buyer" - relative to their cost in Asia, which is something like 1/10 or 1/100 the sales price. And again, as supplies expanded, prices fell.
As luxury goods, The main consumers of spice were the wealthy, middle to upper class members of society. Religious and supposed medicinal uses aside, these people valued spices because it is expensive; it formed a social status symbol, desirable for showing off. The marriage of Duke Georg of Bavaria to Polish Princess Hedwigis was celebrated with a feast involving almost 1300 pounds of various spices, for example.
While the rich could afford the a wide variety of expensive spices in ridiculous amounts (i.e., Duke Georg), it wasn't beyond the means of the common folk either. As noted above, a pound of pepper might cost a craftsman several days work - enough to make it a luxury goods, but not so expensive that it was unaffordable. So even (well to-do) peasants could afford pepper, the cheapest of the common spices.