What a wonderful question! Dom Henrique is one of the chief characters of this story, and that means we need to explore him more than the general Portuguese knowledge. These explorations as a whole were a manifestation of the soul of this great man who was well beyond his contemporaries in what he wanted and expected from his sailors. I'm basing this answer on Bradford's 'A Wind from the North'.
In short, the answer is that when Dom Henrique started with the voyages of exploration, he was unsure about what would be found, but by the end of his life he was sure there was a way around Africa, as evidenced on the map of Fra Maura. However, for the average sailor the religious connotations of sailing towards 'the end of the world' and their possible demise there were far more important than what else was there.
It doesn't look as Ptolemy's map was in common circulation though his other works were; Dom Henrique's resolve may also have been strengthened by hearing about the Phoenician circumnavigation of Africa. Yet, the overarching purpose of these missions was to learn more and to discover—often in these cases the final goal may come unexpectedly.
1420's
Firstly, far more important than a map of the world was the understanding that the sailors of the time had. For them, Cape Not was the beginning of the end of the world:
Quern passar o Cabo de Nam, ou tornara ou nam.
He who would pass Cape Not, either will return or not.
Only a touch further south lay Cape Bojador which was the limit of the 'known' world, and the two things beyond it were "the boiling sea and the Ocean of Darkness":
At that point it seemed as if the waters ran downward in a curve, so that, it was said, no ship could ever sail back. The winds drew always from astern, so that even if the curved ocean was a myth, the fact remained that a square-sailed vessel had little chance of beating her way north again.
Bradford speculates that the tale of the Phoenician circumnavigation of Africa was known to Dom Henrique—from the scholars who ended up gathering at Sagres—which would possibly be a better medication than any map.
Knowledge of Ptolemy
What makes a survival of the Ptolemaic map in Portugal or for the Portuguese suspect is that Azurara, the chronicler of Dom Henrique's life, writes:
"he desired to know what lands there were beyond the Canary Isles and a cape called Bojador. For at that time there was no knowledge, either in writing or in the memory of any man, of what might lie beyond this cape."
Yet, Ptolemy is mentioned in specific terms, however (all emphasis and addition original; "he" refers to Dom Henrique):
We know, in fact, that he suspected there were further islands or land masses in the Atlantic from the chronicle of Diogo Gomes, a ship's captain who spent many of his years in Prince Henry's service, and who was with him when he died. "The Prince wished to know," he wrote, "about the western ocean, and whether there were islands or continents beyond those that Ptolemy described. [The Alexandrian astronomer's Guide to Geography was a standard reference book of the period.] So he sent caravels to search for lands."
Common Understanding
The main problem, however, was not whether a way was shown on a map. Dom Henrique's sense of the unknown prompted him to send the ships out; he was expecting to find new things and was willing to wait out until he got results.
What was more difficult was finding good captains and brave sailors to sail that way. At least before the discovery of Maderia, no riches or wealth was expected from the south given it was the traditional Christian 'end of the world'. Indeed, many sailors (perhaps more superstitious than the captains or Dom Henrique and also likely to profit less than the leaders of the expeditions) were very reluctant to proceed to the south:
The Sea of Obscurity, or the Ocean of Darkness, it was called. There dwelt monsters, and the sun stood so high overhead that the water boiled. The men who lived on the outermost edge of the African shore were known to be burned black by the sun, so how could man go farther without being roasted alive? ...
"To sail south," murmured the sailors. "What is the point of that? How can we possibly pass beyond the bounds that are established? What profit can the Prince gain from the loss of a few poor sailors?..."
Bradford notes the dialogue is not made up, but he doesn't specify a source. Azurara is brought out as one example, but Bradford has used other chroniclers as well.
The desolate nature of the desert next to the Atlantic in that part of the North African landmass would not have incentivized the Portuguese any more, at least not until they got past those to the rich delta of the Senegal. Cabo Branco was only reached in 1441, and it's near there that some trade was conducted with local tribesmen, but that was a good two decades into the exploration progress.
Lastly, the specific virtue of the Prince as the overseer of these voyages is brought out with respect to these world-views:
"The Prince always welcomed the captains of his ships with great patience, never disclosing any resentment, listening graciously to the stories of their adventures, and rewarding them as men who were serving him well. But immediately he sent them back again to repeat the voyage — either them or others of his household — impressing upon them, more and more strongly, the mission he required them to accomplish ..."
1460's
Towards the end of his life, the Prince was quite certain that Africa could be rounded. The map by Fra Maura incorporated knowledge that the Prince's voyages had uncovered, and that shows the Cape of Good Hope (described below):
Bradford describes the making of this map:
The map was finished in the spring of 1459, and it is evidence of the information that Henry had collected from all over Africa that the Cape of Good Hope, called the "Cavo di Diab," is shown on this world map. Forty years before Vasco da Gama rounded the cape, either the Prince or his cartographers had heard of an Indian sailing vessel that in 1420 had rounded the cape from the east after a voyage of 2,000 miles.
The Ptolemy map shows the Indian Ocean as an enclosed sea separated from the Atlantic by land extending from Africa to Malaya. Therefore, it suggests it is impossible to reach the Indian Ocean from Europe by sailing anticlockwise around the African continent. I believe the Ptolemy map was reconstructed in Europe at or before 1410 via thirteenth-century Byzantium whilst the Portuguese began their southern exploration of the West African coast in 1421 under Henry the Navigator.
