As Alex answered, the math was well known, and the mapping problem could be solved by surveying techniques. i.e., by repeated measurements of angles and known distances, as you can see the students doing in any modern civil engineering course.
It was obviously not as exact as modern mapping. I remember seeing a globe from 1938. The shapes were very close to the correct ones, but still they were noticeably different even to the naked eye.
Manual surveying could be an herculean task. The Cassini family is well known by a massive 4 generation, 65 year effort to map France at the behalf of the king:
Here you can see the best known map before and after the Cassinis work:
It is easy to note how Cassini's map is much better than the previous one.
The mark zero was Paris. Triangulation error accumulates from Paris, therefore there are more errors on the south of France.
You can see their map in detail overlayed with modern maps with a google-earth interface here:
It is worth to have a look, it is much more accurate than what you would think.
The Cassinis had lots of funny and tragic stories over their decades long work. Everybody knew that if Le Roi could have accurate maps, then His majesty's taxation would also be more accurate. Therefore, the surveyors had lots of enemies. Sometimes angry landowners would kick them out of their dear lands or even shot them, lest the king have an accurate measure of their riches.