I recently looked at this question again, and thought it might be another example of the colorization process being inaccurate. I thought to find other copies of this map (or other works at least by the same individual) showing the 'right' colors to prove a simple color error.
What I thought would be simple became very not simple. Here's what I have found.
The first logical step seemed to be researching the individual map and cartographer Pedro Reinel as well as other copies of his work. The Wikipedia entry for Pedro Reinel is limited, but does give us some connections to some other works, a starting point.
Four works are attributed to Reinel on the wiki page:
I would add a few other works from the period to consider, including
Lets start with the Reinel charts.
Here is a detail from a photo uploaded to Wikimedia of the 1485 chart:
Another clip of the same chart, from the wiki page overhead view:
We can see that the 'saltire' shape is present on the 1485 chart, but the colors still appear to be incorrect, with a gold coloration over a dark blue field. The shields for England and Ireland are also off, also having a dark field. (this is actually the closest I found to a 'correct' St Andrews Cross, if the Saltire were just white). So we can establish Reinel used a saltire for Scotland as early as 1485.
After that, lets look at a clip from the anonymous 1502 Cantino Planisphere from my second list above:
Here we see what again appears to be a gold colored saltire, over an uncolored background. This is our second view showing the Gold coloration. (An interesting paper here explores the possibility that this map is also Reinels' work.)
Next chronologically would be the OPs 1504 chart from above. The red Saltire on an uncolored field, same background as used for England.
Following this we can look at how the region was represented on the Miller Atlas in
We are back to a red saltire in the foreground over an uncolored field, but this image shows multiple stages of either indecision or correction. The regular (St Georges) cross can be seen lightly sketched in the background, and the beginnings of blue filling of the field around the cross. The English and Irish shields again show the dark field, with a hint of other symbols (charges) possibly faintly visible on the Irish arms.
The last work I have attributed to Reinel, the 1535 portolan of the North Atlantic, has something different yet. (sorry for the poor image, I was forced to resize a smaller clip)
Here we see what appears again to be a golden saltire, but this time over a divided field of both red and blue! The English and Irish arms are also quartered, and appear to be different from one another.
You could assume this might represent a correction or incomplete drawing, except that two other later works also show the saltire over a divided field:
The portoan chart from the civitate Majorica by Mathias Prunes, LOC
and again on the much later 1633 chart by Roaz, also from the Library Of Congress site:
One more entry showing this confusion on the part of the cartographers, this clip from Descaliers atlas of 1546:
Again we can see the artist started with a sketch showing a St Georges cross, then switched to the red saltire.
So, can we conclude anything from this survey of maps and charts from the time? Only that the cartographers consistently tried to represent Scotland with something different than what they used for England. Variations are shown over works by Reinel himself, and over works by varying cartographers.
A quote from The History of Cartography, Ch 19 discussing flags appearing on Portolan Charts seems to sum up the issue (emphasis mine):
Unfortunately, though a highly visible element of a chart, the boldly
colored flags tend to blur to unrecognizability in reproduction. A
further problem is the extent to which the development of distinctive
flags for particular places roughly parallels that of the charts
themselves. Standardization in the design of flags had not yet
taken place; hence it is not surprising that a great variety of forms
were adopted by the different early chartmakers. Designs would be
simplified or altered at will, and colors switched.
So, as shown by the variety of images above, we can not expect to see a consistent representation of what we would expect to see today as a 'flag of Scotland'. Each cartographer was gathering information from various sources, compiling the information they found relevant, and displaying this information in a shorthand format which would be understandable to the mariners of the time.
(I have skipped over a couple of aspects of this question I had originally thought to explore, such as the heraldry involved and the nature of portolan charts themselves. Each is interesting in its own rights, but could triple the length of this answer. If we have no expectation that the information displayed was held to any standard, it becomes less important.)
Here, however, are some of the more interesting sources I looked over:
Video Presentations- The Library of Congress had several full presentations which give a good understanding of the work involved in the creation of some of these charts and maps, and modern efforts to learn more from them:
and just for fun