I know the Portuguese and the Spanish had slaves working for them on the overseas colonies, but what about on their own soil in Europe?
They did, and certainly had quite a few, as an article cited in Wikipedia says:
In mid-sixteenth-century Seville 7.4 percent of censused inhabitants were slaves and [...] between 1682 and 1729 the slave population of Cádiz was extremely large, making up perhaps as much as 15 percent of the total urban population. In other cities, such as Málaga, Granada, Las Palmas, Huelva, and Palos de la Frontera, as many as one in ten residents may have been slaves. [...] Historians who studied slavery in Spain thus concluded that Renaissance and perhaps even early-modern Spain may have had the largest African population in Europe.
Those were mostly African slaves – before the 15th century there were also slaves, Muslim (from Southern Spain and Africa) in the Christian kingdoms and Christian (from the rest of Spain and Eastern Europe) in Muslim-controlled Spain. See Wikipedia article and its sources.
In 1761, the Marquis of Pombal abolished slavery in continental Portugal, which suggests that, indeed, there were slaves in continental Portugal before then.
Here's a painting depicting 16th-century Lisbon. Pay attention to the lower right corner.