Takeovers of this kind often involve "trigger events." There was "not quite" a trigger event for the Spanish takeover of Portugal during Ferdinand of Aragon's time, and there was one late in the 16th century, when Portugal was taken over.
In 1580, the direct line to the throne of Portugal died out (the trigger event). There were a number of "cousins" to the royal family, one of which was Philip II of Spain. It was in this "role" that he took over Portugal, not on behalf of Spain, but under a "personal union," whereby he was king of Spain and Portugal.
In Ferdinand of Aragon's time, on the other hand, the Portuguese royal family was intact. Also, Ferdinand, as king of Aragon, was actually more interested in events to his east, that is, regarding the balance of power in Italy, where both France and Spain had interests. Portugal was a poorer, less populous country by comparison, meaning that it was not as high on the list of Ferdinand's priorities. His interest in Grenada was partly because of his wife Isabella, partly because unlike Portugal, it was occupied by Moors (and Jews), rather than fellow Catholics.
Leon and Castile had been united much earlier in 1301, not in Ferdinand's time, although his marriage to Isabella united Aragon with these two territories. And he did have a part in bringing the Spanish-speaking (but not French-speaking) part of Navarre into Spain.
The "not quite" in the first paragraph refers to the fact that a merger of Spain and Portugal might have happened if things had worked out differently. First, the original heir to the Spanish throne, Juan of Asturias, died early, making his sister Isabella of Aragon the heiress. Second, Ferdinand and Isabella initially offered King Manuel of Portugal the hand of Isabella's younger sister Maria in marriage (whom Manuel later married), but Manuel insisted on Isabella. Third, Isabella consented to marry Manuel if he would agree to expel the Jews for Portugal (he did). This prospect could have brought about the union of Portugal and Spain, except that Isabella and her son (the heir to the Portuguese throne) both died young. The middle sister, Juana, then inherited the Spanish throne ahead of Queen Maria of Portugal, and through Juana, Charles V
Put another way, it was a "near miss," based on "luck of the draw." But since Manuel of Portugal had agreed to align his policy towards Jews with that of Spain, Ferdinand and Isabella weren't about to try to take Portugal by force.