The Talmud says that Onias fled "to Alexandria in Egypt", and "there he built an altar":
הלך לאלכסנדריא של מצרים ובנה שם מזבח
We may take שם "there" as reference to "Alexandria", but it may also reference "Egypt". IMHO, the second version is corroborated by the citation of Isaiah 19:19 in Menahot 109b:
ביום ההוא יהיה מזבח לה' בתוך ארץ מצרים ומצבה אצל גבולה לה
19In that day there will be an altar to the Lord in the heart of Egypt, and a monument to the Lord at its border.
Not to mention the disputes about Isaiah 19:18 (!), regarding the "City of Destruction" (עיר ההרס), maybe the "City of the Sun" (עיר החרס) -> Heliopolis (as in the "New International Version") or the "City of the Lion" -> Leontopolis:
18In that day five cities in Egypt will speak the language of Canaan and swear allegiance to the Lord Almighty. One of them will be called the City of Destruction (alternative reading: City of the Sun / City of the Lion).
As far as I see, the archaeological evidence doesn't help much. In 1905, Flinders Petrie claimed to have discovered the location of the Temple of Onias near Leontopolis (Tel el-Yehudiyah, see the Jewish Encyclopedia on the Temple of Onias), but as far as I can see, this is hotly contested. Among the resources available online, see Taylor, "A Second Temple in Egypt: the Evidence for the Zadokite Temple of Onias", esp. the last chapter on the Location of the Temple.
Another possible location of the Temple of Onias is Tell Basta (Bubastis). See G. Hata, "Where is The Temple Site of Onias IV in Egypt?", in: "Flavius Josephus : Interpretation and History", Leiden 2011, pp. 177-192.
Adding to what @tohuwawohu wrote, Noach Chacham points out in his essay in Hebrew "The High Priesthood and Onias' Temple: The Historical Meaning of a Rabbinic Story", p. 452, n. that it's likely that the Talmud when writing "Alexandria" meant the whole of Egypt, as is the case in other places (the term "Alexandria of Egypt" means, according to him, "Egypt whose capital is Alexandria1). Some examples:
"Rabbi Yoḥanan ben Nuri stood on his feet and, contrary to this statement, said: And what shall the people of Babylonia, who have only sesame oil, do? And what shall the people of Medea, who have only nut oil, do? And what shall the people of Alexandria, who have only radish oil, do? And what shall the people of Cappadocia, who have neither this nor that but only naphtha, do? Rather, you have a prohibition only with regard to those substances with regard to which the Sages said: One may not light with them."
Babylon, Medea and Cappadocia were all countries. Therefore, it is likely that Alexandria was also synonymous with Egypt as a whole.
"Hillel the Elder would explain lay-language [of contracts as if it were the biblical text]. When the people of Alexandria would betroth wives, another would come and seize her from the street.2 [...]"
However, we have evidence of a similar case from a Jew from Heliopolis in P. Polit. Iud. 4 (Papyrus Politeuma Iudaei no. 4):
"Year 36. 19 Choiak. Regarding a marriage. We have given an order to issue a summons. To the archons from Philotas son of Philotas, a member of the πολἰτευμα. In the current year I betrothed Nikaia daughter of Lysimachos. The said father swore to give her to me along with the dowry laid down for her, and with which I was in agreement. So after not only vows(?) were exchanged between us but also the άπόκλυσις (?) according to the law (or the Law)..., we parted on those terms. Not long afterwards, Lysimachos without justification joined Nikaia to another man before having received from me the customary bill of divorce. Therefore, I request, if you think it right, that you give the order to write the Jews in the village to summon Lysimachos to appear before you, so that if the matter is as I write, his case may be decided according to the law (or the Law), and at the same time [he] may be forced ... to me ..." (translation taken from Menachem Kister's essay "From Philotas to Hillel: 'Betrothal' Contracts and their Violation", pp. 57-58.
"Said Rabbi Yishmael: "There was a case in which Queen Cleopatra the Alexandrian brought servant-girls who were liable of the death penalty..."
With that said, Abraham Wasserstein in his essay "Notes on the Temple of Onias at Leontopolis" had another suggestion, which is that external evidence shows that in the Byzantine period, Alexandria may have been called "Leontopolis", which may explain the interchangeability of the names. He himself thought that the rabbis really thought that the temple had been built in Alexandria, though it's possible that this could go either way (i.e., that rather than thinking Alexandria was where the temple was built, they simply referred to Leontopolis as "Alexandria" because Alexandria was also known as "Leontopolis").
A third option may be based on a more precise reading of the Talmudic text: The Talmud states that Onias first escaped to Alexandria, much like what Josephus says. Then it states that he built there an altar - not a whole temple. In other words, the text does not explicitly state that the House of Onias is located in Alexandria - perhaps just some kind of smaller precursor. Later, as Josephus said, he requested permission to build an entire temple and was permitted to do so in Leontopolis.
1 There's probably room to ponder whether there's a connection between the phrases אלכסנדריא של מצרים (Alexandria of Egypt) and other similar phrases such as גרממיא של אדום (Germamia of Edom) which is translated by some as "Germamia which is near Edom (Rome)" or דפני של אנטוכיא (Dafni of Antochia) which generally refers to a suburb of Antiochia which had a large Jewish population.
2 The term "seize her from the street" is not literal, i.e., it does not refer to rape.