Defense of German heritage against Romans
The biggest reason for how the lands east of the Rhine retained their German identity (unlike the Gauls of modern day France who lost their Celtic identity) is the Battle of Teutoburg Forest where the Germans won a decisive victory against Roman invaders. After this battle, the Romans never seriously attempted to conquer German lands in Magna Germania even though sporadic raids and skirmishes continued. This also ensured that German lands would remain German for eternity (or at least till now).
Latin influence on Gaul
The original inhabitants of France were Celtic people who were referred to as Gauls and Belgae by the Romans. Eventually Gaul was conquered by Julius Caesar and soon a dialect of Italiano-Celtic origin began developing which can be referred to as Vulgar Latin.
This Vulgar Latin is the core of the modern French language but Gaulish still left its impacts on modern French - for example, the loss of unstressed syllables, the vowel system etc.
Germanic Tribes
As you have noted, the Franks and Alemanni aka Suebi or Swabians were both Germanic tribes and Romans were not wrong in calling them both Germans even though they both spoke different dialects of German.
However they were not the only German tribes; there were dozens of them, many of whom spoke their own dialects of Proto-German. Chief among them were of course Franks, Alemanni, Angles (who settled in England and gave it their name), Dani (modern day Danes), Goths (From modern day Sweden, Swedes are considered by some as descendants of Goths. Divided into Visigoths who conquered Hispania and Ostrogoths who conquered Italia), Langobardes (later known as Lombards and gave Lombardy (Italy) their name), Saxones (Sailed to England and laid foundations of Anglo-Saxon states), Vandals (Who conquered North Africa).
Francia or Carolingian Empire
There once used to be a confederation of German Tribes which was called Frankland or Francia or Regnum Francorum or later Carolingian Empire (Under Karling Dynasty) which was ruled by a German Tribe named the Franks. It existed from 481 AD to 843 AD.
Eventually Gaul was conquered by German Tribes. The Franks settled down in Northern France while the Alemanni settled down in the Rhine region at the border of Modern France and Germany. The Alemanni were also later conquered by the Franks under Clovis I. Unlike the Franks who were converting to Catholicism and adopting local culture gradually, The Alemanni were however very conscious about their roots and remained pagan until the 7th century.
The Franks did give their own influences to the native language but linguists today believe only 500 French words have Frankish roots.
Division of Frankish Empire
Anyhow, after the death of Louis the Pious, the Carolingian Empire got embroiled in a Civil war between his sons which resulted in the division of the Empire.
It was only ended by the Treaty of Verdun, according to which Charles the Bald got West Francia which went on to become France and Louis the German got East Francia which went on to become Germany.
It must be noted that Latin Speakers were present only in West Francia. East Francia however remained the traditional heartland of the German People.
This division formed the line between East and West forever as from the 10th century, East Francia was known as the Kingdom of Germany which was further reinforced when the Salian dynasty took over the Holy Roman Empire.
Conclusion
In the end however, the Franks assimilated into the local Latin-speaking population and from Old Frankish and Italo-Gaulic, the French language was born.
East Francia, which contained the territories of the Alemmani, did not have any such political aspects to consider and therefore retained their dialects and language. Alemannian German is still spoken in various parts of Germany and also abroad e.g. in Switzerland etc.
The French language however remains distinct among the Romance languages which is due to Germanic influence from the Franks and Normans. I read somewhere that:
French is what happens when Germans learn Latin.
A good similar example for the case of the Carolingian dynasty would be the Yuan dynasty of China. The Yuans were Mongols who invaded China, but in the end, they were the ones who adopted the native Chinese culture and language rather than the other way around. Similarly the Ilkhanate adopted Turco-Persian identity. Their cousins in Mongolia however retained their Mongol identity.