Pieter Geerkens answer is excellent and should be selected, but I want to add a few well-known facts which show that the "brutal statistic" is actually wrong by an order of magnitude.
Western Front 1940
More than 2,200,000 French soldiers fought on the Western Front in 1940. This alone dwarfs the total number of any kind of military personnel under Vichy-France, of which only a fraction fought for the Axis.
French Free Forces before Operation Torch
It is not so easy to determine the relative size of the Forces Françaises Libres compared to Vichy-France in the Levant and Madagascar campaigns. About 100,000 troops from Vichy-France seem to have fought these campaigns. In late 1942, there were 60,000 Forces Françaises Libres.
French Liberation Army
In 1943, after their fusion with the African Armies, the FFL became the French Liberation Army. 72,000 men fought in Tunisia, 100,000 in Italy and 267,000 in the liberation of France (mostly in Operation Dragoon). Again, these forces alone seem to outnumber the total number of military personnel of Vichy-France.
French Forces after the liberation
In september 1944, so after the liberation of Paris, the French Liberation Army numbered 550,000. In december 1944, 1,000,000. At the end of the war, 1,300,000.
Conclusion At the beginning and the end of WWII, French forces fighting with the Allies outnumbered the maximum number of French forces fighting with the Axis by almost an order of magnitude. Frenchmen who "bore the arms" for the Axis were at most 300,000; and among them perhaps 100,000 or 150,000 actually fought.
Why there is some truth in the assertion Depending on the context of the quote, though, I think there might be some truth in the assertion. Indeed, among the hundred of thousands who fought for the Forces Françaises Libres and the Armée de Libération, most were from France then colonial empire. So it is conceivably true that more metropolitan French bore arms for Vichy-France on the soil of metropolitan France in the late 1940/early 1944 period than metropolitan French bore arms for the Allies during the same period. And indeed, this fact is not easily discussed in the historiography of France. So with the qualifications "metropolitan French" and "from late 1940 to early 1944", the assertion might be correct. Without the qualifications, it verges on the insult to the French soldiers of 1940 and of the Armée Française de Libération.
Update
@jwentig correctly remarks that the answer did not account for the French volunteers and conscripts who fought in the Waffen-SS and other Axis units. In fact, a number seem hard to track down (leading credence to the accusation of uneasiness in the original quote). Nevertheless, I found this book which aims at a complete listing. It arrives at a total of 40,000 French soldiers for the Axis, so maybe 2% of the numbers who fought with the Allies. Among these, 18,000 to 22,000 fought voluntarily in the Waffen-SS according to Wikipedia, which is in fact rather low (compared for instance to the 40,000 from Belgium or Hungary and the 50,000 from the Netherlands, which all had a fraction of France's population). Another point that wasn't discussed in any answers so far, I think, is the case of forced conscriptions of former French citizens in the annexed part of Alsace and Moselle. There were about 130,000 so that even including them (which would seem quite historically unfair as well as inaccurate), the total number of Frenchmen arguably bearing arms for the Axis (total Vichy-France army plus French in German units plus forced conscriptions) hardly reaches 500,000. About one sixth of those who indisputably fought with the Allies.
Sources
Any reference about WWII, starting with Wikipedia.