I found a free source of the German women's magazine "Der Bazar" which often contains patterns for sewing various items of clothing or decorative items.
The issue Der Bazar Nr. 37, 1. October 1865, 11. Jahrgang contains the following image of a "Corset for children of 2 - 4 years of age":
The instructions for sewing this corset (see link above) include (machine translation abbreviated for shortness):
The present corset comfortably fits the delicate forms of a child and at the same time gives the body the necessary support; the original is made of white fine drill and lined throughout with white shirting. At the sides it is provided with inserted elastic and at the back is arranged for lacing.
[...]
Following the indication of the pattern, stitch through fig. 24 and 26 along the dotted lines to accommodate the whalebone, which must be very fine and flexible, and provide the back with lacing holes according to the draft.
Figure 24 (half of the front) contains 5 whalebones right in the center front, figure 26 (half of the back) contains 3 whalebones right next to the lacing holes. In total, 16 "very fine and flexible" whalebones are supposed to go in there.
Another issue (Der Bazar Nr. 19, 11. Mai 1896, 42. Jahrgang on page 224) contains:
The translation of the image description is:
jacket, blouse and bodice for babies
Unfortunately the actual pattern paper where the detailed description would have been is lost. It's unclear whether there would have been whalebone, cording or any stiffening at all in the bodice.
I found more examples of a corset or stays-like item intended for babies or infants, but non of them are clearly dated. Please notice the reoccurring vertical stripes which indicate a posible stiffening of the garment using whalebone or cording.
Bottom row: corset for infants from 6 months to 3 years and from 6 months to 4 years.
I'm well aware that corsets were an indispensable item of clothing for women at that time - to shape the fashionable silhouette and support the bust, but until now I assumed that girls would start wearing corsets with the onset of puberty or a few years before that.
Why would parents in the 19th century make their infants wear stiffened bodices? I'm particularily interested in the fact that you were supposed to add whalebone to the first example. If it were just an item to keep the child warm or to attach the skirts or other clothing items, it would suffice to sew the bodices out of sturdy fabric.
So I assume they had some practical reason or false medical belief for stiffened baby bodices, but the magazine doesn't mention it with even a single word. The magazine also mentions that baby boys would have worn clothes identical to baby girls, so I'm not sure if this type of bodice was exclusively for girls or not.