The commonest form of underwear was the subligaculum, a basic loincloth worn by men and women.
Most people wore the subligaculum under other garments. For example,
men wore the garment under the tunica (shirt) or the toga, and women
wore it under the stola, a long gown. But others wore the subligaculum
alone. Common workers often labored wearing only a subligaculum, and
Roman gladiators, warriors who fought for entertainment in Rome,
usually fought wearing just a subligaculum.
Source: S. & T. Pendergast and S. Hermsen, Fashion, Costume and Culture, vol. 1: The Ancient World
As shown in the picture in your question,
Women also wear loincloths (and sometimes breast bands) and tunics.
Two-piece garments resembling a bikini are popular when taking
exercise at the baths.
Source: N. Bancroft Hunt (ed), Living in Ancient Rome
Underpants were not worn (these were a much later invention) and there is literary evidence (e.g. the poet Martial d.102/4 AD) that sometimes women didn't wear anything under a toga around the loins. Also,
There is no literary evidence stating or even implying that a Roman
woman wore underdrawers.
Source: Kelly Olson, 'Roman Underwear Revisited' (The Classical World, Vol. 96, No. 2 (Winter, 2003)
"Mosaic from a bedroom at the Villa Romana del Casale, outside Piazza Armerina, Sicily (4th c. CE)." Image & text source: History From Below
A band of soft leather, called a mamillare, was sometimes used to provide support under or over the breasts.
Wikipedia notes that,
Since the Romans regarded large breasts as comical, or characteristic
of aging or unattractive women, young girls wore breast bands
(fascia) secured tightly in the belief that doing so would prevent
overly large, sagging breasts.
One should also include the tunica interior as an undergarment as this was often worn under the stola (tunica exterior) by married women. The tunica interior
was sometimes supplied with sleeves, and as it reached only to the
knee did not require a belt to keep it from interfering with the free
use of the limbs. A soft sash-like band of leather (strophium),
however, was sometimes worn over it, close under the breasts, but
merely to support them
Tunica interior with strophium. Source: http://www.forumromanum.org/life/johnston_7.html
Clothes, including underwear, were usually made of either wool or linen, but the elite would have been able to afford cotton (imported from India) and silk (from the Far East). Evidence for leather lower undergarments has also been found. Lots of sewing in garments was rare as it was difficult to do (needles were made of bone and hard to use).
One final point: what was underwear for a wealthy Roman was often all that a slave wore (subligaculum) while a poor Roman (or a young girl) would probably wear subligaculum and a simple tunic with no stola.
Aquincum Museum (Budapest) (Description of Gravestone of Pattevilla and his family)