Swimming in the sea was necessary for many in ancient Rome or Greece, but did they do it for recreation?
Yes, they did. Cumea for example, was a popular resort for the rich and famous. They fled the summer heat (and the malaria) of Rome to the beach.
I don't have any references, but some (very) rich Romans had heated seawater swimming pools constructed on their properties.
Going to the beach is not exactly what we do today. Going back a wee bit in time: my grandfather lived in the Dutch beach resort Scheveningen (period 1880-1900). In summer he rented out rooms to tourists. I've seen old photos in which people were dressed in bathing costumes (with long sleeves for both arms and legs) wearing hats went in the water. That was about it; they splashed each other with water. Some swam a bit. People went into the water in enclosed carriages. In the carriages they changed clothing and went into the water.
Sunning was frowned upon. The general believe was that a suntan was something bad and besides, only peasants had a suntan. You don't want to look like a peasant, do you?
Sunning on the beach, playing beach volleyball or other games, huge crowds on the beach in Roman times - forget about it. The Romans, most certainly the upper crust, didn't want to get suntan. They weren't racist in the modern meaning of the word, but definitely didn't want to look like sun burned peasants or slaves.
The beaches wouldn't have been crowded with holiday makers. People had to make a living. Only the wealthy could afford relaxing in a beach resort. Everybody else worked.
What did they exactly do? I have no idea. But I can make educated guesses: first of all, we're talking about the very rich. That's not a big group. Most very rich people got to be wealthy and important later in life. I can't imagine Roman senators or business tycoons racing each other to the beach. They probably strolled along the beach - which was fairly empty. Some fishermen at work likely, perhaps a few people offering snacks, refreshments, etc to them.
I'm sure their younger offspring would have had fun on the beach. But again, not exactly as we perceive beach holidays today.