There wasn't enough work. This was originally a problem created by slave labor: wealthy nobles owned slaves and large estates, and small farmers could not compete with them. They had to sell their land or work for a wealthy patrician, and a lot of them eventually gave up and moved to the city. This was a constant source of civil unrest in Rome and quite a lot of Roman politics was occupied with this problem in one way or another. One of the solutions was that the state guaranteed a basic income in the sort of the cura annonae. This meant that Roman citizens were guaranteed grain, and later even other supplies, for a cheaper price and eventually for free. Roman emperors were very aware that they had to guarantee the annona if they wanted to stay in power, because otherwise unrest would break out in Rome. That is why the administration of Egypt, as the main supplier of grain, and the organization of the annona were two of the most critical posts in Imperial bureaucracy. At the time of Augustus, there were around 200,000 recipients of this in Rome, although not all of them were poor or unemployed.
As there was not enough work and quite a lot of it was done by slaves anyway, there were two other options for the Romans: join the army or become client of a wealthy noble. The Roman citizens of the "middle class" were apparently often occupied with being the entourage of a patron. The clients were protected and often financed by the patron, and in turn had to swear fealty to him. Their work day consisted of going to the villa of the patron in the morning and wait until they were given a task: make a delivery, pose as bodyguard, go to the assembly and vote for a proposal, and so on.
We can say that, similar to Classical Athens, there was quite a large percentage of the population in Rome which was not occupied with any productional work. There is however a lot of debate about the extent of this in the Roman empire. Roman economy is still quite ill understood, because it is very hard to quantify anything – Roman demography, agricultural production, degree of urbanization and so an are all very hard to grasp in numbers. Urbanization seemed to reach a very high level in general, in particular in Italy and the East, without the need for redistribution on the same scale as in Rome. Factors that allowed this were the intake of slave labor and other goods by conquest, a gain of farmable land by expansion, and the possibility of long-distance trade. One of the many, many theories for the collapse of the Roman Empire is that the Roman economic system in this form was not sustainable: the Roman Empire collapsed when it could no longer gain the necessary surplus production for the redistribution of supplies to the Roman population and army, but that's another question entirely.
Some sources:
M. Weber, Economy and Society, 1925 (deals briefly with Greek/Roman cities, but had a large impact)
M. Rostovtzeff, The Social and Economic History of the Roman Empire, 1957 (bit outdated in some regards, but the best general overview)
W. Scheidel – I. Morris, The Cambridge Economic History of the Greco-Roman World, 2007 (gives an overview over the current problems and discussion on this topic, you can find further literature here)