The answer is threefold:
1) Transportation costs: agricultural societies had, since the beginning, been restricted by the amount of food one could produce locally. What 'freed' the British poor from having to work the land (please note I'm not arguing that this was in their favor) was the import of large amount of cheap food, as well as the materials to start producing fabrics in factories. In the Roman world, these would both have had to be produced locally, which puts severe strains on the amount you can sell. This also means selling stuff to far away places can only happen if the product is very valuable, especially over land. There are notable examples of food being transported long distances (see Rome for example), however these are exceptions and only possible due to its special political position. (I would also argue the Romans never imported more than one-half of the free grain, and thus even less of the total amount of food from Egypt.)*
2) Competition: there are some much better candidates for the industrial revolution to happen earlier, namely China, but also the large Muslim empires. What these all have in common with Rome is a large autocratic empire with little competition and strong lone rulers. In early modern Europe, if one ruler did not want to back you, you could go to another one (which is why Columbus could go to America, after the Portuguese king said no).
3) Different kinds of city: there is a notable difference between consumer and producer cities. Roman cities were the first kind: the nobles who had become wealthy with sustained (but essentially small) surpluses of their land spent much of these surpluses on craftsmen in the cities who used it to buy the food these nobles had brought to the city. The city did not produce any wealth itself. This became obvious when the Western Roman empire's cities steadily declined after the nobles started to live on their estates. (They were expected to spend their own money on the functioning of certain institutions in the city.)
Medieval and early modern cities were dependent on merchants and craftsmen, the latter creating products while the former sold them. The wealth was created in and by the city, making this a producer city, which is a lot more viable then the first kind.
*Perhaps someone with more knowledge about transportation costs in both periods, and where and how Britain imported its raw materials, could expand on this.
P.S. As a small aside, there may also have been some fundamental differences in the thought of the higher social classes between early modern western Europe and the Roman empire, because of their background. Roman elites were very reverent of their ancestors, maybe because their wealth was a consequence of their birth. The bourgeoisie was a self-made elite who was thus more interested and fond of the future.