Question:
Where did all the money go?
What are the causes that prevented us from being infinitely richer today than we actually are?
Short Answer
2% compounded growth over 2000 years is not a conservative number but rather an unreasonable and profoundly optimistic number. Average 2% growth for just the last 150 years was experienced by just 1 economy. That's spanning the most productive economic period in history, the industrial age. Before the last 200 years such growth was entirely unknown.
Still any fraction of 2% growth compounded over more than 100 years still leaves a lot of increased wealth, where is that Wealth?
It's in the factories and Universities and production facilities which enable the services produced for consumers. It's in the automobile, roads, medicine, other transportation(planes, trains, ships etc...). It's in the communications industry and any number of other industries which make all of our lives better.
As Adam Smith, the father of modern economics stated, the wealth of nations is not found in gold and silver, but in the services the nation produces. Our modern global economy is truly wealthier by thousands of orders of magnitude because not only do we produce more food and other commodities the ancients might recognize; we have entire industries of goods the ancients wouldn't recognize. All to produce even more goods and services to be sold, yielding even more wealth to be invested.
The wealth of Nations
The prevailing view (mercantilism) was that gold and silver was wealth, and that countries should boost exports and resist imports in order to maximize this metal wealth. Smith’s radical insight was that a nation’s wealth is really the stream of goods and services that it creates.
Smith also went on to say how a nation manages it's wealth, investing it in improved production is the only way a country can maintain it's wealth, not hoarding.
The wealth of Nations
Smith’s third theme is that a country’s future income depends upon this capital accumulation. The more that is invested in better productive processes, the more wealth will be created in the future.
Detailed Answer

The Facts of Economic Growth
Reliable increases in productivity and interest appreciation on the scale you describe are both factors only since the Industrial revolution (began mid 18th century in the UK, took hundreds of years to spread globally) and were not consistently associated with previous ages dating back to Caesar. So 2% compounded interest over 2,000 years is not a conservative estimate but a wild over estimate.
To isolate the Mediterranean in Europe and North Africa for example, its economy actually regressed significantly from the time of Caesar up through the high middle ages (1250 AD). That's 1200 year period where the economy saw no growth.

GDP of Western Europe
From: Historical Statistics of the World Economy.
That's not an uncommon pattern either. China and India had similar lost economic periods spanning centuries dating their high points: the Song Dynasty(1200AD) and the Mughal Empire (1700), respectively.
In fact since 1880 when the Industrial Revolution was in early stages in the United States to today, The United States Economy, arguable one of the most productive over that period only averaged 2% growth.
The Facts of Economic Growth
People, measure their wealth in coin or money or even commodities. When discussing wealth of nations or global wealth it's less confusing to take a page out of Adam Smith's book on the topic, "Wealth of Nations". Smith "the Father of Economics", said the true measure of a country's wealth is not measured in possessions(gold or currency etc), but in production capabilities. More specifically how the country in question invests the money they have to increase their production. By this metric your question comes more into focus. Significant improvement to production methods aren't consistent over time, for most ages over the last 2000 years such improvement are entirely absent.
Where is all the missing wealth? It's been invested in new production for the most part as economists have recommended for centuries (since Adam Smith in 1776 ) in order to improve production, yield more and better products and ultimately grow more wealth.
Sources: