How did ancient armies like the Romans, Greeks, Persians, Chinese, or even in later times, the Arabs and Mongols keep the route of supplies going to support their armies marching through the vast areas they covered? Did they build supply centers in the places they conquered and use caravans to send the soldiers supplies from there? What did they do when they failed to conquer a city where they laid siege for years? What would happen if there was not any food sources nearby?
In most cases, they would forage (or pillage) from neighboring areas. As in invading army, they had no qualms about taking whatever they wanted or needed. In situations where they were sending troops considerable distances or for prolonged periods of time, they had to develop other methods.
In some situations they would create relay stations, which would basically be a small camp where supplies could be delivered and then re-distributed to troops further along the supply line. If these stations were in areas where they were subject to attack, then they would be expanded into outposts or forts. In those cases, a contingent of troops would remain to protect the supplies that were stored up, and also possibly as armed escorts for supplies that were being passed along.
A couple of quotations from The Art of War (Sun Tzu, ca. 500 BC):
Bring war material with you from home, but forage on the enemy. Thus the army will have food enough for its needs.
Hence a wise general makes a point of foraging on the enemy. One cartload of the enemy's provisions is equivalent to twenty of one's own, and likewise a single picul of his provender is equivalent to twenty from one's own store.
The rule is, not to besiege walled cities if it can possibly be avoided.
We may take it then that an army without its baggage-train is lost; without provisions it is lost; without bases of supply it is lost.
Make forays in fertile country in order to supply your army with food.
My interpretation of this is that suppying armies from afar was considered possible but unreliable and inefficient, and thus one would rely on foraging as much as possible.
An important way to supply an army always was the transportation by water. Persian invasions of Greece were actually large scale combined ground force-fleet operations, as it is clearly seen from Herodotes description. Same applies to Hannibal's campain and to several operations of Alexander the Great.
In general, rivers and sea was one of the important ways of transportation, in many places the only way to transport large quantities of supplies.
Foraging is only part of the solution. Once you start pillaging the locals, supplies will dry up fast. Also, pillaging takes a lot of time and makes your men vulnerable. It's not really a viable option if you have an enemy army within striking range.
The truth is that foreign campaigns are always financed by money. Commanders would have a big chest full of money and this would be used to cover expenses. Things are no different now. The US army always tries to source supplies locally and pay for them. We only ship things from the USA when there is no local option.
In many armies soldiers were expected to feed themselves from out of their pay. So, the general would pay the soldiers, then the soldier would buy his food locally. For example, here is an extract from a British army manual published in 1816:
As a soldier, in certain situations, is not at liberty to quit his station, for the purpose even of supplying himself with the necessaries and conveniences of life, it is peculiarly incumbent on those set over him to furnish the means of supply, which he may be precluded from providing for himself, sufficient in their quantity, of a fit and proper quality, and at a reasonable rate. This is effected sometimes, in a limited extent, by public commissaries, or by individual contractors, but more generally, through the agency of licensed individuals, under the denomination of sutlers; who are permitted, under denned conditions, to bring victuals and liquors into the camp, fort, garrison, or barracks, as the case may be, to be retailed out by them to the soldiery at easy and moderate prices. The retailing of these Articles in the manner described, is well known and understood, under the term of sutling.