Armies go around castles all the time, but what usually happens is that the castle is placed under siege. This is done at least with the intention of keeping the defenders in, and hopefully taking the castle via attrition, bombardment, sapping or treachery.
The need to siege the castle is important; if you ignore the castle and march on, this leaves the rear of your army - and worse, your supply line - highly vulnerable to attack by the defenders sallying forth. There are few examples of this happening because ignoring castles without besieging them is a huge tactical blunder.
So why have castles if the attacker can just siege them? Because sieges are very expensive to do. Manning a perimeter is obviously going to take more manpower than manning an isolated strongpoint - the Art of War quotes a tenfold requirement in men, which although not a hard rule, gives you some idea of the scope of the problem. Also it is very likely that this siege is being done in enemy territory, giving the defenders an interior lines advantage. Also, castles tend to be well-stocked with supplies, so that even without resupply, they can often last months, sometimes even years. Take a look at this list of sieges; there are many examples of ones lasting more than one year. With the defenders able to maintain such a huge logistical advantage for a very long time, how can you not see how awesome castles are?
This is also why castles are sometimes assaulted, despite the huge tactical disadvantage: the attackers simply cannot afford to tie up so much of their forces in a siege, or continue supplying them through enemy territory for so long. Ignoring the defenders risks the rest of the army coming under attack at their weakest positions.
Hannibal's invasion of Italy during the Second Punic War shows some of these dynamics at play:
- Despite invading from the north, Hannibal bypassed Rome to take most of southern Italy
- He did not really besiege Rome (but being one of the best generals in history, he had good reasons for this). This enabled Fabius to undertake his famous eponymous strategy: unable to best the Carthaginians in pitched battle, he used small, mobile forces to endlessly harass Hannibal in the rear.
- In the end, although numerically matching the Romans and outmatching them in most battles, Hannibal could not overcome the tremendous home front advantage the Romans had.