Armies go around castles all the time, but what usually happens is that the castle is placed under siege. This is done at least with the intention of keeping the defenders in, and hopefully taking the castle via attrition, bombardment, sapping or treachery.
The need to siege the castle is important; if you ignore the castle and march on, this leaves the rear of your army - and worse, your supply line - highly vulnerable to attack by the defenders sallying forth. There are few examples of this happening because ignoring castles without besieging them is a huge tactical blunder.
So why have castles if the attacker can just siege them? Because sieges are very expensive to do. Manning a perimeter is obviously going to take more manpower than manning an isolated strongpoint - the Art of War quotes a tenfold requirement in men, which although not a hard rule, gives you some idea of the scope of the problem. Also it is very likely that this siege is being done in enemy territory, giving the defenders an interior lines advantage. Also, castles tend to be well-stocked with supplies, so that even without resupply, they can often last months, sometimes even years. Take a look at this list of sieges; there are many examples of ones lasting more than one year. With the defenders able to maintain such a huge logistical advantage for a very long time, how can you not see how awesome castles are?
This is also why castles are sometimes assaulted, despite the huge tactical disadvantage: the attackers simply cannot afford to tie up so much of their forces in a siege, or continue supplying them through enemy territory for so long. Ignoring the defenders risks the rest of the army coming under attack at their weakest positions.