When reading history, it seems that the sacking of cities (rape and/or killing of the inhabitants and pillaging) after a successful siege was a common and accepted practice...
Why was it acceptable?
To the victor the spoils. In other words, the victorious party gets to decide what is "acceptable".
That's the long and short of it. In an era before "international laws of war" there was nothing that wasn't "acceptable" and the only thing that mattered was winning.
Add a siege, your troops being away from home for years, living in horrible conditions camped outside some city where the defenders live in apparent comparative luxury (at the very least they have a solid roof over their heads rather than living in drafty, wet, uncomfortable army tents).
Now you win the city. Your troops are angry, many of them are mercenaries you have to keep happy enough to not desert you and turn on your own men. There's an ample supply of women, something your men haven't had a lot of lately, not since leaving home. And there's all those defending men who've caused your men so much hardship.
Try controlling tempers, you're not going to make yourself very popular if you tell them a little raping, murder, and pillaging isn't allowed because they have to "consider the sensibilities of the civilians".
So the city is ransacked, the women are raped and probably enslaved, the men killed or enslaved (and set to building the wagons and ships that will get your own men home). A fire breaks out somewhere, the city is burning. But nobody is going to put himself in danger to put out the fire, so it's destroyed.
Now, if you are a calculating invader, you can put that process to good use. Send a few of those survivors, maybe with their hands cut off, to other cities in the area to tell what happens to those who dare defy your armies, and they'll hopefully surrender, reducing the cost of your campaign significantly.
That's what happened a lot, and it worked.
It wasn't until the idea took root again (it had existed for a while in various civilisations, e.g. the ancient Greeks) that warfare is not to involve the civilian population if this could be avoided, that was the way to wage war.
And it of course resurfaced in WW2 with the bombings of Warsaw, Rotterdam, Coventry, London, and the retaliatory bombings of German cities in reprisal.
Of course those were in part attempts to knock out enemy military targets (and that includes factories producing weapons), but a lot of it was directly aimed at demoralising the civilian population and hopefully get them to pressure their leaders into surrendering.
I agree with @jwenting's answer, although would put more emphasis on the "put that process to good use" paragraph: for a few medieval invading armies the impending suffering of civilians was the main threat in pre-war negotiations.
It was the modus operandi for most Mongols' strikes: they would approach a city, demand it to surrender w/out a fight and pay tribute, or face complete destruction. The threat of what would we now call "total war" was enough for most cities to lay their arms before a single arrow was fired. The cities who wouldn't do that were suffer the worst pillage and intentional destruction by fire.
Think of a school bully. What's more logical to him if somebody refuses to surrender his lunch money: bit him up or let him be because he's "just a civilian"?
Forcing an army to besiege a fortified town is a hostile act - you are consuming the campaign season and the army outside risks disease ravaging its ranks. In ancient times a city that surrendered on first approach usually was spared a sack as a quid pro quo - although it might have to pay a cash or goods payment to the army.
By the late 1600s and 1700s in Europe where the "rules of war" were formalized to a great extent as a reaction to the 30 Years War, there was a ritual set of steps
1) Army approaches, if the city doesn't surrender... 2) Army must put in investment trenches and use artillery force a breach in the wall. 3) If the Garrison surrenders with the "honors of war" it goes free and the city is taken. If the town has a citadel, the Garrison can retire to it, giving up the town and a new mini-seige of the citadel could result with part of the army. 4) If the Garrison decides to fight on, and the assault takes the town, a sack is allowed and the Garrison can be massacred.
So basically a defender could avoid a sack by giving up earlier and saving the attacking army the time and losses continued resistance entailed.
To sack a city is a concept imported from the latin languages [esp. French, mettre à sac] in the mid-16th Century to mean, essentially, putting a town or community ‘in the sack’, in the sense of taking its goods. Thus, in the decade after 1577, the chronicler Holinshed is found describing the Earl of March entering an English town, ‘sluing’ [slaying] the English inhabitants, putting their goods to the sack, and burning the town. Certainly, the term was used more loosely to describe the addition of murder to looting, but it appears to be wrong. Macaulay in his History of England, vol I, page 614, is found still distinguishing between ‘sack’ and ‘massacre’.
So, first one has to distinguish between different degrees of unpleasantness that crouch under the umbrella of the word ‘sack’. The looting side of the activity is, in modern legal parlance, called ‘pillage’. In describing the murder of the inhabitants of a town successfully besieged, one can sometimes see a distinction between the treatment of the military garrison and its military leaders, civil leaders, priests, and the ordinary citizenry.
As to behaviour on entry into a town, it all depends when you are talking about and who we are dealing with. In ancient Israel, one can distinguish between two types of war, obligatory and optional [mirzva and reshat]. In the former case, the Israelites seem to have regarded obligatory war – as a result of the order of their God - as necessitating massacre of conquered towns. Shabtai Rosenne, The Influence of Judaism on the Development of lnternational Law [ 5Netherlands ILR 119, 139 (1958)] interprets the distinction between the two forms of war as a legal one, resulting in different consequences for the victims. The legal consequences may be seen in the divine order contained in the notorious passage in the Book of Deuteronomy 20: 16-18-where the Israelites are commanded to attack the Hittites, Amorites, Canaanites, Per'izzites, Hivites and Jeb’usites, and ‘leave nothing alive that breathes, but you shall utterly destroy them’. Interestingly, there is a reason given for the order too massacre everyone – to avoid the Israelites from contamination with the theological practices of the conquered. There are other equally bloodthirsty commands from this extraterrestrial source in the same Book of Deuteronomy, q.v. Optional wars seem not to have the same results and citizens could be preserved by prior warning of the attackers followed by compliance. Failure to surrender led to the male citizens being killed, but the women and children were to be allowed to live, sometimes in captivity, sometimes incorporated into Jewish society. This legal distinction seems to be confined to the Old Testament era of Jewish history, although a case might be made for its apparent resurrection in the 1948 war between Israel and the Arabs, and the consequent massacres of communities [such as Deir Yassin] ‘pour decourager les palestiniens’.
In the classical world, massacres of besieged citizenry abound in the histories. The most horrifying because of its cold-blooded formulation and the preceding discussion, is in Thucydides’ description of the Melian Dialogue in 416 BC. The small island of Melos had been associated with Sparta but was trying to remain neutral in the war between Sparta and Athens. Athens heard the Melians’ plea to be allowed to remain aloof from the war but simply replied that they were strong and Melos was weak, that Melos was historically an ally of Sparta and, in any case, allowing Melos to remain neutral would cause the Athenians problems with the reliability of their allies. So, Athens invaded the resisting Melos and killed all males capable of being soldiers and consigned the rest of the citizenry to slavery. Again, pour decourager les autres. Generally, if a city agreed to surrender, it could expect to save its citizenry from death or slavery. An exception is the capitulation of Plataea to the Spartans when the citizenry was thrown into slavery. Property of the inhabitants was generally at the disposal of the conqueror.
The Romans developed quite early a system of rules that demanded restraint when the enemy city was ‘civilised’ according to Roman conceptions. Polybius notes that, when a city was defended against the Romans, then the Romans could kill the males after entering the city. Killing civilians after an enemy town that had complied with Roman demands before the first Roman battering ram touched the outer wall was forbidden by the Roman law of war. There are examples of this happening from the third century BC. An instance of failure to abide by this law by Marius when he entered the town of Capsa in 107 BC, led to his widespread condemnation in Rome. There are half a dozen or so examples of Roman failures to abide by their own rules in the treatment of conquered towns.
To summarize a story that could last for chapters, the rules of warfare throughout the imperial Roman period and the Middle Ages tended to follow the Roman model with plenty of exceptions. Demand for surrender followed by surrender – lives saved, property at the command of the invader. Demand followed by resistance – killing of garrison, civil leaders or male civilians at the choice of the invader. Killing tended to be used when either the defiance was enthusiastic or the example to other target towns was needed.
An often-cited example of the application of the rule is the 1474 trial of Peter Von Hagenbach, who was accused of committing `crimes against the laws of nature and God' in taking the town of Breisach. He was tried and condemned by 26 judges of the Holy Roman Empire and executed.
The 30 Years war and the wars of religion in France led to a series of very nasty massacres of towns, especially when religion was invoked. At one siege of a town containing heretics, it is reported that when a monk was asked to advise if the Catholics should be spared, because it was feared that the heretics would pretend to be orthodox Catholic, he advised ‘Caedite eos. Novit enim Dominus qui sunt eius.’ [Kill them all. God knows his own.’]
Subsequently, Grotius wrote a book, On War and Peace, intended to enshrine the rules of warfare and the protection of civilians. It was followed by other similar works and from then on, one could say that the rules of warfare contained prohibitions against massacres of civilians. Of course, the rules were and are consistently breached until this day, but a rule that is breached may nevertheless be a rule if people know what they should and should not do.
Fast forward to 1863, and, in the USA, the Lieber Code is adopted by Lincoln to govern land warfare. It contains prohibitions against harming unarmed civilians in most cases – except when military necessity requires such harm as a by-product. This code continued to set the base standard for all future agreements restricting warfare. It has been modified by convention and case law – for example, starvation of civilians was contemplated in the mid-19th Century as a legitimate method of warfare but that is now prohibited. The modern law of armed conflict applicable to sieges is enshrined in a set of more general prohibitions against harming unarmed civilians [but that is another story].