In ancient Western times civilians were considered legitimate spoils of war for the victor. Sometime in Western Europe that seems to have changed. While Otto von Bismarck waged symbolic wars in Europe with very little casualties besides largely willing soldiers on the battlefields, colonial wars with indigenous people saw things like Indians scalping civilians, provoking hate in return for having broken the rules of war. In the Middle East today the ubiquitous use of suicide bombers, human shields, terror against civilians and the stateless warfare of some "rebel groups" indicates that they still do not make much of a difference between military and civilian targets or actors.
Where does this invention come from, that in war soldiers and civilians have different rights? For example that it is right to take a soldier as prisoner, but not a civilian.
Clarification: This is not a question about the formalism of official law, but about actual practice in warfare. Compare for example the European wars of the 1870s with today's wars in Syria and Palestine. I'm asking for the origin of the cultural valuation that a soldier has another status in war than a civilian has. von Bismarck didn't have the war aims to exterminate or enslave all Austrians and all the French, but that's the war aim of the Palestinians today with respect to their enemy the Israelis. And there's an ancient history behind that. When was that tradition broken to give rise to the special rights of civilians?