I think your presumption that Japanese power through military force was not employed in Ezo during Edo is wrong. You may read about
Sakushin's revolt
and it's suppression in 1669.
UPDATE: 9-18-2017
Here's a much better source
Shakushain's War
describing the rebellion and going into great depth about the background leading up to and following the rebellion.
Early in the Edo era (~1620) the Shogunate gave the Matsumae clan the right to build a castle on Ezo and the most importantly the right to monopolize all trade there - The Ainu could only trade with Matsumae representatives and it was the Matsumae representatives who decided the prices. This had a great economic impact on the Ainu right from the beginning of the Edo Era.
Interestingly however, the second reference persuasively describes a much more complicated set of circumstances leading to the rebellion: it started out as a feud between rival Ainu clans, which eventually spun out into a rebellion when the Japanese appeared to be favoring one side.
The revolt failed militarily as arrows and spears were no match for the muskets the Japanese were armed with. However, it was a blockade on any trade with the Ainu which really forced the remaining communities to abandon any attempts at resistance. They had long depended on rice obtained through trade with Japan ( since early Edo via Matsumae-clan monopoly only), and without the rice they were starving.
My own conjecture: Originally the Ainu never or rarely ate rice at all. Their population remained bound by the quantity of wildlife they could hunt and the wild nuts and vegetables they could gather. They lived in equilibrium with their environment. Once they started trading captured wildlife with the Japanese for rice, they began to voraciously deplete their natural resources while at the same time thriving on the rice they received in return. This in turn led to feuds between rival Ainu clans as increasing population vied for diminishing natural resources to sell for rice. (The second resource does describe both the competition for resources and the vanishing resources, e.g., bears being unavailable for traditional ritual).
Once this effective domination of the Ainu is put in perspective, one part of the OP's question still remains: Why didn't mass colonization of Hokkaido occur until the Meiji era?
A possible answer to that: Look what happened to Great Britain with their colonists in the thirteen colonies.