Why didn't Colonists in New England import massive numbers of Slaves?
One important factor was the warmer weather in most parts of the south. That led to two important effects: 1) the cultivability of cash crops such as cotton and sugar, and 2) relatively short winters. The cash crops were important because that's how slave labor was transformed into profits. Long winters were a liability, because slaves had to be fed while they were not working, and many died in cold weather.
In the early days (17th century), the North had some slaves because these advantages were not so pronounced. But as time went on, "specialization" insured that slaves would be profitable in some parts of the country and not others.
In another post, I noted that pro-slavery sentiment was predominant in most "hot" areas of the United States (other than California), and anti-slavery sentiment where it was "cold." The dividing point between hot and cold that I used was Richmond, VA, at the edge of "Confederate" territory, and a place where slavery was barely profitable. North of there (and at higher altitude), there would be little economic rationale for slavery, and south from that point, there would be increasing economic incentives for slavery.
Because keeping a slave costs money 24/7 whether they're working or not: it's like owning a limousine that you only use a few times a year rather than hiring one on an as-needed basis. Hired hands only have to be paid when they're actually working.
As the North was becoming more and more industrial, it became less and less economically viable to own workers (for whose upkeep and medical bills the owner was responsible). Hired hands were easily found and cheap, and when they got sick, injured, or killed, it was their problem, not the employer's.
The South continued as mostly an agrarian society, with the exception of the areas where industrialization was inevitable, such as the harbor: it did not make economic sense to have a $1000 slave perform hard and oftentimes dangerous tasks that an immigrant worker was willing to tackle for 10 cents a day.
Of course there's the whole growth of manufacturing in the North as others have stated. Moreover, the farther south you go, the more labor intensive the act of yielding the commodities (rice, indigo, sugar, tobacco) are. The South and the West Indies kept slaves because indentured servants (who were mostly white immigrants from England) were not immune to the diseases (malaria), were not accustomed to the southern/tropical climate, and were quite frankly not as good. Cultivation of rice was an incredibly dangerous task for a white man from England as he had no immunity to malaria. Therefore the Africans and natives were made to do it. The crops grown in New England did not require such arduous work and thus were manageable for a midsize family. You have to remember the residents of New England had on average 8 kids who lived to adulthood while southern families typically lost half their children. Thus more workers for the farm were available for a family in New England as compared to the South.
The answer is mainly that the South is where cotton grows , also this and this. Cotton was a very valuable cash crop, contributing about 2/3 of U.S. export value by 1840. It's also labor intensive. Cotton production really took off after short staple cotton became profitable (due to the invention of the cotton gin). The cotton belt is essentially the confederacy.
Here is a nice plot of cotton production vs number of slaves.
There was slavery in New England. That's who built all the walls. The full name of Rhode Island is Rhode Island and Providence PLANTATIONS. If you visit, you will see stone walls everywhere, all built by slaves. I used to work in Rhode Island and every day at lunch I would go walking in the woods past those endless lengths of 4-foot high stone walls, each stone set by a slave. In 1754 there were 4500 slaves in Massachusetts (see Moore, George H. "Notes on the history of slavery in Massachusetts"). Concerning these fences note the following letter:
"But let me add this by way of commendation of the Narragansett and Warwick Indians who inhabit this jurisdiction, that they are an active, laborious, and ingenious people; which is demonstrated in their labors they do for the English; of whom more are employed especially in the making of Stone Fences."
-- Report of Captain Daniel Gookin to the Massachusetts General Court, 1662.
At the time of the Gookin report the Indians did this work for pay, or in debt slavery, euphemistically called "indentured servitude", but after the terrible war in 1675, all the surviving Narragansetts were enslaved and put on what virtually amounted to chain gangs (although they were not actually chained). Slavery in New England was not ended until after the Revolutionary War and the cause of this had nothing to do with economics but was purely political. Even before the war many people in New England opposed slavery on religious grounds, especially the Quakers, the Presbyterians and Congregationalists. All of these were far more numerous in north. However, what really turned the tide was the Revolutionary War. In New England, New York and Pennsylvannia, slavery was strongly associated with colonialism and after the war the mood turned from tolerance to slavery to intolerance against it, mainly because it was seen as a vestige of colonialism. In the south, the reverse happened. There, the laws England was passing against slavery offended the plantation owners of the south as an attack on their way of life. So, they saw the Revolution as way to protect and extend slavery from England's encroachments. Thus, after the war a bipolar politics arose, the north rejecting slavery and the south embracing it.