I'm from the states, so I'm not so sharp on the history of various European monarchs. Having been told as a child that my ancestors emigrated from Scotland, I've always had an interest in the country. So to sum up, what led up to, and eventually caused the merger of the Kingdom of Scotland and the Kingdom of England?
The first attempt at unification was sparked by succession disputes, after Margaret of Scotland died in 1290. This lead to a series of conflicts, spanning from 1296 to 1357, known today as the Wars of Scottish Independence. Scotland retained its status as an independent nation after the end of the wars.
The claim of Mary, Queen of Scots to the English throne ended up with her losing her head (literally), however when Elizabeth I of England died, Mary's son, James VI of Scotland was proclaimed King of England and Ireland (as James I), leading to the union of the crowns (1603). Scotland still remained a sovereign state, sharing a monarch with England.
The Glorious Revolution (1688) created fertile ground for union debates and unification eventually happened in 1707, with the Acts of Union. The years prior to the Acts of Union, Scotland had faced a famine, the "ill years", and was in a poor financial state after the failure of the Darien scheme, a series of expeditions to establish a colony on the Isthmus of Panama.
To answer the question here are some extracts from specified sources:
Scottish Historical Documents, by Professor Gordon Donaldson (p. 266, ISBN 1-897784-41-4):
England retaliated in 1705 with the Alien Act, which declared that, until Scotland accepted the Hanoverian succession, all Scots would be treated as aliens in England and the import of cattle, sheep, coal and linen from Scotland into England would not be allowed; this measure stimulated the Scots into appointing commissioners to treat for union.
The Scottish Insurrection of 1820, by Peter Berresford Ellis and Seumas Mac A'Ghobhainn (p. 42, ISBN 0 85976 519 9):
Professor Lodge, an English historian and pro-Unionist, admits...that
"They [the English Government] had commercial inducements to offer and the ruin of Scottish agriculture to threaten, and by a judicious combination of bribes and menaces, they succeeded in bringing about the negotiations of 1706."'
The Scottish Nation 1700-2000, by T.M. Devine (p. 16, ISBN 0-713-99351-0):
Furthermore, there were grounds for believing that England might impose a military solution in order to safeguard her northern borders if the union project failed. Godolphin had made veiled threats to this effect and, as has been seen, troops had been stationed in the north of England and reinforcements also sent to northern Ireland.
Scotland: The Shaping of a Nation, by Gordon Donaldson (p. 57, ISBN 0 7153 6904 0):
England was not going to permit a disruption of the existing union, and the scanty and ill-trained Scottish regiments could not have resisted Marlborough's veterans.
A simple answer is because of the lines of inheritance.
In the 16th century, there was a rivalry between two queens, Mary Queen of Scots, and Queen Elizabeth I, who each wanted the throne of the other, and who were cousins.
Mary Queen of Scots fled to England after being overthrown in her own country, and was imprisoned for nearly 20 years, before being behead by Elizabeth I for plotting against her (Mary's letters to supporters were intercepted by English spies).
But Mary had married Lord Darnley, and had a son, James VI of Scotland, while Elizabeth I, the "Virgin Queen" had no children. As such, Mary's son, James VI of Scotland was next in line for the throne of England, and ruled as King James I of that country. That's because Mary's grandmother was the older sister of Henry VIII, Elizabeth I's father. (None of Henry VIII's other siblings or his children had children of their own.)
The ascension of Scotland's King James I to the throne England started the unification process, which was completed over a century later when the Scots claimants to the throne of Scotland were defeated in battle.