Kerr did not need to remove the Prime Minister to be advised to conduct an election. Gough Whitlam was going to see Kerr to advise him to call an election, after lunch and a few beers. Before the tinnies were sunk, Malcolm Fraser has sought to speak to Kerr, told Kerr that he commanded the majority of the House of Representatives and should be appointed Prime Minister, and by the way call an election of the sort that Whitlam didn't want to call. (There are multiple kinds of Australian federal election, which kind was to be called was a matter of party politics.) Malcolm Fraser then passed a supply bill (unlike the United States, a loss of supply is a major issue in Australian westminster). Kerr had been heavily gotten drunk on sherries on just this point by friends who worked for Captive Nations in Europe and the like. Transparently so. Kerr was an easily manipulated fool, a self-important partisan without convictions, an arsehole who adored flattery: exactly not the kind of person you really want to hold unennumerated reserved powers designed to get the state out of a major political hole when normal politics have collapsed.
In Australian westminster Governors General and Governors have power without authority. They may act, but may not choose to act.
For background, see this answer: Why did The Crown give assent to Apartheid legislation during the 1950s? citing HV Evatt 1936, The King and His Dominion Governors, 298-300ff. In summary: The Crown is represented by either the Governor General, or the Governor, or the person of the Monarch. The person of the Monarch cannot advise a Governor to act, nor can a Governor seek advice from the person of the Monarch. Each can be the Crown, but they cannot be the crown in mutual conversance. The person of the Monarch may only act for the Crown in person. While the Crown holds reserve powers, these powers are held with the understanding that the Crown's purpose is to act only on the advice of the leader of the House of Commons, unless a separate action by the Crown is absolutely necessary to maintain responsible government. The reason this is the case is the Commonwealth, and the trial and execution of the King. Should the Crown not establish responsible government, the past action of terrorising influence for the Crown is that the lower house in question will seize and execute the person of the Monarch, or their household, or their formal representatives; or, that should the lower house lose control over the mobility and plebean mob, that the working class's institutions could do likewise. The Crown fears that the person of the Monarch is liable for failure to act only on the advice of responsible government or to ensure the continuity of responsible government. The mild case, which I previously put in the referenced answer, is that a failure of the Crown of this magnitude will, at a minimum, threaten to return a lower house that will lawfully abolish the Crown's constitutional role.
The Governor General has a reserved power to call for an election without acting on the advice of the leader who commands the confidence of the lower house. The exercise of this power lies so far outside of the conventions of responsible government that it has been used twice in Australia: against Lang by the UK banks, and against Whitlam by the CIA.[*1] The Governor General's reserved powers are enormous, and their employment is an enormity to the point of monstrosity. A Governor General could prorogue parliament and govern with an executive council drawn from non-parliamentarians of the Governor General's choice, implementing acts of the executive council through fiat including comandeering private property and the like. To do so in a circumstance where Sydney, Melbourne and Canberra hadn't been hit by multiple independent nuclear warheads, would result in the Trade Union movement and half the business community going atomic.
Westminster systems are built out of day to day legalism representing a constitutional balance based on past terrors. In particular westminster systems usually leave a hellish volume of reserved powers that would be unthinkable in a French or United States living constitution. In Australia it is far better to get drunk at the Melbourne Cup, speculate on property, and wait until you're given an Order of Australia; than it is to launch a coup d'etat against parliament and hope to god that the Army doesn't side with the former prime minister. The understanding is that we all play cricket. Until some Australian bowls underarm. And nobody bowls underarm.
Excepting circumstances such as Lang effectively offering to float a NSW pound, or Whitlam suggesting a trivial satellite uplink and data monitoring centre be closed.
[1] A scholarly commonplace, for example, Stockwell, S (2005) 'Beyond Conspiracy Theory:US presidential archives on the Australian press, national security and the Whitlam government' [refereed] Journalism Education Conference (Griffith University); Cottle D. (1989) ‘Sailing to Byzantium’: Whitlam’s Welfare and the Australian Working Class. In: Kennedy R. (eds) Australian Welfare. Palgrave, London
