(This answer was originally here: https://history.stackexchange.com/a/32715/1569)
The referendum was not about "continued governance by the communist party" because by the time the referendum was held the communist party had already effectively abdicated its power - the train had certainly left that station by then.
What the referendum did show was that a majority of Soviet people were in favour of retaining a federal state with an unspecified system of government rather then breaking it up to create a number of smaller independent countries, as had eventually transpired in reality.
The end of communist governance in the USSR
The communist party (CPSU) high-handedly ran the Soviet state ever since its inception. This was formally acknowledged in Article 6 of the Soviet constitution of 1977 (the earlier 1936 constitution had a similar article too, of course). But during perestroika the reins of power began to slip from the party's grasp and on March 14, 1990 this article was amended from the clear-cut statement of communist paramountcy:
The leading and guiding force of the Soviet society and the nucleus of
its political system, of all state organisations and public
organisations, is the Communist Party of the Soviet Union [...] The
Communist Party [...] determines the general perspectives of the
development of society and the course of the home and foreign policy
of the USSR [...]
to the toothless platitude:
The Communist Party of the Soviet Union, other political parties as
well as labor, youth and other public organisations and mass movements
[...] participate in the policy-making of the Soviet state [...]
This was no trifle - it was the main demand of the newly-formed democratic opposition movement and Gorbachev's giving in to it was a clear indication that the party's grip on power was effectively broken.
Indeed, the very same bill which abolished the party's monopoly on power introduced the new office of President of the USSR, to be filled by Gorbachev who urgently needed a new job title. Clearly, being merely Secretary General of the CPSU was no longer enough to command full obedience.
The referendum question: another look
If you will now reread the referendum question, you'll find that it does not mention the CPSU or communism at all! Rather, it reads:
Do you consider it necessary to preserve the Union of Soviet Socialist
Republics as a renewed federation of equal sovereign republics, which
will fully guarantee the rights and freedoms of all nationalities?
Another telling fact is that if one read's carefully the wiki page on the referendum, it turns out that originally there were 5 questions planned, rather than one. Question 2 and 3 were to refer to the preservation, respectively, of the socialist system and of the Soviets. However, all the extra questions were dropped from the final ballot. Probably because the organizers did not feel they would be able to secure a majority on them.
The struggle to preserve the federal state
During 1990 and 1991 the Soviet state went into implosion mode. Long-repressed ethnic rivalries escalated into violence, pogroms, and open warfare. The federal leadedrship, no longer sure of itself, tried half-hearted repression and then when things went sour, reneged on its own repressive measures, blaming local army commanders. This cost it its credit with the military and police.
The regional elites now sensed weakness on behalf of the central government and saw an opportunity to assert themselves. Thus during 1990 many republics of the USSR either seceded or asserted their "sovereignty", finding themselves at loggerheads with the central authorities. Which laws were now to take precedence? The federal or the republican ones?
Gorbachev, an astute enough politician, saw what was going on, saw that the old Soviet system was by now beyond saving and took a reasonable step: he invited those republics who have not seceded (9 out of 15) to negotiate a new federal compact. In effect, the Soviet Union was going to be replaced by a new creation, the Union of Sovereign States.
The referendum was a political move by Gorbachev to shore up support for the federal center vis-a-vis the republics and to leverage it to gain more power during the negotiations for the drafting of the USS. It probably achieved its purpose.
In the event, the new federal compact was to the signed on August 20, 1991. A day before that, on August 19, a group of communist blowhards in Gorbachev's government tried to turn the clock back with their infamous abortive putsch. In the aftermath, both Gorbachev's political reputation and the idea of a federation took such a hit that the final scuttling in December 1991 was almost unavoidable.