This question has been debated for a very long time, and I've never seen a conclusive answer one way or the other. I don't believe there is a single reason Japan surrendered. Rather there was a long series of defeats leading to a war they could not win. Two major blows hammered the point home and a year of even greater insanity (the invasion of Japan) was avoided.
The question of surrender was hotly debated for months within the Japanese military and government (really the same thing) prior to the Soviet invasion or atomic bombs, and still debated afterwards.
The situation Japan was facing in August 1945 was dire. Everywhere they were in retreat. Everywhere there were shortages of fuel, equipment, and trained men. Everywhere Japanese equipment, once superior or adequate, had failed to keep up with the Allies' pace of development. Unlike the Allies, they didn't have the industrial capacity to recover.
The Strategic Situation
The Allies had succeeded with Japan where Germany had failed with Britain: Japan's trade was, effectively, blockaded. It was a distant blockade where ships were sunk in transit rather than close to shore, but it crept ever closer with more and more aircraft joining in.
Unlike the Allies, the Japanese never figured out how to defend their sea lanes, curious for an island nation dependent on shipping. US and British Navies and Air Forces had whittled the Japanese merchant fleet down to a quarter of what they had when the war started. Unlike the Allies, they did not have the capacity to build more ships.
No merchant navy meant no supplies to or from Japan. No shipments of fuel. No rice from China. No replenishment for their far flung island defense ring. Their population and military would figuratively and literally starve.
I have another answer which examines this in more detail.
Like Germany, Japan was being "bombed back to the stone age". Unlike Germany, they did not have a strong industrial economy to begin with, and they never performed the same miracles of reorganization to mitigate the bombing and keep production going.
While new designs, new ships, new planes, new upgrades, and new tanks were all designed, most never got out of the prototype stage as production increasingly turned to producing enough of what they had to make good on their increasing loses.
As production and supply problems got desperate, corners were cut and quality plummeted. In contrast the avalanche of Allied supplies and ships coming at them were of increasing quality and effectiveness.
Japan was reliant on imported fuel. With their merchant navy effectively blockaded there wasn't much coming in. Domestic supply was a single digit percentage of what a hungry war machine needs.
The fuel shortage was so desperate, they sent the battleship Yamato to defend Okinawa with only enough fuel to get there. The mightiest battleship in the world sent on a suicide mission. It was sunk by the blockade well before reaching its target.
No fuel means no fuel for training. This is often not appreciated. Training a good pilot means dozens or hundreds of hours of flying time. Training and working up a ship means sailing around for hundreds of miles with fuel gulping turbines.
When there's a fuel shortage, training is the first thing to go. Japanese naval and pilot training went from some of the best in the war to some of the worst in a couple years because of an inefficient training program, and an increasingly dire lack of fuel. Lopsided air battles such as the Marianas Turkey Shoot would be the norm from mid 1944 onward.
The Allies demonstrated again and again the importance of training on all theaters as they learned to use the advantages of their own equipment against the disadvantages of the enemy. For example, in skilled hands the Zero is a deadly opponent, but it's a fragile one. Initially it dominated the Allies with its range and maneuverability, but the Allies quickly devised counters, such as the Thatch Weave, and learned to exploit the Zero's weaknesses (lack of armor, weak engine) to fight more effectively with what they had.
No fuel means no navy. No navy means no way to conduct offensive operations in the far flung Pacific theater. No offensives means no way to win the war.
High performance navy turbines are fuel hungry beasts. Even though the Japanese navy had capital assets available, they had to be husbanded for lack of fuel.
While the Allies continuously upgraded their equipment and devised new models, the Japanese ended the war with basically the same equipment as they started with. The Japanese started the war with some impressive technological advantages in the air and at sea, and some breathtaking disadvantages on the ground which were never made good.
On the ground, Japanese tanks and rifles were fine against the Chinese, but appallingly bad against what the Allies were fielding. Long, bolt action Arisaka rifles vs semi-automatic M1 Garands. Paper, leather, and cloth equipment that would disintegrate in the hot, humid jungle. Light tanks sporting small guns against the relatively heavily armed and armored Sherman.
While better tanks were available, like the Type 3 Chi-Nu, less than 200 were made. Quality problems, inexperience, and Allied air supremacy would have made them little more than a nuisance.
In the air they had the Mitsubishi A6M Zero. In skilled hands it dominated early war Allied fighters who had never seen anything so maneuverable. Something to keep in mind about WWII is many air forces were fielding their first all-metal monoplane fighters and thought that was pretty cool. The F4F Wildcat had previously been a pudgy biplane!
But while the Zero was top of the line in 1940, it was never upgraded in significant numbers. The Allies got newer and more powerful aircraft. Their existing aircraft got upgraded engines, and more guns. An F6F Hellcat sported a 2000 HP engine to haul around armor, six 50 cal machine guns, and two 20 mm cannon at 330 knots.
The Zero had an engine half as powerful which meant it couldn't add armor and armament and still remain competitive. Increasingly unskilled pilots were being sent into battle against increasingly skilled Allied pilots in an increasingly obsolete aircraft.
At sea a similar problem, while the US sported new fast battleship and carrier designs bristling with new radar and more powerful anti-aircraft armaments, the Japanese navy basically ended the war with the same navy it started with... just a lot less of it.
No wonder weapons
Unlike the Germans, the Japanese did not place their faith in technology but in the bravery of their soldiers. As such, they had no wonder weapons to give them hope of victory. No dreams of Japanese jets to sweep the sky of Allied bombers. No Japanese super tanks to roam the countryside blasting Shermans. No Japanese super submarines to roam indefinitely underwater and attack Allied shipping.
The best they had were kamikaze weapons, and they weren't stopping the Allies.
The Tactical Situation
War with China turning against them
The Second Sino-Japanese War had been going on since 1937 with skirmishes back to 1931. While the Japanese continuously defeated the Chinese, there was still no end in sight. By 1945 they were increasingly fought to a stalemate or defeated by combined Chinese and Allied forces. Even the hope of holding onto territory in China was fading and still draining ever tightening resources.
Allies threaten Indonesia, Reopen Supply Lines To China
Earlier in the war the Japanese threatened India. Now they were retreating through southeast Asia. Supply lines to China were reopened making the situation in China even worse for the Japanese.
Burma had fallen, and Indonesia (aka the Dutch East Indies) was threatened. Indonesia was the major Japanese source of fuel... if they could get it shipped. Without Indonesia, there would be no fuel.
Americans on Okinawa
With Okinawa captured, the Allies had a large base of operations within just 340 miles of the mainland.
This allowed more of their bombers to reach more of Japan with larger payloads. It allowed them to be escorted by fighters. It gave them shorter trips to make more sorties.
It gave the US Navy an anchorage close to the mainland for supply, refueling, and repair. This increased the Navy's ability to absorb damage and keep ships on the line.
It gave the Allies a huge base to build up troops and supplies for the inevitable invasion of Japan.
Allied Air Supremacy
The Allies were undisputed masters of the air over Japan. With this supremacy they could bomb and strafe troop concentrations, transport, and industry. The Japanese army would be immobilized by the threat of destruction from the air if they left their fortifications. Fortifications, tanks, and artillery could all be destroyed with impunity from the air. And unlike on Pacific islands, they did not have a jungle canopy to hide under.
Their one remaining ally, Germany, had surrendered in May 1945. There would be no wonder weapons from Germany. No supplies of know-how and strategic materials.
The Allies would no longer be distracted by a fight in Europe, they could turn everything against Japan, and they were.
Allies showing the will to invade
The war started with the presumption that the Allies would not have the will to fight a long, protracted war against dug in Japanese defenses. This was proven wrong again and again. The hope that the Allies would not have the will to invade the home islands and take casualties was an increasingly forlorn one.
Watching the Allies fight Germany to the bitter end must have given even the most staunch believers in repelling an invasion pause. Watching them take Okinawa despite fighting to the last, must also have strained the Japanese belief they could stop an Allied invasion.
Fears of a revolt
In February 1945, the Prince and the Emperor discussed the possibility of civil unrest and the possibility of being overthrown in a revolt. The idea that Japan would collapse in on itself was so humiliating that surrender was mooted as a means to save face. Minister Of The Navy Mitsumasa Yonai told the Emperor on August 12th...
...the atomic bombs and the Soviet entry into the war are, in a sense, divine gifts. This way we don't have to say that we have quit the war because of domestic circumstances.
Kamikaze tactics were not working
Japan had one great tactical innovation, if human bombs can be termed so benignly, in late WWII and that was the kamikaze. Using humans as smart bombs meant they could drive home their attack with an accuracy and durability that an untrained pilot in an outdated bomber could not. A diving aircraft with a suicidal pilot could sustain damage that would normally throw a bomber off its course and still reach its target.
While this increased the effectiveness of Japanese air forces, it did not deter the Allies. They responded by increasing their anti-aircraft defenses. More radar. More pickets. More guns firing bigger shells to knock aircraft out of the sky rather than simply fill it full of holes. The invention and use of the precious VT (proximity) fuse for anti-aircraft shells made Allied firing even more deadly to increasingly inexperienced kamikaze tactics.
And through it all, the Allies kept advancing.
No path to winning the war
What this all adds up to is in August 1945, Japan had no path to winning the war. They didn't have the fuel, equipment, or the training to carry out offensive operations. They didn't have the fuel, equipment, or experienced pilots to prevent the US from bombing them at their leisure. They had no wonder weapons to look to. Even kamikaze tactics hadn't worked, the elan of their soldiers could not stop the Allies.
...but can they end it with honor?
Anyone with any sense only saw two glimmers of hope. First, they still held large swaths of mainland China and Southeast Asia, perhaps it could be used as a bargaining chip.
Second, Japan held out the hope that if they could deliver one last decisive blow, even if it would not win the war, it might bring them to the bargaining table. That blow might be in defeating an invasion, or making it terribly costly. This was critical because the Allies were demanding unconditional surrender.
The Problem of Unconditional Surrender
To the Japanese, "unconditional surrender" meant a fate worse than death. According to the Japanese it threatened...
occupation, disarmament, elimination of militarism, democratic reforms, punishment of war criminals, and the status of the emperor.
Foreign Minister Tōgō directed his ambassador to Moscow to convey...
His Majesty the Emperor, mindful of the fact that the present war daily brings greater evil and sacrifice upon the peoples of all the belligerent powers, desires from his heart that it may be quickly terminated. But so long as England and the United States insist upon unconditional surrender, the Japanese Empire has no alternative but to fight on with all its strength for the honor and existence of the Motherland.
The Potsdam Declaration
At the end of July 1945 the Allies made it clear what surrender meant with the Potsdam Declaration. They gave a few assurances:
We do not intend that the Japanese shall be enslaved as a race or destroyed as a nation, but stern justice shall be meted out to all war criminals, including those who have visited cruelties upon our prisoners.
Japan shall be permitted to maintain such industries as will sustain her economy and permit the exaction of just reparations in kind, but not those which would enable her to rearm for war. To this end, access to, as distinguished from control of, raw materials shall be permitted. Eventual Japanese participation in world trade relations shall be permitted.
The occupying forces of the Allies shall be withdrawn from Japan as soon as these objectives have been accomplished and there has been established, in accordance with the freely expressed will of the Japanese people, a peacefully inclined and responsible government.
But made it clear what would happen if they refused.
We call upon the government of Japan to proclaim now the unconditional surrender of all Japanese armed forces, and to provide proper and adequate assurances of their good faith in such action. The alternative for Japan is prompt and utter destruction.
But they left the question of what would happen to the Emperor unanswered. Would he be tried as a war criminal for "[misleading] the people of Japan"? Or would he be left alone as part of "a peacefully inclined and responsible government"?
While this was being pondered, things started falling apart fast with Japan still debating surrender all the while.
August 6: Hiroshima
Hiroshima was destroyed in a "blinding flash and violent blast" demonstrating what the Allies meant by "prompt and utter destruction".
President Truman made this broadcast.
We are now prepared to obliterate more rapidly and completely every productive enterprise the Japanese have above ground in any city. We shall destroy their docks, their factories, and their communications. Let there be no mistake; we shall completely destroy Japan's power to make war. It was to spare the Japanese people from utter destruction that the ultimatum of July 26 was issued at Potsdam. Their leaders promptly rejected that ultimatum. If they do not now accept our terms they may expect a rain of ruin from the air, the like of which has never been seen on this earth...
The Japanese military knew about atomic bombs. They knew how difficult they were to make. There was some debate whether the US had actually built one, and there was more debate about whether they'd built more than one. Chief of the Navy General Staff Admiral Soemu Toyoda thought the US had only one bomb, that it was a bluff.
August 9: Soviets Invade Manchuria
Early morning on August 9th, the Soviets broke their neutrality pact with Japan set to expire in April 1946 and invaded Manchuria. They rapidly sliced through the once elite Kwantung Army and were moving almost as fast as they could be supplied.
It was clear China could not be held. This source of supplies and potential bargaining chip was rapidly melting away giving a new urgency to the surrender debate.
They'd be facing the specter of a Soviet invasion in weeks. Japanese defenses were previously concentrated on defending the south and east portion of the home islands, the north and west was considered safe from invasion. Now they also had to defend Hokkaido and the north which had been left understrength and lightly unfortified.
Only two divisions defended Hokkaido with defenses facing east. Even a hasty Soviet invasion could overwhelm them.
The Japanese were debating surrender with conditions. On one side was a single condition: they keep their Emperor. On the other side was more conditions: that Japan handle their own disarmament, deal with any Japanese war criminals (meanwhile they were destroying as much documentation as they could), and that there be no occupation of Japan.
August 9: Nagasaki
While that debate was happening, the US demonstrated they're not bluffing by dropping another bomb on Nagasaki. A downed US pilot told the Japanese they had 100 bombs, they believed him enough to keep him alive.
Unknown to the Japanese, the US did not have any more bombs ready, but the US expected to have another one in a couple weeks and to average about three a month. The point was moot because US bombing and naval shelling was doing a fine job on its own.
Truman again made it clear that surrender was the only option.
Having found the bomb we have used it. We have used it against those who attacked us without warning at Pearl Harbor, against those who have starved and beaten and executed American prisoners of war, against those who have abandoned all pretense of obeying international laws of warfare. We have used it in order to shorten the agony of war, in order to save the lives of thousands and thousands of young Americans.
We shall continue to use it until we completely destroy Japan's power to make war. Only a Japanese surrender will stop us.
August 10: The Emperor Consents To Surrender Negotiations
In another late night cabinet meeting, the Emperor finally decides to negotiate for surrender. He cited some specific concerns including the suffering of his people and a lack of faith in the Japanese military to repel an invasion.
I have given serious thought to the situation prevailing at home and abroad and have concluded that continuing the war can only mean destruction for the nation and prolongation of bloodshed and cruelty in the world. I cannot bear to see my innocent people suffer any longer. ...
I was told by those advocating a continuation of hostilities that by June new divisions would be in place in fortified positions [at Kujūkuri Beach, east of Tokyo] ready for the invader when he sought to land. It is now August and the fortifications still have not been completed. ...
There are those who say the key to national survival lies in a decisive battle in the homeland. The experiences of the past, however, show that there has always been a discrepancy between plans and performance. I do not believe that the discrepancy in the case of Kujūkuri can be rectified. Since this is also the shape of things, how can we repel the invaders?
It goes without saying that it is unbearable for me to see the brave and loyal fighting men of Japan disarmed. It is equally unbearable that others who have rendered me devoted service should now be punished as instigators of the war. Nevertheless, the time has come to bear the unbearable. ...
I swallow my tears and give my sanction to the proposal to accept the Allied proclamation on the basis outlined by the Foreign Minister.
That basis was unconditional surrender provided it would not "prejudice the prerogatives" of the Emperor. This was transmitted to the Allies.
August 12: The Allies Respond, The Bombing Stops
The Allies respond. They continue to refuse to give a straight answer on the status of the Emperor, instead stating he will be subject to the Supreme Allied Commander and that Japan will have a democratic government.
From the moment of surrender the authority of the Emperor and the Japanese government to rule the state shall be subject to the Supreme Commander of the Allied powers who will take such steps as he deems proper to effectuate the surrender terms. ... The ultimate form of government of Japan shall, in accordance with the Potsdam Declaration, be established by the freely expressed will of the Japanese people.
After an incorrect news report that there was a cease fire, Truman ordered a temporary halt to the bombing to not jeopardize the peace process.
August 13: Bombing Resumes
Not hearing from the Japanese, and fearing an all-out attack, Truman orders bombing be resumed "so as to impress Japanese officials that we mean business and are serious in getting them to accept our peace proposals without delay." In addition to some of the largest bombing missions of the war, the US 3rd Fleet began bombarding the coast to demonstrate their ability to strike at Japan at will.
This also came with leaflets dropped to the people of Japan stating:
The Japanese people are facing an extremely important autumn. Your military leaders were presented with thirteen articles for surrender by our three-country alliance to put an end to this unprofitable war. This proposal was ignored by your army leaders... [T]he United States has developed an atom bomb, which had not been done by any nation before. It has been determined to employ this frightening bomb. One atom bomb has the destructive power of 2000 B-29s.
This information provided to a people who had been lied to about the war for so long added to the fear the Emperor would be dethroned by a popular uprising.
Even then surrender continued to be debated, but the Emperor made his decision.
I have listened carefully to each of the arguments presented in opposition to the view that Japan should accept the Allied reply as it stands and without further clarification or modification, but my own thoughts have not undergone any change. ... In order that the people may know my decision, I request you to prepare at once an imperial rescript so that I may broadcast to the nation. Finally, I call upon each and every one of you to exert himself to the utmost so that we may meet the trying days which lie ahead.
August 14: Attempted Coup
Meanwhile, a military coup was brewing to prevent surrender by the Army. When they read the reply to surrender-with-one-condition proposal stating the Emperor would be made subordinate, the decision was made to go ahead and overthrow the government (not the Emperor). Anticipating this move, many members of the Army swore an oath to carry out the Emperor's orders.
Late at night Major Hatanaka captured the Imperial Palace, but failed to gather support. Neither could they find and destroy the recordings of the Emperor's surrender speech to be broadcast, they were hidden in a vault under the palace. Nor did they manage to assassinate the Prime Minister who had been warned just minutes earlier.
By morning's light the coup had been peacefully disbanded. Hatanaka rode through the streets distributing leaflets explaining his actions. Then he shot himself. Before he did, he wrote:
I have nothing to regret now that the dark clouds have disappeared from the reign of the Emperor.
August 15: Japan Surrenders
Finally at noon, Tokyo time, on August 15th the Emperor announced Japan's surrender. His speech is a masterpiece of spin. No where does he use the word "surrender" nor accept Japan's responsibility for the war.
It's worth reading in its entirety, here's some excerpts.
After pondering deeply the general trends of the world and the actual conditions obtaining in Our Empire today, We have decided to effect a settlement of the present situation by resorting to an extraordinary measure.
We have ordered Our Government to communicate to the Governments of the United States, Great Britain, China and the Soviet Union that Our Empire accepts the provisions of their Joint Declaration.
To strive for the common prosperity and happiness of all nations as well as the security and well-being of Our subjects is the solemn obligation which has been handed down by Our Imperial Ancestors and which lies close to Our heart.
Indeed, We declared war on America and Britain out of Our sincere desire to ensure Japan's self-preservation and the stabilization of East Asia, it being far from Our thought either to infringe upon the sovereignty of other nations or to embark upon territorial aggrandizement.
But now the war has lasted for nearly four years. Despite the best that has been done by everyone—the gallant fighting of the military and naval forces, the diligence and assiduity of Our servants of the State, and the devoted service of Our one hundred million people—the war situation has developed not necessarily to Japan's advantage, while the general trends of the world have all turned against her interest.
Moreover, the enemy has begun to employ a new and most cruel bomb, the power of which to do damage is, indeed, incalculable, taking the toll of many innocent lives. Should we continue to fight, not only would it result in an ultimate collapse and obliteration of the Japanese nation, but also it would lead to the total extinction of human civilization.
Such being the case, how are We to save the millions of Our subjects, or to atone Ourselves before the hallowed spirits of Our Imperial Ancestors? This is the reason why We have ordered the acceptance of the provisions of the Joint Declaration of the Powers...
The hardships and sufferings to which Our nation is to be subjected hereafter will be certainly great. We are keenly aware of the inmost feelings of all of you, Our subjects. However, it is according to the dictates of time and fate that We have resolved to pave the way for a grand peace for all the generations to come by enduring the unendurable and suffering what is unsufferable. Having been able to save face and maintain the structure of the Imperial State, we are always with you, our good and loyal subjects, relying upon your sincerity and integrity.
While the atomic bomb is mentioned, and there is no mention of the Soviets, this is more a piece of masterful political speech writing to justify a sudden change of policy than to be used as a true list of reasons for surrender.
I hope after reading all that I've built up my case why asking whether it was the Soviets or the Bomb that caused Japan to surrender doesn't make much sense. It was a large confluence of factors, no hope of winning the war, facing destruction on all fronts including a revolt at home, and a rapidly crumbling situation. Just as the assassination of Franz Ferdinand did not really cause WWI, neither the Soviet invasion nor the atomic bombs definitively ended the Pacific War.