To answer your last question first: no, there was not really any agreement to protect cultural sites. In principle one existed under the 1899 Hague Convention (II, article 27) and the 1907 Hague Convention (IV, article 27) but the wording did not really make sense in an era of indiscriminate aerial bombing, and it was something of a dead letter.
At this point, the assumption was that WWII would begin with heavy, indiscriminate, and devastating bombing of urban areas, with little practical defence available. In the event that did not happen, but it very much shaped plans for evacuation.
Broadly speaking, the most valuable items in the Museum were indeed evacuated, in line with the other big national collections. There is a chapter on this in ES Kehoe The British Museum : the cultural politics of a national institution, 1906-1939 (PhD thesis, 2002).
Material was evacuated in late August 1939 to the National Library of Wales Aberystwth (books), rural country houses ("perishable antiquities"), and the Tube at Aldwych ("imperishable antiquities", ie ones made of stone); the latter took some time to complete. Less valuable "or less portable" material went to the basements or to sandbagged areas of the Museum; this included some key material including the Elgin marbles. (Kehoe, pp 283-4).
I have turned up a contemporary news story by a journalist (HV Morton) who visited the Aldwych site and the Museum in April 1940, and wrote about it for the Daily Herald. He explicitly describes the Museum's "empty galleries", and mentioned that:
But it is not what is left in the British Museum that interests one, so much as the space left vacant by well-remembered objects. The queerly shaped stand on which the Rosetta Stone was once poised is an arresting sight: and I never noticed it in the old days!
Some of the other major institutions reopened during wartime - for example, the National Gallery held individual exhibitions as well as a scheme where they would bring back one major evacuated painting each month. The Museum did not do this, though the library remained open - in fact there was substantial damage to the book collections from raids in 1940/41. Those raids also meant heavy rebuilding work was required, and so the Museum was reopened in stages from 1946 onwards (Sphere, 4 May 1946) and the Rosetta Stone was noted as going back on display in September 1949 (Tatler, 31 August 1949).
I can only conclude that this was a bit of a mistake. It may have been in the museum along with some of the other material - it's not clear what exactly went to Aldwych versus the basement, and at least a few of the most valuable items remained on site - but it does not seem to have been on display.