It seems to be related to a tale told of Domitian, who was reacting to a prophecy concerning his death. From The History of the Roman Emperors: From Augustus to Constantine, Volume 6 (original French) by Jean Baptiste Louis Crevier (emphasis mine)
He took a farther precaution to guard against any unexpected attack. A
stone was found in Nero's time in some quarries in Cappadocia of a
very extraordinary nature hard as marble and at the same time
transparent or rather luminous for according to Pliny the naturalist.
Nero built a temple of this stone into which one could see distinctly
when all the doors were shut. Domitian to turn this discovery to the
best advantage he could and prevent any one from coming behind him
without being perceived had all the walls of the portico's where he
used to walk cased with this stone so suitable to his views.
You can see what Pliny had to say concerning this transparent stone in Natural Histories, BOOK XXXVI. THE NATURAL HISTORY OF STONES.
CHAP. 46.—PHENGITES. During the reign of Nero, there was a stone found
in Cappadocia, as hard as marble, white, and transparent even in those
parts where red veins were to be seen upon it; a property which has
obtained for it the name of "phengites." It was with this stone
that Nero rebuilt the Temple of Fortune, surnamed Seia, originally
consecrated by King Servius, enclosing it within the precincts of his
Golden Palace. Hence it was that, even when the doors were closed,
there was light in the interior during the day; not transmitted from
without, as would be the case through a medium of specular-stone, but
having all the appearance of being enclosed within the building.
Arabia, too, according to Juba, there is a stone, transparent like
glass, which is used for the same purposes as specular-stone.