'Ghost walls' is a concept that is used in archaeology. But maybe not just so fanciful as in these historical fiction books:
Meanwhile, trial trenches at the north end of the adjacent long and narrow meadow—on the surface of which, when ploughed, stray finds of Roman pottery and coins had often been made—revealed a well-defined layer of Roman building debris associated with ‘ghost-walls’, or foundation-trenches from which all masonry had been removed by stone-robbers. Below this was a stratum of dark occupation-earth with an abundance of pottery indicating the presence of an extensive Early Iron Age settlement.
The Oxford Archaeology Dictionary refers to this as:
Or examplified in Hadrian's Ghost Wall:
The key to understanding Hadrian’s Wall is that the Romans built a temporary frontier of wood in the East, and wood and earth in the West, to protect them while they constructed the permanent stone frontier. It follows that there would also be temporary forts and other structures amounting to a whole ghost timber and earth version of Hadrian’s Wall.
It seems reasonable to assume that the fancy name gives rise to imagination if you hear it.
Some books mention 'ghost fences' en passant in the way described in the question:
Alistair Moffat: "The Borders: A History of the Borders from Ealiest Times", Birlinn, 2011.
Alistair Moffat: "The Sea Kingdoms: The History of Celtic Britain and Ireland", Birlinn, 2011
And a few more. Seemingly with a claim to be real science.
But, as you can see, this concept seems to be a pet peeve of one man, outside of fiction and academia.