...was Walden Pond ever used for large scale ice harvesting involving
rail transport?
It seems we can rely on Henry David Thoreau's account to verify this. Thoreau mentions your railroad (emphasis mine):
I have thought that Walden Pond would be a good place for business,
not solely on account of the railroad and the ice trade; it offers
advantages which it may not be good policy to divulge; it is a good
port and a good foundation.
again here:
The Fitchburg Railroad touches the pond about a hundred rods south of
where I dwell. I usually go to the village along its causeway, and am,
as it were, related to society by this link. The men on the freight
trains, who go over the whole length of the road, bow to me as to an
old acquaintance, they pass me so often, and apparently they take me
for an employee; and so I am. I too would fain be a track-repairer
somewhere in the orbit of the earth.
Note the Fitchburg Railroad is the exact same railroad mentioned in the OPs link concerning Fresh Pond. Some details concerning the production force and figures of the ice trade:
To speak literally, a hundred Irishmen, with Yankee overseers, came
from Cambridge every day to get out the ice. They divided it into
cakes by methods too well known to require description, and these,
being sledded to the shore, were rapidly hauled off on to an ice
platform, and raised by grappling irons and block and tackle, worked
by horses, on to a stack, as surely as so many barrels of flour, and
there placed evenly side by side, and row upon row, as if they formed
the solid base of an obelisk designed to pierce the clouds. They told
me that in a good day they could get out a thousand tons, which was
the yield of about one acre.
A thousand tons per day would fill 30 of the above pictured boxcars. They gave estimates on transport loss:
They calculated that not twenty-five per cent of this would reach its
destination, and that two or three per cent would be wasted in the
cars. However, a still greater part of this heap had a different
destiny from what was intended; for, either because the ice was found
not to keep so well as was expected, containing more air than usual,
or for some other reason, it never got to market. This heap, made in
the winter of ’46–7 and estimated to contain ten thousand tons, was
finally covered with hay and boards; and though it was unroofed the
following July, and a part of it carried off, the rest remaining
exposed to the sun, it stood over that summer and the next winter, and
was not quite melted till September 1848. Thus the pond recovered the
greater part.
So 10,000 Tons of ice were left stored on the pond, and took two years to melt.
So yes there was a railroad and large scale ice extraction at Walden Pond.
Was Walden pond or Fresh pond the primary source?
That probably will vary year by year. It would appear that many different ponds were used. From an article at cambridgehistory.org
Beginning in 1805, Tudor harvested ice from across Massachusetts at
places including Haggett's Pond in Andover, Spy Pond in Arlington,
Sandy Pond in Ayer, Fresh Pond in Cambridge, Walden Pond in Concord,
Suntaug Lake in Lynnfield, Spot Pond and Doleful Pond in Stoneham,
Lake Quannapowitt in Wakefield, Horn Pond in Woburn, and Wenham Lake
in Wenham.
A different source, an 1888 publication of the US Census Office gives us some details concerning the Ice business around Boston in 1888, which certainly indicates Fresh Pond as a major producer at that time, and doesn't even mention Walden Pond by name:

I found another publication, Report of the Secretary of Agriculture, by the US Dept. of Agriculture gives us some production numbers for the year 1864.

This fits the questions 1860s timeframe, and again shows Fresh pond as the top listed producer for the area, and once more no mention of Walden Pond.
Still just snapshots of information(1864 and 1888), but my inclination is that, Fresh Pond would appear to consistently be the top producer of Ice for this region and timeframe. The distribution of the ice collection across so many ponds would also seem to dispute the contention that the ice could have come 'mostly' from any 'particularly suitable pond'. (The top producer only accounting for about 30% of the production listed for 1864.)