I'm going to address your question in two parts:
1. The accuracy of the claim that attendance dropped.
2. The effect that the lectures had on the attendees.
Firstly, as you rightly hypothesised, the tale of dropping attendance was false. This stemmed from a preface to the book, written by David Goodstein and Gerry Neugebauer. They wrote that
"… as the ...