I have a similar recollection. You might consult the British History Magazine podcast on workhouses (within the last 6 months, although I have trouble searching their archives); that is where I'm most likely to have heard it.
Third google result is "Recollections of Workhouse Visiting and Management During Twenty-five Years" link Page 68 of this book contains a quote similar to what you suggest, (I haven't mastered the art of quoting from google books) The same quote is in Women's Work, where in a footnote it is attributed to Sir F.B. Head.
There is some vaguely similar language in the 1832 Commission report; not close enough to be useful, but I suspect that the debate, and possibly the writings of Chadwick or Bentham might be a useful line of research.