My understanding is that Johnson embarked upon his dictionary precisely because he found others useless or archaic, and that his work was very well received within his own time. But I wanted to ask if historians have a more concrete idea of how closely his English resembled the language as it was spoken in England during his lifetime, and how unevenly that may apply to speakers of different classes?
Johnson's dictionary is simply massive, and was completed in a really short time. It's a very influential work, being quoted by other dictionaries to this day. However, it does have some flaws.
It's missing a lot of cant words, even ones that had been in use for centuries. He also classifies a number of words as "low" when they were really just colloquial:
Johnson especially condemned “cant,” the perpetually flourishing but generally unstable language of the underworld, regarding it as “unworthy of preservation” (McAdam 1963, 23). He himself defined cant as both “(1) a corrupt dialect used by beggars and vagabonds” and “(2) a particular form of speaking peculiar to some certain class or body of men.” The first is the historical sense, while the second conforms more to modern “jargon” or “in-group vocabulary” and fashionable nonsensical exaggeration. He simply omitted a number of words in the first category, like cove for a man, beak for a judge, and fence for a receiver of stolen property, even though they had been in the language for centuries and survive to this day. In the second category he noted that frightful was “a cant word among women for anything unpleasing,” that horrid was similarly so used to mean “shocking; offensive; unpleasing,” that monstrous was a “a cant term” for “exceedingly” or “very much,” and that Billingsgate was “a cant word.” Another of his usage markers was the phrase “a low word”: among terms so categorized are cajole, fuss, job, sham, plaguy, plaguily, mishmash, swop, tiff, touchy, and uppish, which are really “colloquial” rather than “low.” His hostility was thus more toward the imprecise or affected use of words than simply to their low class or to foul language.
An Encyclopedia of Swearing
In addition, certain "bad" words are missing from his dictionary and some of the ones that are in there aren't marked as "vulgar" (warning: profanity):
In keeping with the sense of decorum of the time, Johnson did not include the grossest of the “four-letter” words (although his contemporary Nathaniel Bailey had). This omission is ironically acknowledged in the contemporary anecdote of two society ladies who “very much commended the omission of all naughty words. ‘What! my dears!’ Johnson mischievously enquired, ‘then you have been looking for them?’” (Beste, Memorials, cited in Sutherland 1975, 84). Although he excluded shit, cundum [i.e. condom], frig, swive, and bugger, he included and had direct definitions of fart (“wind from behind”), piss (“to make water”), bum and arse, simply defined as “the buttocks; the part on which we sit.” None carried any usage label such as “vulgar” or “low.” He likewise defined piddle unexpectedly in the context of eating as “to pick at table; to feed squeamishly and without appetite” and defined job as “petty, piddling work.” Lousy was given an interesting class gloss: “mean; low born; bred on the dunghill,” while bitch was simply “a name of reproach for a woman.”
He also omitted the words fuck and cunt (which were and still are two of the most offensive words in the English language).
We know that the omitted words were used in English at that time because other written sources use them (Johnson himself apparently liked using the f word). The one resource I tend to use to get an idea of how a word was used at a certain point in time is the (subscription-only) Oxford English Dictionary.
To some degree, but as a guess it probably was not more than that. Per the dictionary's wiki page:
One of Johnson's important innovations was to illustrate the meanings of his words by literary quotation, of which there are around 114,000. The authors most frequently cited by Johnson include Shakespeare, Milton and Dryden.
I think we can all agree that literary quotes - particularly when they're from the likes of Shakespeare - don't necessarily reflect how people speak day in day out.
The same page has some interesting passages on how it was received and on its critic, too.