This is a matter of taste. And "so bad, it's good" is an 'acquired taste'.
As a more or less mass phenomenon it is indeed a recent one, although slightly older thatn the question presumes.
Tastes differ. Tastes develop. And not all people have the same opinion on what's good, and certainly not all at the same time.
It's also quite the difference to see an audience just mocking the artist or the product, or an audience genuinely enjoying "so bad it's good".
What's described in the question is are sleeper hits which due to their small and dedicated cult followers.
There is a cultural development in Western art to observe that in Shakespeare's time mob and queen went to the same theatre and watched the same plays, equally enjoing or loathing what they saw. While paintings and music were reserved to be created by the the financing from the upper classes. This differentiated taste became really solidified during the 19th century with avantgarde artists.
One such example would be van Gogh or impressionists, shocking the cultured elites and just provoking head shakes in the lower classes. It's that bad, it can't be art. In all cases it took quite some time until more people were convinced, "hey, this is actually pretty good". They took to their time to 'understand'. But that is then clearly a re-interpretation.
Homosexual artists like Oscar Wilde or Henry Scott Tuke were equally provocative and can in modern terms be decsribed as 'bad' in the sense of being kitsch or camp.
Plan 9 is terrible. There is no way around it.
But curiously, Plan 9 doesn't make the list in Bad Movies We Love:
Ask anyone who’s been to their plex anytime recently and they’ll tell you we live in a world polluted by Bad Movies. Occasionally, though, there are Bad Movies that separate themselves from the pack, special Bad Movies: those big- budget, big-star, big-director, aggressively publicized fiascos that have gone wonderfully, irredeemably, lovably haywire. We call them Bad Movies We Love. To rate a special place in our hearts and in this, our tome, not only did the movies have to be jaw-droppingly, astoundingly bad, they had to be fun bad—the kind of fun that means that, when you’re wandering the aisles at the video store looking for a good time, if you’re hip to these movies, you can’t stop yourself from yanking them off the shelves. (xvii–xviii)
For Ernest Mathijs, trash cinema has become important for the challenge it provides to reception studies in the ways the films’ reputation never seem to settle, moving in and out of favour. His example was the Harry Kümel-directed Daughters of Darkness (1971), a film ‘described as both a masterwork and rubbish’, and thus ‘an excellent example for the study of the reception of trash cinema’ (2005: 453). His conclusion was that the unfinished nature of the film’s reception paralleled a significant shift in film studies. Since its release
This needs seperation for the kind of trash Plan 9 represents:
‘trash’ has become a very different word in cinema studies. If it first referred to straightforward rubbish, it now carries a much more subtle and complex status. When, in a recent discussion of Kümel’s work in Sight & Sound, Daughters of Darkness is intro- duced as ‘commercial trash’ the word is used in a far less negative way. It has come to signify a particular kind of film, characterised by its openness to different interpretations, much more than just a bad film. The change parallels a change in film discourse, in which issues of aesthetic quality have become less absolute, more dominated by what Jeffrey Sconce has called paracinematic taste. (2005: 471)
One earlier example than Plan 9 would be the outrageously bad Marihuana (1936)
a.k.a. Marihuana, the Devil's Weed by Dwain Esper which people watched as genuine propaganda and paranoia thrill. Watching it now can only be done for "oh my god how terrible, it's hilarious" just like the now 'classic' Reefer Madness and many more of the early exploitation films.
On the same level, or "every bit as demented as Ed Wood Jr" would by Denver Dixon, real name Victor Adamson also with some pearls on archive.org.
But this illustrates the difficulty in the sought after concepts here. For these differentiations in taste to develop, you have to allow for quite some time.
To not only look for fore-runners or very roughly comparable artists and movies, this whole concept of "bad is good" really only took off in the 1950s for movies.
In a review contrasting a series of French low-budget TV features about teenagers with a set of related but different American ‘Drive-In Classics’ that reimagined AIP films from the 1950s, Jonathan Rosenbaum dismissed the idea that the latter were based on B-films. In his account, in the 1950s teenagers with enough pocket money and autonomy led to a completely new strain of filmmaking and film-going: ‘For virtually the first time, “bad” movies that teenagers could feel superior or at least equal to became a significant part of movie culture’ (2016). For Rosenbaum, this was not the naïve ‘badness’ evident in the films of Edward D. Wood Jr., but the ‘more calculated and ironic “badness” of a Corman quickie’, though ‘only later generations, with their approximate grasp of film history and market distinctions, would call both of them B-movies’.
Quotes from
Guy Barefoot: "Trash Cinema The Lure Of The Low", Short Cuts –
Introductions To Film Studies, Wallflower: London, New York, 2017.
Largely the same story is presented in
Greg Taylor: "Artists In The Audience. Cults, Camp, And American Film Criticism", Princeton University Press Princeton, New Jersey, 1999.