30

I have been sifting through old messages on the rec.games.chess archives part of Google Groups in particular, and I wondered where all of these messages from the '80s and '90s came from. To be specific, where the actual physical manifestations of them were/are.

According to Wikipedia, "Google Groups became operational in February 2001, following Google's acquisition of Deja's Usenet archive. Deja News had been operational since March 1995."

In addition, "The Google Groups archive of Usenet newsgroup postings dates back to 1981.[3] Through the Google Groups user interface, users can read and post to Usenet groups."

But where did all of these old messages (emails?) actually come from? What physical computer network locations where they all sourced from? Where were these messages all stored before the advent of the World Wide Web?

I find the Wiki, to be blunt, vague as hell, and I cannot seem to actually find any details anywhere. Sources and explanations would be helpful in particular. I have a very poor understanding of the early Internet, having been born into the modern age of cellphones and PCs.

11
  • 43
    I suspect that the answer is Usenet. We had an active internet before the big services were founded. Many of us were active on Usenet back in the day.
    – MCW
    Jan 8 at 21:09
  • 4
    Yes, they all came from Usenet. Sadly, they seem to have broken any advanced search functionality from when it was first ported to groups, so finding anything is non-trivial Jan 8 at 21:30
  • 13
    The "World Wide Web" is just a single type of application running on top of the Internet. The Internet itself is much older than the World Wide Web. Many of the older systems running on the Internet (gopher, usenet, etc.) are now dead, their functionality subsumed into the single http(s) protocol. Jan 8 at 21:33
  • 3
    @GorttheRobot I wouldn't say "dead", as both of the examples you cite are still used today, albeit much more rarely than in the past.
    – JBentley
    Jan 9 at 11:21
  • 5
    @MarkC.Wallace Hell, we had an active internet long before there was HTML or HTTP. I actually think hyperlinks are cool (I mean, look at our brain) but both scripting and multimedia are a regrettable wrong turn taken ;-). Jan 11 at 11:00
55

The old discussions were messages on USENET, which still exists. rec.games.chess is a newsgroup in USENET it is not firstly a part of google groups. Messages can be sent and read in the newsgroup without going near Google.

The data of the newsgroups are held on USENET servers of which there are many. USENET is distributed. Messages get copied to each server using NNTP

Each server chooses which groups to load from other servers it is configured to read from.

To read the messages a user has to have a NNTP client or Usenet reader of which there are several, Google Groups' web interface is one of these. You can also run your own server - which I did as I needed messages from more than one place and also needed a quick download on a dialup line instead of reading messages and download each one to read.

Google happens to run some of these servers and provides a front end client to read the messages. The servers got the history from Deja News which had looked and got messages from those servers that had a longer history. Normally a server is only asked for messages that have arrived since the last time that it was asked for them. From Wikipedia it appears

Web-based archiving of Usenet posts began in 1995 at Deja News with a very large, searchable database. In 2001, this database was acquired by Google.[90]

Google Groups hosts an archive of Usenet posts dating back to May 1981. The earliest posts, which date from May 1981 to June 1991, were donated to Google by the University of Western Ontario with the help of David Wiseman and others,[91] and were originally archived by Henry Spencer at the University of Toronto's Zoology department.[92] The archives for late 1991 through early 1995 were provided by Kent Landfield from the NetNews CD series[93] and Jürgen Christoffel from GMD.[94] The archive of posts from March 1995 onward was started by the company DejaNews (later Deja), which was purchased by Google in February 2001. Google began archiving Usenet posts for itself starting in the second week of August 2000.

Usenet servers tended to be in each University and many companies and when dialup internet providers started each of them often had a server, one example was AOL. As the web took over many of these were shutdown and servers outsourced to various companies. Another reason for this is that this was where Spam was first named Wikipedia history of spam and eventually I think produced much of the messages, also binary files e.g. porn, copies of music, films etc where uploaded making the volume of legal useful information too low for most of the server owners. to pay the costs of having to deal with Usenet.

There are now several Usenet hosting companies, I think, mainly catering for the binary files. There are also a few servers that provide a feed of the text only groups

The advantage of USENET clients is that they provide a fully threaded interface rather than the limits of web fora and also you only need to connect to one place rather than each web site.

9
  • 2
    Thanks for the answer. To me, it seems that legitimate posting stoped around 2004/5, any reason for that? Also, those universities where those archives were, how did that compile in the first place? Jan 8 at 23:56
  • 16
    Around 2004/5 USENET shifted to being a cryptographically secure binary distribution system due to its use by copyright violators and sexual criminals. At about the same time USENET became radically vulnerable to spam attacks by bots. The topic curated nature of open discussion by minority intellectual communities was taken over by hosted forum. Jan 9 at 1:57
  • 13
    This is the answer. If you look in the right newsgroups, you can find rather a lot of posts from a young (at the time) T.E.D. That's where I honed all my mad pointless internet essaying skillz back in the day.
    – T.E.D.
    Jan 9 at 3:14
  • 11
    @RewanDemontay - a large part of the decline of Usenet was due to "The September that Never Ended" when AOL started offering access its users. That was in 1993. Jan 9 at 16:02
  • 3
    @LaconicDroid not really usenet grew for another 10 years. Eternal September might have reduced the quality but not the use.
    – mmmmmm
    Jan 10 at 10:13
23

A point that the other answers so far perhaps don't make clear enough is the decentralised nature of the Internet back then.

We've got used to the idea that every type of content is accessible at some central location (whether that's a single server, a whole data centre, or even a group of data centres all accessible at single address or web page).  But that's not how the Internet started, nor even how it was back in the '90s when Usenet was still hugely popular.

In fact, the Internet developed from military systems which were specifically designed not to rely on big central servers with single points of failure.  And some of that mind-set persisted.

Usenet messages didn't live on a single server anywhere, because there weren't any organisations you could trust to run such a server reliably over the long term — certainly not without charging its users for the privilege.  And even if there were, you couldn't rely on being able to access them reliably at any time.  And even if you could, that access might be slow, and/or cost.

So Usenet developed to be decentralised: messages were send on from machine to machine, in batches, ending up on your nearest server from which you could collect them in one go — and then read them off-line at your leisure.

That's a very efficient approach: the transfers are done across direct links, perhaps when they're less busy (or cheaper); messages only need to be transferred once regardless of how many users will end up reading them; and reading can be done off-line without incurring any further costs.  (Yes, Internet access could be expensive back then.  I remember using dial-up access — which incurred per-minute charges even though it was a local number in the UK — and having to watch the time carefully and transfer as much as possible in one go so that I could then use it after disconnecting.)

With the advent of unmetered, always-on Internet connections (that didn't tie up your only telephone line!), rapid transfers, highly-reliable data centres, and business models that provide all sorts of services without any direct charges (discounting the indirect costs associated with advertising, usage of your data, malware, censorship…), things have undergone an enormous shift towards centralised services.  And there are of course advantages to that (as well as some disadvantages).

You can see a similar shift with another non-web application on the Internet: email.  Back in the day, email would be sent via a series of email servers (some belonging to big organisations, others less so), but it would end up on your machine, whether that was an account on a system belonging to your employer or university, or your own microcomputer/PC.  That's where the mail would live; the only place it would be accessible.  Some people still use mail clients which work that way, but most now use web mail instead, with the mail living on big servers belonging to your ISP, employer, educational institution, or some unrelated organisation — accessible from anywhere with a net-connected web browser, but no longer under your control.

Even the early web was much less centralised; most web sites were small, and finding them was hard enough that they organised themselves into web-rings and link pages.


So, to answer the question: Usenet messages physically lived on all the news servers carrying the relevant newsgroup, along with all the clients to which people had downloaded them.  Those servers were mostly at universities, Bell Labs sites, Unix-related companies, and ISPs.

7
  • I don't normally use a central http/imap server (ie gmail) for email reading, though i do download my email from the internet provider's pop/imap server. Though i could take over that too ! Including creating my own nttp server with spool and get a feed from somewhere. Jan 10 at 17:37
  • 3
    This is slightly OT, but the bigest change in "The Internet" has been the size of the content, both eMail and web. I don't have details, biut I'm guessing most text Usenet messages were in the 1-5 Kbyte range. Nowadays, it's not uncommon to see eMail content (thank you HTML and Java) in the megabyte range. Duplicating these large messages on all the NNTP servers is simply not feasible.
    – dave58
    Jan 10 at 19:25
  • 5
    @dave58 Indeed. In fact, if memory serves, most early web pages were of a similar size!  With little or no embedded Javascript, let alone CSS or embeds, even a 20KB image would dwarf the HTML itself.  Browsers would often have an option not to load images unless clicked on, for speed.  How times change — these days you can't even stop multi-MB videos from loading and playing…
    – gidds
    Jan 10 at 19:49
  • 2
    FWIW, big platforms are still unreliant on single servers; the difference is that they are owned and managed by singular entities (corps). Jan 11 at 11:00
  • 1
    @AsteroidsWithWings Agreed. (And I've tweaked the wording slightly to match.) The point is that there's a single point of access (the hostname and port), so it looks like a single server, even though that may just be the front for multiple servers, or even multiple data centres.
    – gidds
    Jan 11 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.