These are not similar. Wikipedia says the same thing to everybody, and when what it says is wrong, anybody can correct it, and they do. Consequently it's always been fairly reliable.
Lies and mistakes persist on Wikipedia for many years. They just need to sound truthy so they don't jump out to Wikipedia power users who aren't familiar with the subject. I've been keeping tabs on one for about five years, and its several years older than that, which I won't correct because I am IP range banned and I don't feel like making an account and dealing with any basement dwelling power editor NEETs who read Wikipedia rules and processes for fun. I know I'm not the only one to, because this glaring error isn't in a particularly obscure niche, its in the article for a certain notorious defense initiative which has been in the news lately, so this error has plenty of eyes on it.
In fact, the error might even be a good thing; it reminds attentive readers that Wikipedia is an unreliable source and you always have to check if citations actually say the thing which is being said in the sentence they're attached to.
That's true too, but the bigger difference from my point of view is that factual errors in Wikipedia are relatively uncommon, while, in the LLM output I've been able to generate, factual errors vastly outnumber correct facts. LLMs are fantastic at creativity and language translation but terrible at saying true things instead of false things.
Comments like these honestly make me much more concerned than LLM hallucinations. There have been numerous times when I've tracked down the source for a claim, only to find that the source was saying something different, or that the source was completely unreliable (sometimes on the crackpot level).
Currently, there's a much greater understanding that LLM's are unreliable. Whereas I often see people treat Wikipedia, posts on AskHistorians, YouTube videos, studies from advocacy groups, and other questionable sources as if they can be relied on.
The big problem is that people in general are terrible at exercising critical thinking when they're presented with information. It's probably less of an issue with LLMs at the moment, since they're new technology and a certain amount of skepticism gets applied to their output. But the issue is that once people have gotten more used to them, they'll turn off they're critical thinking in the same manner that they turn it off when absorbing information from other sources that they're used to.
Wikipedia is fairly reliable if our standard isn't a platonic ideal of truth but real-world comparators. Reminds me of Kant's famous line. "From the crooked timber of humankind, nothing entirely straight can be made".
The sell of Wikipedia was never "we'll think so you don't have to", it was never going to disarm you of your skepticism and critical thought, and you can actually check the sources. LLMs are sold as "replace knowledge work(ers)", you cannot check their sources, and the only way you can check their work is by going to something like Wikipedia. They're just fundamentally different things.
> The sell of Wikipedia was never "we'll think so you don't have to", it was never going to disarm you of your skepticism and critical thought, and you can actually check the sources.
You can check them, but Wikipedia doesn't care what they say. When I checked a citation on the French Toast page, and noted that the source said the opposite of what Wikipedia did by annotating that citation with [failed verification], an editor showed up to remove that annotation and scold me that the only thing that mattered was whether the source existed, not what it might or might not say.
I feel like I hear a lot of criticism about Wikipedia editors, but isn't Wikipedia overall pretty good? I'm not gonna defend every editor action or whatever, but I think the product stands for itself.
Wikipedia is overall pretty good, but it sometimes contains erroneous information. LLMs are overall pretty good, but they sometimes contain erroneous information.
The weird part is when people get really concerned that someone might treat the former as a reliable source, but then turn around and argue that people should treat the latter as a reliable source.
I had a moment of pique where I was just gonna copy paste my reply to this rehash of your original point that is non-responsive to what I wrote, but I've found myself. Instead, I will link to the Wikipedia article for Equivocation [0] and ChatGPT's answer to "are wikipedia and LLMs alike?"
Wikipedia occasionally has errors, which are usually minor. The LLMs I've tried occasionally get things right, but mostly emit limitless streams of plausible-sounding lies. Your comment paints them as much more similar than they are.
In my experience, it's really common for wikipedia to have errors, but it's true that they tend to be minor. And yes, LLMs mostly just produce crazy gibberish. They're clearly worse than wikipedia. But I don't think wikipedia is meeting a standard it should be proud of.
> Whereas I often see people treat Wikipedia, posts on AskHistorians, YouTube videos, studies from advocacy groups, and other questionable sources as if they can be relied on.
One of these things is not like the others! Almost always, when I see somebody claiming Wikipedia is wrong about something, it's because they're some kind of crackpot. I find errors in Wikipedia several times a year; probably the majority of my contribution history to Wikipedia https://en.wikipedia.org/wiki/Special:Contributions/Kragen consists of me correcting errors in it. Occasionally my correction is incorrect, so someone corrects my correction. This happens several times a decade.
By contrast, I find many YouTube videos and studies from advocacy groups to be full of errors, and there is no mechanism for even the authors themselves to correct them, much less for someone else to do so. (I don't know enough about posts on AskHistorians to comment intelligently, but I assume that if there's a major factual error, the top-voted comments will tell you so—unlike YouTube or advocacy-group studies—but minor errors will generally remain uncorrected; and that generally only a single person's expertise is applied to getting the post right.)
But none of these are in the same league as LLM output, which in my experience usually contains more falsehoods than facts.
> Currently, there's a much greater understanding that LLM's are unreliable.
Wikipedia being world-editable and thus unreliable has been beaten into everyone's minds for decades.
LLMs just popped into existence a few years ago, backed by much hype and marketing about "intelligence". No, normal people you find on the street do not in fact understand that they are unreliable. Watch some less computer literate people interact with ChatGPT - it's terrifying. They trust every word!
If you read a non-fiction book on any topic, you can probably assume that half of the information in it is just extrapolated from the authors experience.
Even scientific articles are full of inaccurate statements, the only thing you can somewhat trust are the narrow questions answered by the data, which is usually a small effect that may or may not be reproducible...
No, different media are different—or, better said, different institutions are different, and different media can support different institutions.
Nonfiction books and scientific papers generally only have one person, or at best a dozen or so (with rare exceptions like CERN papers), giving attention to their correctness. Email messages and YouTube videos generally only have one. This limits the expertise that can be brought to bear on them. Books can be corrected in later printings, an advantage not enjoyed by the other three. Email messages and YouTube videos are usually displayed together with replies, but usually comments pointing out errors in YouTube videos get drowned in worthless me-too noise.
But popular Wikipedia articles are routinely corrected by hundreds or thousands of people, all of whom must come to a rough consensus on what is true before the paragraph stabilizes.
Consequently, although you can easily find errors in Wikipedia, they are much less common in these other media.
Yes, though by different degrees. I wouldn't take any claim I read on Wikipedia, got from an LLM, saw in a AskHistorians or Hacker News reply, etc., as fact, and I would never use any of those as a source to back up or prove something I was saying.
Newspaper articles? It really depends. I wouldn't take paraphrased quotes or "sources say" as fact.
But as you move to generally more reliable sources, you also have to be aware that they can mislead in different ways, such as constructing the information in a particular way to push a particular narrative, or leaving out inconvenient facts.
And that is still accurate today. Information always contains a bias from the narrators perspective. Having multiple sources allows one to triangulate the accuracy of information. Making people use one source of information would allow the business to control the entire narrative. Its just more of a business around people and sentiments than being bullish on science.
And they were right, right? They recognized it had structural faults that made it possible for bad data to sip in. The same is valid for LLMs: they have structural faults.
So what is your point? You seem to have placed assumptions there. And broad ones, so that differences between the two things, and complexities, the important details, do not appear.
people used to say the exact same thing with wikipedia back when it first started.