Elon Musk’s Wikipedia rival Grokipedia got off to a “rocky start” in its public debut, but Wikipedia founder Jimmy Wales didn’t even have to take a look at the AI’s output to know what he expected.

“I’m not optimistic he will create anything very useful right now,” Wales said at the CNBC Technology Executive Council Summit in New York City on Tuesday.

Wales had plenty of choice words for Musk, notably in response to allegations that there is “woke bias” on Wikipedia. “He is mistaken about that,” Wales said. “His complaints about Wiki are that we focus on mainstream sources and I am completely unapologetic about that. We don’t treat random crackpots the same as The New England Journal of Medicine and that doesn’t make us woke,” he said at the CNBC event. “It’s a paradox. We are so radical we quote The New York Times.”

“I haven’t had the time to really look at Grokipedia, and it will be interesting to see, but apparently it has a lot of praise about the genius of Elon Musk in it. So I’m sure that’s completely neutral,” he added.

Wales’ digs at Grokipedia — which has its own wiki page — were less about any ongoing spat with Musk and more about his significant concerns about the efforts by all large language models to create a trusted online source of information.

“The LLMs he is using to write it are going to make massive errors,” Wales said. “We know ChatGPT and all the other LLMs are not good enough to write wiki entries.”

Wales gave several real-world examples of why he doesn’t have faith in LLMs to recreate what Wikipedia’s global community has built over decades at a fraction of the cost — he estimated the organization’s hard technology costs as $175 million annually versus the tens of billions of Dollars big tech companies are constantly pouring into AI efforts, and by one Wall Street estimate, a total of $550 billion in AI spending expected by the so-called hyperscalers next year.

One example Wales cited of LLM’s inaccuracy relates to his wife. Wales said he often asks new chatbot models to research obscure topics as a test of their abilities, and asking who his wife is, a “not famous but known” person, he said, who worked in British politics, always results in a “plausible but wrong” answer. Any time you ask an LLM to dig deep, Wales added, “it’s a mess.”

He also gave the example of a German Wiki community member who wrote a program to verify the ISBN numbers of books cited, and was able to trace notable mistakes to one person. That person ultimately confessed they had used ChatGPT to find citations for text references and the LLM “just very happily makes up books for you,” Wales said. 

Wales did say the battles into which he has been drawn, by Musk and by AI, do reinforce a serious message for Wikipedia. “It’s really important for us and the Wiki community to respond to criticism like that by doubling down on being neutral and being really careful about sources,” he said. “We shouldn’t be ‘wokepedia.’ That’s not who we should be or what people want from us. It would undermine trust.”

Wales thinks the public and the media often give Wikipedia too much credit. In its early days, he says, the site was never as bad as the jokes made about it. But now, he says, “We are not as good as they think we are. Of course, we are a lot better than we used to be, but there is still so much work to do.”

And he expects the challenges from technology, and from misinformation, to get worse, with the ability to use LLMs to create fake websites with plausible text getting better and likely able to fool the public. But he says they will have a hard time fooling the Wiki community, which has spent 25 years studying and debating trusted information sources. “But it will fool a lot of people and that is a problem,” he said.

In some cases, this same new technology, which “makes stuff up that is completely useless,” may be useful to Wikipedia, he said. Wales has been doing some work on finding limited domains where AI can uncover additional information in existing sources that should be added to a wiki, a use of gen AI he described as currently being “kind of okay.”

“Maybe it helps us do our work faster,” he said. That feedback loop could be very useful for the site if it developed its own LLM that it could train, but the costs associated with that have led the site to hold off any formal effort while it continues to test the technology, he added.

“We are really happy Wiki is now part of the infrastructure of the world, which is a pretty heavy burden on us. So when people say we’ve gotten biased, we need to take that seriously and work on anything related to it,” Wales said.

But he couldn’t resist putting that another way, too: “We talk about errors that ChatGPT makes. Just imagine an AI solely trained on Twitter. That would be a mad, angry AI trained on nonsense,” Wales said.