University of Sheffield lecturer Wei Xing argued that the AI industry wouldn't be economically incentivized to fix hallucinations.

“In short, the OpenAI paper inadvertently highlights an uncomfortable truth,” Xing concluded. “The business incentives driving consumer AI development remain fundamentally misaligned with reducing hallucinations.”

  • PKMKII [none/use name]@hexbear.netEnglish
    48·
    6 days ago

    “However, consumer applications still dominate AI development priorities,” [Xing] added. “Users want systems that provide confident answers to any question.”

    The belief in infinite technological progress is so strong that people would rather be given garbage information than be told the chatbot does not contain all the possible wisdom of the universe.

  • prole [any, any]@hexbear.netEnglish
    29·
    6 days ago

    I am begging someone to make one that doesn’t confidently answer everything and just says it doesn’t know or gives multiple possibilities, confidence levels, anything other than blind confidence. I use them to write code sometimes and it’s wild how often it is like “you’re totally right!” And I am, in fact, wrong. I rarely turn to an LLM for something I don’t already know the answer to because of this shit.

    Once again capitalism ruins everything around me

    • fox [comrade/them]@hexbear.netEnglish
      28·
      6 days ago

      Yeah they can’t do that because they’re unable to discern truth from falsehood and all that’s happening is matrix algebra on vectors representing how statistically likely an output is for any given input. Hallucinations are produced by the exact same process as useful answers

    • DogThatWentGorp [he/him, they/them]@hexbear.netEnglish
      14·
      6 days ago

      It’s really a shame how LLMs could be legitimately useful for certain things but the obsession capital has fixated onto is tricking investors into using it anywhere but those things.

      It must’ve felt like a fucking concert a year ago in every executive board room in the country, yuppies doing these flashy tech demos that made suits feel like geniuses who saw god.

      And, you know what, every executive type I’ve met in my life is an absolute addict for flattery and praise. I bet chatgpt’s tendency to make sure you always feel like you’re correct appealed sooooo bad to them.

    • Wheaties [she/her]@hexbear.netEnglish
      11·
      6 days ago

      The article lays out a pretty simple fix, but points out that none of the AI companies are financially incentivized to do so. Probably gonna be another Deepseek thing, where it takes a Chinese company to once again undercut the US firms that refuse to actually compete with one another because they’d rather keep the money train going by throwing more and more data-centers at the problem.

    • Cimbazarov [none/use name]@hexbear.netEnglish
      1·
      5 days ago

      I do feel like this is impossible, since I dont think an LLM could ever give a 100% correct response, and how to determine something is correct also seems wishy washy. I’ve seen an LLM that gives multiple possibilities and tbh I didn’t like using it since each possibility was a ton of slop to read through and I felt like I could just do the thing myself faster than read through each possibility and verify its correctness. So in Xing’s article, he is kind of right when he says that when LLM’s admit uncertainty, people tend to use it less (which maybe would be a good thing!).