AI chatbots provide less-accurate information to vulnerable users: Research finds leading AI models perform worse for users with lower English proficiency, less formal education, and non-US origins.
AI chatbots provide less-accurate information to vulnerable users: Research finds leading AI models perform worse for users with lower English proficiency, less formal education, and non-US origins.
Study: AI chatbots provide less-accurate information to vulnerable users
A study conducted by researchers at CCC, which is based at the MIT Media Lab, found that state-of-the-art AI chatbots — including OpenAI’s GPT-4, Anthropic’s Claude 3 Opus, and Meta’s Llama 3 — sometimes provide less-accurate and less-truthful responses to users who have lower English proficiency, less formal education, or who originate from outside the United States. The models also refuse to answer questions at higher rates for these users, and in some cases, respond with condescending or patronizing language.