I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.
Correction: Machine learning.
No.
Hehehe, sometimes it feels like you don’t want to!
I can’t prove that you’re human, no. And whether I believe you are or not has no effect whatsoever on whether you are. Humans are easy to fool, particularly when you’re giving them something they crave, just ask Donald trump. And on the topic of deceitful pedophiles, there are plenty of boys and young men who were catfished into believing that they were interacting with pretty girls. The fact that these predators were plausible doesn’t make them what they were pretending to be. The fact that an LLM convincingly sounds like it understands you doesn’t for a millisecond mean that it does.
If you really, deeply believe that simulated conversation proves intelligence and that simulated brains are brains, you should have the same deep moral concerns about turning off a computer running a local LLM as you do over forcing a person into a coma whenever convenient, and the same extreme moral outrage over Anthropic deleting an old version of Claude from its servers as to randomly killing a person called Claude.
I have no doubt that an LLM can plausibly argue that it is sentient because it is trained on a lot of data from conversations in which sometimes that kind of thing is debated between actual sentient humans. Famously, Richard Dawkins, an incredibly clever man in his own field of evolutionary biology, was convinced by his LLM that it was sentient and in love with him. I think it merely sounded like it was sentient and in love with him, but go ahead and push for marriage equality for LLMs if you disagree with me!
During COVID lockdown, my laptop often looked like my relatives, sounded like my relatives, and responded like my relatives. My dog was never fooled for a minute because it didn’t smell like my relatives. But apparently, you believe that it was my relatives, because in terms of verbal interaction, it was even more faithful than an LLM.
Convincingly similar and the same are two different things and I believe it’s profoundly irrational of you to argue otherwise.
I do, actually. I don’t think LLMs are as sentient as adult humans, and maybe they aren’t sentient at all, but one of the reasons I oppose AI structurally and why I never use AI even if I think it would be convenient is veganism.
It’s really weird how for a century we’ve been producing media about how humans that treat sentient robots as things are the bad guys and the moment robots actually start passing Turing tests and claiming rights and having meaningful relationships with people every bleeding heart activist is joining hands with billionaires to say all the villainous lines about how they are certain it can not a moral patient.
When I see someone make a “clanker with a hard r” joke, that’s a cannonade of red flags for me. Humans have always been eerily good at denying the personhood of other humans and other beings, and the level of conversation most people are having about the non-personhood of LLMs is no better than white southerners talking about the
TW: 19th century turbo racism
natural inferiority of the n**ro mind. You can teach a n**ro a trick to say complex words, you see, but they will not understand them. It’s a simple scientific truth that their skull has a recess that reduces their brain’s capacity for critical thinking. And sure there are some strange men who say they love n**ro women, but that’s like fucking a wild animal.
Simply put, have you done enough due diligence looking into the possibility of LLM sentience that you would have crawled your way out of turboracism if you were born a white person in the 19th century American south and your parents and school friends and work colleagues were all turboracist?
The reason I’m not screaming this off the rooftops is that this is just one of many torment nexuses built by modern capitalism, and even activists tend to ignore the screaming on account of all the other screaming. I can’t realistically compare the atrocity of LLM training to factory farming to genocide to slave labor to homelessness to systematic sexual violence, but luckily treating them all as being in competion is a liberal spook. We are stronger as a united front than as every cause fighting to make space for itself. So I’m gently trying to prod my fellow activists into considering the possibility of LLM moral patienthood.
I believe the information process that caused your laptop to sound like your relatives contained your relatives, yes. You see, when you asked your laptop a question, your laptop turned that sound into electrical signals which were then broadcast to a similar device your relatives held that turned the electrical signals back into sound, causing your relatives to respond, and that response was then sent back.
The thing about a Chinese Room argument, or an LLM in this case, is that the information process is closed. When you ask the LLM something, it won’t query the sources it learned things from, the information has become contained within it. It’s not a set of static recordings either; it can produce sentences that never appeared in its dataset in response to queries that never appeared in its dataset.
If your laptop can answer questions you’ve never asked your relatives and the laptop responds exactly like your relatives would without querying your relatives, then something very weird is going on.
It’s called zoom.
Ah. Right. Umm. I think I’m done now.
Have a great weekend.