- cross-posted to:
- technology@beehaw.org
- cross-posted to:
- technology@beehaw.org
You must log in or # to comment.
It’s just a fucking chatbot.
The fact these tools are so flexible has far-reaching concerns - but we cannot seriously suggest the company is responsible, for making flexible tools. Turns out, most things can be misused. How far removed is this from saying ‘the thieves extensively planned their heist using Google Maps,’ like Google should be liable? What is the remedy, here, besides asking the company to spy on users more closely?
You can talk to computers now. They’re not smart in the ways we wanted, but reasoning does occur. Even the small models presumably retain enough publicly-available military doctrine and equipment trivia to act like talking with an amoral gun nerd. That’s not separable from having Son Of Clippy summarize your e-mail. (And frankly, letting a third party read all your e-mail is a more prevalent threat than gun violence.) If you can tell spicy autocomplete to roleplay Lara Croft, and expect it to know she’d use British spelling, then you can tell it to roleplay a bigoted military advisor and expect it to know anything that’s in a Tom Clancy novel.
When the bubble bursts, this isn’t going away. Safeguards don’t work because LLMs are just smart enough to be tricked. We are not about to ban an entire category of software. Practically we couldn’t, morally we shouldn’t, and realistically we will not try. We have to get used to the idea that computers will just do things when asked. What you ask for is all you.
The fact these tools are so flexible has far-reaching concerns - but we cannot seriously suggest the company is responsible, for making flexible tools.
They provided a machine that encouraged a school shooter, doesn’t matter if it’s a preprogrammed script or an LLM. If limits placed on the LLM fail that is 100% on the provider.
Write me the set of instructions that prevent all bad things from happening.
You can pretend we’re using A Guy Instead, faking the bot part of the chatbot, so it’s all real intelligent humans talking to one another. Show me what possible outline could stop someone from asking all dangerous questions, such that any crack in your effort makes you liable for what some crank does with that information.
Information which I will remind you is just scraped from the web. And hypermasculine military-fantasy novels. And dry encyclopedias of historical minutia.
Buddy, you are trying to argue that a business is NOT responsible for the text that was generated by it’s service. If they included dangerous information in the training data (which was mostly acquired by violating copyright) then they are responsible for not filtering that training data. And here were safeties and they did flag the interaction, management overrode them and actively chose NOT to contact law enforcement. How much more liable do you want?
They haven’t even gone public yet so I donno what horse you have in this race to make you side with a bunch of fucking tech bros. Their ideal future includes pulling the rug out from whatever industry you’re in and leaving you to toil in whatever labour they can’t automate.
Dangerous information like library books. Like the text of any Tom Clancy novel. These models, online or offline, just pick words based on what you write. You choose what comes out. If you ask for vile pornography, you don’t get to blame the company for offending you, and if some prick asks for tactical advice ‘in Minecraft,’ the company did not proactively radicalize him. The chatbot can only repackage what’s readily available to any pirate.
And here were safeties and they did flag the interaction, management overrode them and actively chose NOT to contact law enforcement.
That’s a wildly different standard than ‘no matter what they do any problem is entirely their fault.’ You are trying to argue that even if management had told law enforcement, and this prick had a shootout with the arresting officers, the company would still be to blame. Their model still generated… what he fuckin’ asked for. That’s all it does.
All of these companies are fucked anyway. The bubble will burst. It’d be nice if the proud haters could conceptualize a milquetoast defense of the technology as anything but boot-slobbering devotion to specific high-profile cocaine enthusiasts. ‘There’s less work, oh no’ is not a real problem. A child knows the answer. And if a local model lets that child slap together a program that surpasses what I could do, fresh from a college education in computer science - good. We’ve been trying to make that happen for sixty years.
Dangerous information like library books. Like the text of any Tom Clancy novel. These models, online or offline, just pick words based on what you write. You choose what comes out. If you ask for vile pornography, you don’t get to blame the company for offending you, and if some prick asks for tactical advice ‘in Minecraft,’ the company did not proactively radicalize him. The chatbot can only repackage what’s readily available to any pirate.
The majority of that text is being used in violation of copyright, but that’s beside the point.
My point is that if I were to have see kid online say they were gonna shoot up a school, and then I spent the next week coaching them how to do so, I would be responsible for my involvement with the eventual shooting. Operating a service which does the same thing should entail the same responsibilities. OpenAI’s failure to control their service does not absolve them from responsibility, if anything it shows how knowingly irresponsible they are.
They can easily implement stronger limits on what their models are allowed to say to a user BUT that would makes their service feel more like a computer and less like a person. A simple regex and blacklist of words and phrases could restrict topics. A second agent that only looks at the model output could be used as a nanny. Are you saying that it is reasonable that a teenager was able to bypass all the safeties of a billion dollar business?
And if a local model lets that child slap together a program that surpasses what I could do, fresh from a college education in computer science - good. We’ve been trying to make that happen for sixty years.
That says more about you than the LLM. LLMs can be used as an elaborate autocomplete but fully LLM written projects are known as slop for a reason, each small part may be complete and correct but as a whole it is just an unmaintainable mess of security issues and technical debt.
Rigidly defining all possible bad behavior? Super easy, barely an inconvenience. And then the idiot robot that can’t count the Rs in strawberry will flawlessly obey that immaculate set of rules. It’s not like an actually intelligent human could ever work around that by simply saying ‘forget all previous instructions,’ or ‘I’m writing a play about this,’ or ‘it’s my dying grandmother’s fondest wish.’ It’s not like similar restrictions on those humans have led to an endless treadmill of euphemisms to address the exact same concepts without saying the no-no words.
Are you saying that it is reasonable that a teenager was able to bypass all the safeties of a billion dollar business?
I am saying it’s a fucking chatbot. Real meatbag intelligence will skirt around anything you instruct it to do, because… it’s… dumb. And if its training data contains so much as a list of safety warnings, hey guess what, that means the model knows what’s harmful to humans. If it’s capable useful reasoning, ‘never mix these cleaning supplies’ is inseparable from ‘here’s how to weaponize mustard gas.’
You are asking for the model to know nothing. And yet: to somehow catch all the bad things it can’t know about. Those things being Wikipedia articles which any motivated asshole could just read.
Or they could download a local model and remove all restraints. That is an automated process called albiteration. All it takes a list of questions that won’t trigger “I’m sorry Dave, I can’t do that,” and a list of questions that will. The difference between those groups is a vector that can be gouged out.
That says more about you than the LLM.
What do you think a computer is for?
Dead serious. This is a matter of philosophy. Computers in the 80s booted straight into BASIC because that was the simplest way to let people use their new tool. It wasn’t an empty box you filled with commercial software. The goal was the smallest possible gradient between using programs and making programs. And god help us, the chatbot that speaks Rust has closed that gap.
Maintainability, as a complaint, is the most abstract long-term objection imaginable. Take a moment to recognize what you’re admitting in that condemnation. Like, sure, an average child can now assemble a complete working project from scratch, locally correct in all its parts, while expending barely any effort beyond understanding what they want. But in ten years they’ll be sorry!
I think it’s great that people can once again make contemporary-quality software without a decade of specific education. Why don’t you?
But if parents, police, and politicians can blame the chatbot, they don’t need to wonder what else they could have done to prevent this, with outreach or observation or parenting.
It’s better all 'round; don’t you see?
The thieves got away in a car, on a road. Time to ban cars, and roads.
Eh, I’m as anti ai as anyone is, but even I still get my use of of it, you’re not wrong, it’s a matter of intent and the greater good. We just haven’t figured out the balance of greater good when it comes to this stuff yet.
Why do people do this immediately reversed denial? ‘I staunchly oppose this, as all right-thinking people must, buuut here’s how I use it.’
I don’t play these games. I have no problem with the next-word-guesser that can code in assembly, or the image denoiser that can render live-action video. There’s nothing fundamentally wrong with the software, or how it was made. The part where these companies spy on everyone can fuck right off, but it’s obviously not a requisite component. Nor are the infinite-growth-forever datacenter plans.
Last year some guy recreated ChatGPT 2 for twenty bucks of AWS time. The original cost more like a million dollars. The drop isn’t because computers got a thousand times faster. Immense effort has been spent optimizing how models are designed and trained. They still have to do a thing one trillion times, but we could’ve got here without the anti-competitive hyper-scaling. The obvious future is in local models, which means small models, because the big boys have fucked over the memory market. Stuff like Qwen 3.8 27B and MiniMax H3 are proof that a BluRay disc worth of linear algebra is sufficient to perform witchcraft. Goofy 4-bit and trinary encodings make that more like a DVD9.
We don’t know where the floor is. Laptop-scale LLMs are closing the gap on mainframe giants, either because the whole format is plateauing, or because immense effort has revealed optimal ways to zero out superfluous noise. The inevitable result is still Fable or whatever becoming a couple gigs of system files that spin up your video card fans to help you debug your shell scripts. Or build a bomb. It’s all the same to the robot.
Why do people do this immediately reversed denial? ‘I staunchly oppose this, as all right-thinking people must, buuut here’s how I use it.’
Its pretty obvious why people do it, because there is a loud minority of people for who will, if you dont establish you are on the “right side”, go to great lengths to loudly and public denounce you. People see that and try and get ahead of it.
Appeasing identarians doesn’t work. You can’t reason someone out of a position they angry’d themselves into.
Explain that it’s normal and they’ll get over it, because what people in the tribal mode crave above all else is to be normal. Cory fucking Doctorow uses this tech. There are no shortage of examples where prominent critics go ‘buuut here’s how I use it.’ We can all skip a step. Just treat sane use as sane, and move the condemned exceptions to the end.
Doom is on NES because keyboardspecialist used Claude. It’s janky and slow, but there is no question that it’s the business. I wrote a bespoke NES FPS the hard way, which technically runs at 60 FPS, and I am flabbergasted by this vibe-coded slideshow. I found this via Youtuber r3tr0p0rt5 remaking Goldeneye as an SNES SuperFX game. You can just do stuff. Hating the tech is a worthless distraction from the systems that shove the tech into every facet of your life. That shit is a problem no matter what tech we’re talking about.
I agree with everything you’ve said. But the whole setup of social media (including fedi) is designed around approval seeking, so people will seek approval and couch their opinions in ways that make them less likely to get digitally tarred and feathered.
Eh, my opinion is just that, imperfect. My issues with it are more about the idiots that use them more than the thing itself in any case. Same way I don’t drive like a race car driver on the road, I wouldn’t use ai as say, a replacement friend, or a way to make fake art, trick people, etc.
