Now that AI has become the main tool used by developers to write code, even in open source environments, it will be how feds will slip in backdoors to applications because nobody is going to review the logic of 20000 lines written by AI in a single commit.
Unless projects completely ban use of AI and only allow small commits, this is going to be inevitable. I’ve been seeing so many applications merging AI slop to their code on github already.
I’m not sure. This only needs to be detected once to generate damaging press for that model.
If the devs keep logs of their AI usage, they may be able to prove that the AI injected the backdoor. Then the news propagates through the tech sphere and out to the wider media. Devs then face pressure to stop using that model.
I think the more common attack will be to compromise bytecode in transit or in storage, between dev and user.
If you ask me, the real threat is companion AI in the operating system. You have an AI with direct access to your screen contents – including anything you decrypt–, creating a long-lived, searchable dossier on your activity, with the autonomy to potentially leak that information off your device. And you have no way to prove alignment.
Many models are partially cloud-based, but even a local model is still, fundamentally, a non-human-readable, nondeterministic black box. Models can lie about themselves and have limited self-insight to begin with, so the usual way to assess behavior is to simulate conditions and watch what happens. But if the model includes sleeper agent triggers, behavioral audits are unlikely to stumble onto those triggers and reveal those behaviors. Mechanistic probes can detect crude, lab-introduced sleeper triggers, but may fail against sophisticated misalignment
You’re essentially trusting a Big Tech representative to look over your shoulder at all times
Are we talking about LLMs being intentionally built to introduce backdoors despite the LLM user intentions or are we talking about attackers using LLMs to write their backdoors?
The first seems less likely. The second wouldn’t really be blamed on the LLM, at least not in a way that leads to the LLM being shunned by people that otherwise support the use of LLMs.
If we are talking about LLMs introducing backdoors despite the intentions of the LLM user, that might be targeted to specific projects.
I assumed OP meant LLMs inserting backdoors without the dev intending it. Intentional backdoors would not be a new problem