Skip Navigation

User banner

FaceDeer

@ FaceDeer @fedia.io

Posts
0
Comments
3071
Joined
2 yr. ago

Basically a deer with a human face. Despite probably being some sort of magical nature spirit, his interests are primarily in technology and politics and science fiction.

Spent many years on Reddit before joining the Threadiverse as well.

  • A Project to Poison LLM Crawlers

    Jump

  • I have a sneaking suspicion that the vast majority of the people raging about AIs scraping their data are not raging about it being done inefficiently.

  • A Project to Poison LLM Crawlers

    Jump

  • You're thinking of "model decay", I take it? That's not really a thing in practice.

  • A Project to Poison LLM Crawlers

    Jump

  • Raw materials to inform the LLMs constructing the synthetic data, most likely. If you want it to be up to date on the news, you need to give it that news.

    The point is not that the scraping doesn't happen, it's that the data is already being highly processed and filtered before it gets to the LLM training step. There's a ton of "poison" in that data naturally already. Early LLMs like GPT-3 just swallowed the poison and muddled on, but researchers have learned how much better LLMs can be when trained on cleaner data and so they already take steps to clean it up.

  • A Project to Poison LLM Crawlers

    Jump

  • I have no idea what "established means" would be. In the particular case of the Fediverse it seems impossible, you can just set up your own instance specifically intended for harvesting comments and use that. The Fediverse is designed specifically to publish its data for others to use in an open manner.

  • A Project to Poison LLM Crawlers

    Jump

  • Are you proposing flooding the Fediverse with fake bot comments in order to prevent the Fediverse from being flooded with fake bot comments? Or are you thinking more along the lines of that guy who keeps using "Þ" in place of "th"? Making the Fediverse too annoying to use for bot and human alike would be a fairly phyrric victory, I would think.

  • A Project to Poison LLM Crawlers

    Jump

  • A basic Google search for "synthetic data llm training" will give you lots of hits describing how the process goes these days.

    Take this as "defeatist" if you wish, as I said it doesn't really matter. In the early days of LLMs when ChatGPT first came out the strategy for training these things was to just dump as much raw data onto them as possible and hope quantity allowed the LLM to figure something out from it, but since then it's been learned that quality is better than quantity and so training data is far more carefully curated these days. Not because there's "poison" in it, just because it results in better LLMs. Filtering out poison will happen as a side effect.

    It's like trying to contaminate a city's water supply by peeing in the river upstream of the water treatment plant drawing from it. The water treatment plant is already dealing with all sorts of contaminants anyway.

  • A Project to Poison LLM Crawlers

    Jump

  • I think it's worthwhile to show people that views outside of their like-minded bubble exist. One of the nice things about the Fediverse over Reddit is that the upvote and downvote tallies are both shown, so we can see that opinions are not a monolith.

    Also, engaging in Internet debate is never to convince the person you're actually talking to. That almost never happens. The point of debate is to present convincing arguments for the less-committed casual readers who are lurking rather than participating directly.

  • A Project to Poison LLM Crawlers

    Jump

  • Doesn't work, but I guess if it makes people feel better I suppose they can waste their resources doing this.

    Modern LLMs aren't trained on just whatever raw data can be scraped off the web any more. They're trained with synthetic data that's prepared by other LLMs and carefully crafted and curated. Folks are still thinking ChatGPT 3 is state of the art here.

  • [Meta] RE: Starfleet Academy

    Jump

  • even bad trek is more trek.

    What this attitude tells executives is "guess we don't need to bother trying to make Trek good, they'll eat up whatever we shovel their way."

    I'm a big fan of Star Trek. That's why I haven't watched anything other than Lower Decks since Discovery started airing.

  • ‘The streets are full of blood’: Iranian protests gather momentum as regime cracks down

    Jump

  • Meanwhile Americans are moaning about how they can't protest because they need their jobs to provide their health insurance.

  • EU_irl

    Jump

  • Yeah, the list goes on and on.

  • EU_irl

    Jump

  • It's "I've taken Venezuela, now give me Greenland. And Cuba. And Panama. And Canada. And.,.."

  • A $400,000 payout after Maduro's capture is putting prediction markets in the spotlight

    Jump

  • Seems likely to have been the result of inside information, sure.

    But that's the point of prediction markets, isn't it? Their purpose is to get people who have knowledge about a topic to put that knowledge out there to the public, with money put up against that to back their certainty. It's a way of surfacing that knowledge to the general public. People who don't have knowledge and are just making gambling bets based on their "feelings" are going to get screwed now and then. They probably shouldn't be doing that.

  • Nobel Institute rejects María Corina Machado’s offer to share peace prize with Trump

    Jump

  • Eh, I could see it as a reasonable deal from her perspective. If giving Trump a pointless shiny metal disk got him to actually back democratic reform in Venezuela, sure, give him the shiny metal disk. Whatever, it's just a thing.

    Unfortunately it wouldn't work, though; Trump never sticks to his deals after he's got what he wanted out of them or when he sees an opportunity to squeeze even more out of you. So might as well keep the shiny metal disk.

  • Has Canada's government done anything concrete to reduce dependence on the US since Trump took office? Maybe even since the first term?

    Jump

  • Carney didn't "clap for the attack on Venezuela." He called for international law to be followed, which should be an obvious rebuke to anyone who isn't at a Trump level of understanding of how diplomacy is done.

  • Oh no! Linus doesn't know AI is useless!

    Jump

  • I've long found it funny how some people claim that generative AI produces terrible slop, and simultaneously that it's a huge threat to their jobs.

  • AI insiders seek to poison the data that feeds them

    Jump

  • People have been doing this to "protest" AI for years already. AI trainers already do extensive filtering and processing of their training data before they use it to train, the days of simply turning an AI loose on Common Crawl and hoping to get something out of that are long past. Most AIs these days train on synthetic data which isn't even taken directly from the web.

    So go ahead and do this, I suppose, if it makes you feel better. It's not likely to have any impact on AIs though.

  • U.S. launches large-scale strikes on ISIS targets across Syria

    Jump

  • Perhaps he wants to be the only one left so that Sempai will notice him.

  • Finland: For the first time, a Russian grey-zone sabotage relying on unclear responsibility was exposed by enforcement rather than inference, making the shield protecting hybrid warfare disappear

    Jump

  • Can you find a link to a news story about it?