D•Scribe
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
tofu@lemmy.nocturnal.garden to Self-hosting@slrpnk.net · 22 天前

Aggressive AI scrapers are making it kinda suck to run wikis

weirdgloop.org

external-link
message-square
7
link
fedilink
  • cross-posted to:
  • technology@lemmy.zip
  • technology@beehaw.org
  • technology@lemmy.world
  • fuck_ai@lemmy.world
  • pulse_of_truth@infosec.pub
48
external-link

Aggressive AI scrapers are making it kinda suck to run wikis

weirdgloop.org

tofu@lemmy.nocturnal.garden to Self-hosting@slrpnk.net · 22 天前
message-square
7
link
fedilink
  • cross-posted to:
  • technology@lemmy.zip
  • technology@beehaw.org
  • technology@lemmy.world
  • fuck_ai@lemmy.world
  • pulse_of_truth@infosec.pub
Bots are currently scraping the internet for LLM training data at unprecedented rates[1][2][3], driving up costs and destabilizing public-facing websites. I want to talk about how this has been particularly difficult for wikis, and has gotten much worse in the last few months.

Cross posted from: https://reddthat.com/post/66285192

  • grrgyle@slrpnk.net
    link
    fedilink
    arrow-up
    6
    ·
    22 天前

    We use NGINX’s 444 response A LOT.

    Hmm interesting. I wasn’t aware of this one

Self-hosting@slrpnk.net

selfhosting@slrpnk.net

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosting@slrpnk.net

Hosting your own services. Preferably at home and on low-power or shared hardware.

Also check out:

  • Homebrewserver.club
  • XMPP chat
Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 16 users / day
  • 112 users / week
  • 354 users / month
  • 1.06K users / 6 months
  • 7 local subscribers
  • 4.41K subscribers
  • 268 Posts
  • 1.01K Comments
  • Modlog
  • mods:
  • poVoq@slrpnk.net
  • Sam_uk@slrpnk.net
  • UI: unknown version
  • BE: 0.19.18
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code
  • join-lemmy.org