michael

Mike @piefed.chrisco.me 2w ago

Nice!

Mike @piefed.chrisco.me 2w ago

nice!

Mike @piefed.chrisco.me 2w ago

Im not sure. I agree with you.

All I can tell you is that my server was hammered by a couple of IP addresses. When I did a lookup they were ALL openai and they just would not stop. Until I added protection that is. Then they got bogged down and eventually stopped.

Ive heard talk that their engineers said they dont need to abide by robots.txt since they are not an indexer like google. Which is BS.

4

Mike @piefed.chrisco.me 2w ago

The bigger companies are looking at robots.txt to see if they can scan your stuff for AI scraping purposes. I get a couple from google, bing and others. Im not sure about facebook, but I do see the bigger ones usually abide by robots.txt and stop there. It doesn't stop them from hammering your robots.txt though.

If you get fail2ban and/or block the ip range from one actor, it usually goes away.

The worst offenders is openai which does NOT hit robots.txt and just scrapes/DDOS my small site. Until I put in a couple of infinite loop/nefarious solutions on the server. Then you have fail2ban see what ip addresses try and go deep and block them.

1

Mike @piefed.chrisco.me 3w ago

So....is Xbox done?

Mike @piefed.chrisco.me 3w ago

I just downvote and move on.

Can we get some examples? I just took a look and i dont see any ai posts here?

Mike @piefed.chrisco.me 3w ago

Check out !peertube@lemmy.world most videos dont have ads.

I remember when cable had very little ads. Its sttange that we pay so much for cable/internet/premium but companies chasing that dollar keep throwing ads in there.

Mike @piefed.chrisco.me 3w ago

It sure makes things easy to host!

2

Mike @piefed.chrisco.me 3w ago

Wait so both ssdand mechanical drives have been sold out...

Thats not good.

Mike @piefed.chrisco.me 3w ago

He later goes and talks to the lady who did the segment. It makes me smile because even she says its the BBC recipe thats the issue haha.

1

Mike @piefed.chrisco.me 3w ago

Looks delish!

2

Mike @piefed.chrisco.me 3w ago

https://m.youtube.com/watch?v=53me-ICi_f8

10

Mike @piefed.chrisco.me 4w ago

You can call your code enforcement in your city and get them cited for light regulations.

Ive done that when I was at an apartment and a business decided to put lights on all night at our building. They had to take them down so we could sleep. Imagine daylight through your window bright.

They got fined and everything. It was glorious.

Mike @piefed.chrisco.me 4w ago

Oh interesting! Ive done something similar but not didnt put as much effort.

For me, I just made an unending webpage that would create a link to another page...that would say bullshit. Then it would have another link with more bullshit....etc...etc...And it gets slower as time goes on.

Also made a fail2ban banning IPs that reached a certain number of links down. It worked really well, traffic is down 95% and it does not affect any real human users. Its great :)

I have a robots.txt that should tell them not to look at the sites. But if they dont want to read it, I dont want to be nice.

Mike @piefed.chrisco.me 4w ago

Same. Its been pretty great!

3

Mike @piefed.chrisco.me 4w ago

So other than matrix, is there any other open source system that people recommend?

Ive had a heck of a time with matrix for years. Im glad it exists, but its just not what I like to use. The signup process + logging in system is much too complex for a normal user.

Mike

@ michael @piefed.chrisco.me

Posts

13
Comments

82
Joined

3 mo. ago

Mike

Linux in California is in deep trouble.

Linux in California is in deep trouble.

Linux in California is in deep trouble.

What do you do when a video doesn't load in peertube?

What are things that bring you joy?

My odometer rolled to this just as I pulled into the driveway

This is a federated test post from a nodeBB forum.

Facebook's Fascination with My Robots.txt

Facebook's Fascination with My Robots.txt

Xbox chief Phil Spencer is leaving Microsoft

Captain Picard sings Make it So Let it Snow

Flood of vibe-coded/slopware spam?

Consumerism ahhhhh moment

Battle of Wesnoth for the Miyoo Mini Plus - start screen

A newbie's guide to self-hosting with YunoHost. Part 4: backups

SSD prices in yet more trouble as two of the biggest hard drive makers have already sold out their 2026 stock

Chicken Fried Rice

Chicken Fried Rice

Chicken Fried Rice

Tesla's flashing lights

Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers

Is it just me has there been a lot of Spam and abuse on Lemmy lately?

Discord is about to require age verification for everyone

Paul the Octopus

Good command line ebook reader?

Sleepy days with Teddy

Artemis I / Orion Re-Entry & Splashdown (2022) (full un-edited video from inside capsule, including audio)

Oh thats neat! Events from communities show up on the dashboard.

There is something mildy infuriating about Discord putting Tux behind a paywall.

Terminal-Based Web Browsing With Modern Conveniences

A newbie's guide to self-hosting with YunoHost. Part 2: installation & setup - Elena Rossini

A newbie's guide to self-hosting with YunoHost. Part 2: installation & setup - Elena Rossini

Thanks to @rimu, ericgaspar, and tituspijean, yunohost has a working piefed setup!