SerotoninSwells

I feel you and feel for you. I really do hope you get a reprieve because dealing with this is nonsense.

👋 Hi, I really hope iocaine works for you and I think it still might be wise to temper expectations. Some background, I work in bot detection and mitigation.

I quickly tried reading through their code and documentation but I don’t see the main detection mechanism that determines human vs bot other than what you mentioned as an example. If it’s user agent based, it is trivially easy to spoof as you already know. I am finding in my work that these companies do not keep the user agent they report in their documentation when challenged.

My second concern was the page the reverse proxy served when spoofing my user agent. The DOM was nowhere close to that of Lemmy and I think it’s important to point out that a simple check for specific elements on the page will keep the bot from poisoning itself.

I admit I could be too close to this problem to see other solutions, and I really hope it works. It sucks that this is a problem. I wish there were more open source options too.

If for some reason this solution doesn’t work, and if anyone is interested in help, I am more than happy to freely offer my knowledge.

Veritasium did a video on this topic a few days ago. I highly recommend it. There’s a bit of nuance here, from what I understand, regarding PTFE which is the chemical composition that Chemours markets as Teflon. The video talks about PTFE being rather inert, passing through our bodies if we ingest it. The real issue is heating the substance above 350° C (662° in freedom units).

I’m not an expert but I think it’s worth reading up on the subject. If there’s anyone else more read up on the subject please let me know if I’m wrong here.

It can be both. Reddit has a history of fabricating conversations. The way they sell advertising implies a certain level of engagement from their user base which can lead to bots pushing products in the form of reviews or by mention.

I think it’s worth noting that Reddit, at one time, did have third party bot protection; however, it only protected their advertising. I can only imagine what the rest of their traffic looks like, but I would not be surprised if they were using bots of their own.

Like you said, they can make some money selling your information but they can also control the narrative how they choose.

Thank you for the reminder. I donated 👍

“Those who control the present, control the past and those who control the past control the future.”

George Orwell, 1984

Omg yes how do I throw money at you.

We’ve had one, yes. But what about second bird flu strain?

Unfortunately it is out of date.

• IPs used by bots are now *highly * distributed. We will see the same bot use hundreds of thousands of IP addresses. Each IP can easily only make one or two requests which is hard to limit with volume based detections. Also, I’m not sure where you’re at in the world, but it’s more common in countries outside of North America to have IP addresses that are heavily shared. Not to mention, there are companies in Europe that will pay you for use of your IP address explicitly for bots.

• You might think you could limit by IP classification but bots increasingly use residential classified IPs.

• As for allowing good bots, that isn’t so much an issue. They respect the robots.txt that companies implement. We see bots scraping data for LLMs more and more that don’t respect this file. Also, bots that are scraping prices and anything else you don’t want them doing, like credential stuffing, aren’t going to listen or respect that either.

• In terms of using a VPN, absolutely limit outside access to sensitive infrastructure but that’s not really where most companies experience pain from bots. That’s not to say that we don’t see bots attempting vulnerability scanning. These requests can be highly distributed too.

Companies ultimately reach out to companies like Cloudflare because the usual methods aren’t working for them. Onboarding some clients, I’ve seen more bot requests than human requests which can be detrimental for business.

I’m happy to answer any other questions you might have. While I do work in the industry, I don’t know everything. I just want to reiterate that I am not a fan of how things are currently on the Internet. I wish this was illegal as I think it would cut down on a lot of bot traffic which would make it much more manageable for everyone.

Hi! I didn’t forget about your response. I sifted through the links to find the study in question. I imagine my response isn’t going to satisfy you but please hear me out. I’m open to hearing your rebuttals regarding this too.

The study is absolutely correct with what they studied and the results they found. My main issues are the scope and some of the methodologies.

On one hand, I see the “AI” they used was able to solve captchas better than humans. My main issue with this is that this is one tool. Daily, I work on dozens of different frameworks and services, some that claim to leverage AI. The results and ability to pass captcha varies with each tool. There’s an inevitable back and forth with each tool as these tools learn how to bypass us and as we counter these changes. There’s not just one tool that everyone is using as their bot as is the case in the study, so it’s not exactly how this works in the real world.

I recognize that the list of sites they chose were the top 200 sites on the web. That said, there are more, up-and-coming captcha services that weren’t tested. I think it’s worth noting that the “captcha-less”, like Turnstile, approaches are still captcha but skip straight to proof of work and cutting out the human altogether.

We should absolutely take studies like this to heart and find better ways that don’t piss off humans. But the reality is that these tools are working to cut down on a vast amount of bot traffic that you don’t see. I understand if you’re not ok with that line of reasoning because I’m asking you to trust me, a random Internet stranger. I imagine each company can show you metrics regarding FP rates and how many bots are actually passing their captcha. Most do their best to cut down on the false positive rate.

How are you measuring this? On my end, when I look at the metrics I have available to me, the volume of bot requests that are passing captcha does not exceed that of humans. We continually review false positives and false negatives to make sure we aren’t impacting humans while still making it hard for bots.

There’s a whole world of tools you can use that do that for you now. It’s easier than ever. To me it’s concerning. The level of automation, coupled with a halfway decent LLM, can give you the ability to summon hordes of fake humans to social media. I can’t help but think it’s why X and Reddit don’t use any kind of anti-bot solution.

Trust me, my team and I often feel at odds with the part that infringes on privacy. As someone that enjoys and wants more privacy, I wish there were other solutions that didn’t create a type of dragnet. If it assuages some of your fears, I’ve never heard of the fingerprinting being sold or used outside of detections.

ALTCHA uses a proof-of-work mechanism to protect your website, apps, APIs, and online services from spam and unwanted content.

Unlike other solutions, ALTCHA’s Captcha alternative is free, open-source and self-hosted, does not use cookies nor fingerprinting, does not track users.

Emphasis are mine. I honestly do not know how this statement is possible. Captcha-less, proof-of-work solutions have to fingerprint on some level. It’s essentially having the browser prove it is what it claims to be. I get what they’re trying to say but it’s marketing. That said, I don’t know everything and maybe they have some method I’m not aware of. Grains of salt all around.

Given that the last updates to this repo were five years ago, I’m not too sure if it’s still valid. I don’t follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of “bot as a service”, captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don’t readily find much talk on like custom modified Chrome browsers. It’s fascinating how much effort is being funneled into this field.

Thanks for reading and commenting!

Welcome to bot detection. It’s a cat and mouse game, an ever changing battle where each side makes moves and counter moves. You can see this with the creation of captcha-less challenges.

But to say captcha are useless because bots can pass them is somewhat similar to saying your antivirus is useless because certain malware and ransomware can bypass it.

Yes, the industry is well aware of this. We do behavioral detection on both sessions and IPs. This is fairly basic.

Thank you for reading and considering the information.

That’s very kind of you. Thank you for the kind words. 🍻

Greed. I honestly don’t know if they’re even aware of the problem. Most corporations have cut teams to the bone and I can’t see Cloudflare being an exception. The janitor is probably writing detection rules now.