Skip Navigation
  • Data loss and posts going into a black hole is something that you have observed in the instance, or via the Tor interface?

    The instance had some issues but I upgraded the server and I think most of those problems have reduced in frequency. But perhaps some issues remain. It would be good to know if posts are still getting lost.

    If you mean the Tor front-end: I chose the old reddit style UI because it runs without JavaScript, but I have not actually tested it much. I could look into other options if it is broken.

  • Good news! Just got a reply from them and they have increased the connection limit. They did not specify what the new number is, but hopefully it is high enough to not be an issue for the foreseeable future.

    So, if you do run into other similar reports after this comment I would appreciate it if you tag me again.

  • Thanks!

    Cost is not the bottleneck in this case... The problem is that I am rather ignorant about the options and their benefits/limitations. Moving the images the first time was painfully slow because of those same rate limits, and I expect the next migration to be the same, so I want to make a better choice next time and would rather find a solution with the current provider 😅

  • Thanks for the heads up. I am still trying to resolve this without a migration... I will try again to get a response from them as they have not replied in a week.

  • This is the current status:

    • The instance is serving the images via object storage. Specifically, I am making use of Contabo to save and serve the images.
    • I now know that the default limits are 250 requests / second and 80 Mbit/s: https://help.contabo.com/en/support/solutions/articles/103000275478-what-limits-are-there-on-object-storage-
    • It appears to me like when the requests are exceeded, the "Too many requests" error is triggered and it takes a few seconds before the requests are accepted again. This can happen if few users access the front page at once as this will fetch all of the thumbnails and icons on the page.
    • I have been in touch with Contabo's customer support via e-mail. But they mis-understood my original e-mails and thought I was speaking about increasing the maximum number of images that can be stored (3 million by default). I have clarified that I want to increase the rate limit and have been waiting for their response for a few days now.
    • The other solution would be to move the images to a different object storage provider. The migration is also limited to the 250 requests/s and 80 Mbit/s, so it will require turning off the images for 4 - 7 days while all the images are moved... Since I am not familiar with the policies of other object storage providers I would also need to do research to avoid falling into the same trap.

    So, I am hoping that Contabo's support will get back to me soon and allow me to increase the rate limits, as this would be the most straight forward approach.

  • And you are doing a great job at that! 😄

    Very interesting article, thanks for sharing. I agree that it is a good one to pin!!

  • Old AF

    Jump
  • I have been reaching out to the object storage provider to see if I can increase the rate limits... Unfortunately I might need to change to a different provider to overcome this. Since the migration takes several days, especially so because of those same rate limits, I would rather avoid this...

  • Hogs

    Jump
  • That's an error I had not seen before, but I also just encountered with this specific post. I will investigate, thanks.

  • This error is a rate limit from the object storage provider. I did not know of this limit when I chose them, and I still have not found a way to change the limit. I will send them an e-mail. If the limit can't be increased, one option is to pick another object storage provider, but the migration takes days.

  • Ah - does the exit node participate at all when accessing a .onion? Or is it skipped altogether?

    And the HTTP header thing is very cool, I did not know about that!

    I have added the header to the site and it works!

    I just added the following line to the location / {} block in the https server section:

    add_header Onion-Location http://mandermybrewn3sll4kptj2ubeyuiujz6felbaanzj3ympcrlykfs2id.onion/$request_uri;

  • Inderdaad, ik ben ook voor die marketing gevallen.

    Denk je dat mmWave binnenkort beschikbaar zal zijn? Op dit moment heb ik een sub-6 GHz 5G-router voor internet thuis. Ik ben benieuwd hoe een mmWave-router zou presteren als er een zendmast in de buurt is.

  • I did not know of the term "open washing" before reading this article. Unfortunately it does seem like the pending EU legislation on AI has created a strong incentive for companies to do their best to dilute the term and benefit from the regulations.

    There are some paragraphs in the article that illustrate the point nicely:

    In 2024, the AI landscape will be shaken up by the EU's AI Act, the world's first comprehensive AI law, with a projected impact on science and society comparable to GDPR. Fostering open source driven innovation is one of the aims of this legislation. This means it will be putting legal weight on the term “open source”, creating only stronger incentives for lobbying operations driven by corporate interests to water down its definition.

    [.....] Under the latest version of the Act, providers of AI models “under a free and open licence” are exempted from the requirement to “draw up and keep up-to-date the technical documentation of the model, including its training and testing process and the results of its evaluation, which shall contain, at a minimum, the elements set out in Annex IXa” (Article 52c:1a). Instead, they would face a much vaguer requirement to “draw up and make publicly available a sufficiently detailed summary about the content used for training of the general-purpose AI model according to a template provided by the AI Office” (Article 52c:1d).

    If this exemption or one like it stays in place, it will have two important effects: (i) attaining open source status becomes highly attractive to any generative AI provider, as it provides a way to escape some of the most onerous requirements of technical documentation and the attendant scientific and legal scrutiny; (ii) an as-yet unspecified template (and the AI Office managing it) will become the focus of intense lobbying efforts from multiple stakeholders (e.g., [12]). Figuring out what constitutes a “sufficiently detailed summary” will literally become a million dollar question.

    Thank you for pointing out Grayjay, I had not heard of it. I will look into it.

  • I find it satisfying to see the graph come down :)

  • Yes, sorry, there was some serious lagg in fetching posts from Lemmy World that persisted for several days and accumulated a 1-week delay.

    But after upgrading Mander it is now fetching data from LW quite rapidly and it should be back in-sync in about a day and a half from now.

    If you are curious about the ranking algorithm, there is some info here: https://join-lemmy.org/docs/contributors/07-ranking-algo.html

  • First of all, congratulations for bringing a baby girl into this world!! You must be really excited! I am very happy for you!

    This looks very cool. I set up a wiki (https://ibis.mander.xyz/) and I will make an effort to populate it with some Lemmy lore and interesting science/tech 😄 Hopefully I can set some time aside and help with a tiny bit of code too.

  • You can take a lot of control by using search commands. Here is a list of commands for Google, for example: https://www.lifewire.com/advanced-google-search-3482174

    By using commands like these you can narrow down your searches to the point that the impact of SEO is small. You give a much greater weight to the conditions that you have chosen.

    It can be a bit of work to write a good search query, but the database that search engines search through is massive, so it makes sense that it would take some work to do this right.

  • Search engines like google aggregate data from multiple sites. I may want to download a datasheet for an electronic component, find an answer to a technical question, find a language learning course site, or look for museums in my area.

    Usually I make specific searches with very specific conditions, so I tend to get few and relevant results. I think search engines have their place.

  • A botnet could have many unique accounts, and some could even appear like users. So I can't rule it out. I also haven't done a deeper dive into the accounts.

    But when a post gets popular I would expect it to get at least a few downvotes, regardless of what it is.

  • There is one account that has a single comment from 5 months ago that is downvoting most posts and comments. That one is very suspicious

    Other than that... No other accounts are as obvious. A few do have some reoccurrences but most of those votes do seem organic on first inspection.