Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)T

tehnomad

@ tehnomad @lemm.ee

Posts
1
Comments
13
Joined
3 yr. ago

  • What skills are needed to self host without too many headaches?

    Jump

  • One thing I would do differently is setup LDAP and OIDC so you can use the same authentication credentials for different apps (at least the ones that support them). I use LLDAP and Authelia for this purpose.

  • Using Mac M2 Ultra 192GB to Self-Host LLMs?

    Jump

  • I found a VRAM calculator for LLMs here: https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator

    Wow it seems like for 128K context size you do need a lot of VRAM (~55 GB). Qwen 72B will take up ~39 GB so you would either need 4x 24GB Nvidia cards or the Mac Pro 192 GB RAM. Probably the cheapest option would be to deploy GPU instances on a service like Runpod. I think you would have to do a lot of processing before you get to the breakeven point of your own machine.

  • Using Mac M2 Ultra 192GB to Self-Host LLMs?

    Jump

  • The context cache doesn't take up too much memory compared to the model. The main benefit of having a lot of VRAM is that you can run larger models. I think you're better off buying a 24 GB Nvidia card from a cost and performance standpoint.

  • Hardware recs for newb? Please.

    Jump

  • I would suggest an Intel N100 mini PC if you are planning to transcode video files with Plex. Intel Quick Sync performs better than AMD for media transcoding.

  • Is there any way to save storage on similar images?

    Jump

  • I wasn't sure if it was AI or not. According to the description on GitHub:

    Utilizes state-of-the-art algorithms to identify duplicates with precision based on hashing values and FAISS Vector Database using ResNet152.

    Isn't ResNet152 a neural network model? I was careful to say neural network instead of AI or machine learning.

  • Is there any way to save storage on similar images?

    Jump

  • Yeah, the duplicate finder uses a neural network to find duplicates I think. I went through my wedding album that had a lot of burst shots and it was able to detect similar images well.

  • Is there any way to save storage on similar images?

    Jump

  • Not sure if you're aware, but Immich has a duplicate finder

  • What are your favorite tools for monitoring Linux and individual docker containers?

    Jump

  • Super Power-Efficient ITX Nas CPU/Motherboard?

    Jump

  • I am running my disks as mirrors.

  • Super Power-Efficient ITX Nas CPU/Motherboard?

    Jump

  • I have a Terramaster 4-bay DAS

  • Super Power-Efficient ITX Nas CPU/Motherboard?

    Jump

  • I'm running a mini-PC with the N100, 12GB RAM, and 2x18TB mirrored drives on ZFS and it seems to work well.

  • [Immich] What is the "proper" way to navigate migration from another service (all photos are already on the server)

    Jump

  • I migrated using the CLI tool to upload before external libraries came out, and it worked well for me. I think I would have preferred to use external libraries for things like my wedding photos, so there aren't duplicates on my server, but it's not a big deal.

  • Selfhosted @lemmy.world
    tehnomad @lemm.ee

    Best current hardware solution for selfhosting?