I think that the training of small models is relatively ethical, given the modest scale. Even more, if they use open datasets.
Do you know of any models matching these criteria? My understanding is that the cheap-to-produce/open-weight models are either just not good or have been trained on output from ChatGPT and Claude, neither of which sounds like a practical alternative to Big Tech.
Nvidia’s Nemotron is the most prominant one. Its dataset is open.
EDIT:
I like to think “community” generative models fall into this category too. Anima is a recent example, and while its dataset is unknown, its almost certainly a bunch of Danbooru images and whatever is permissively licensed enough to be uploaded there, probably archived some time ago.
Another open-ish category I like is “experimental finetunes.” Like this one from today:
Do you know of any models matching these criteria? My understanding is that the cheap-to-produce/open-weight models are either just not good or have been trained on output from ChatGPT and Claude, neither of which sounds like a practical alternative to Big Tech.
Nvidia’s Nemotron is the most prominant one. Its dataset is open.
EDIT:
I like to think “community” generative models fall into this category too. Anima is a recent example, and while its dataset is unknown, its almost certainly a bunch of Danbooru images and whatever is permissively licensed enough to be uploaded there, probably archived some time ago.
Another open-ish category I like is “experimental finetunes.” Like this one from today:
https://huggingface.co/collections/Pageshift-Entertainment/pagestorm-research-preview
It’s based on a very modest closed data model (Mistral 14B), but its finetuned to do exactly one thing, and their work is reproducible.
There’s a whole wide world of machine learning out there that isn’t Claude or OpenAI.