afk_strats

@ afk_strats @lemmy.world

Posts

26
Comments

325
Joined

2 yr. ago

1d ago

Alright, y'all were right, fuck Proton. This was the last straw for me.
Jump
2

afk_strats @lemmy.world 1d ago
Bummer.

1d ago

A Guide to the Circular Deals Underpinning the AI Boom | A web of interlinked investments raises the risk of cascading losses if AI falls short of its potential.

Great spin, Bloomberg. You were very careful to only talk about "potential" and missing revenue targets when the real problem is that a bunch of grifters pretended they were on the absolute verge of AGI when, in fact, they were/are bulding advanced bullshit machines.

I will eat my words when a model can come up with an original thought

1d ago

Alright, y'all were right, fuck Proton. This was the last straw for me.

Jump

afk_strats @lemmy.world 1d ago

Howdy. For the clarity of users such as myself, can you please clarify which "Proton" you're referring to.

3d ago

Android won't kill sideloading after all, but new verification rules will make it harder

Jump

afk_strats @lemmy.world 3d ago

This framing still sucks. Google is blocking apps THEY don't approve on YOUR phone.

3d ago

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task – MIT Media Lab

Jump

afk_strats @lemmy.world 3d ago

That's a great observation!...

4d ago

Majority of CEOs report zero payoff from AI splurge

Jump

afk_strats @lemmy.world 4d ago

I think organizing labor is a useful skill. I just think doing it to the sole benefit of "shareholder value" is what's killing us. Is that liberal of me? I can't imagine a society where work isn't done by people and work needs some form of organization.

4d ago

Majority of CEOs report zero payoff from AI splurge

Jump

afk_strats @lemmy.world 4d ago

Here are some of the schools I know set the pace for Business education in the US. Feels like social responsibility is more than an afterthought.

Again, not defeding, "the MBAs" running companies. I'm defending the schools

https://www.hbs.edu/mba/academic-experience/curriculum

https://www.gsb.stanford.edu/programs/mba/why-stanford-mba

https://mitsloan.mit.edu/values/our-values

4d ago

Spong Berb Adventures #6

Jump

afk_strats @lemmy.world 4d ago

4d ago

Majority of CEOs report zero payoff from AI splurge

Jump

afk_strats @lemmy.world 4d ago

Broken systems elevate psychopath leaders into positions of wealth and power, and people who want those things exploit the fastest path there by getting degrees who put you on that track.

By this MBA logic, do we close CompSci for the the poor code coming out of Microsoft, close Law Schools because social rights are being lost, engineering schoolings because infrastructure doesn't meet current needs?

My point is to blame the CEOs and their shitty behaviour, not the schools that, to my knowledge, try to educate reasonable policy, law, ethics, HR, etc.

Disclaimer: not an MBA

5d ago

ROCm on older generation AMD gpu

Jump

afk_strats @lemmy.world 5d ago

ROCm on my 7900xt is solid. ROCm on my MI50s (Vega) is a NIGHTMARE

1w ago

Guys, what's the best Linux distro to install on my PC?

Jump

afk_strats @lemmy.world 1w ago

You use Alpine as a daily driver? With a GUI? How does it do with hardware support?

2w ago

Researchers figured out how to run a 120-billion parameter model across four regular desktop PCs

Jump

afk_strats @lemmy.world 2w ago

I still think AI is mostly a toy and a corporate inflation device. There are valid use cases but I don't think that's the majority of the bubble

For my personal use, I used it to learn how models work from a compute perspective. I've been interested and involved with natural language processing and sentiment analysis since before LLMs became a thing. Modern models are an evolution of that.
A small, consumer grade model like GPT-oss-20 is around 13GB and can run on a single mid-grade consumer GPU and maybe some RAM. It's capable of parsing text and summarizing, troubleshooting computer issues, and some basic coding or code review for personal use. I built some bash and home assistant automatons for myself using these models as crutches. Also, there is software that can index text locally to help you have conversations with large documents. I use this with documentation for my music keyboard which is a nightmare to program and with complex APIs.
A mid-size model like Nemotron3 30B is around 20GB can run on a larger consumer card (like my 7900xtx with 24 gb of VRAM, or 2 5060tis with 16gb of vRAM each) and will have vaguely the same usability as the small commercial models, like Gemini Flash, or Claude Haiku. These can write better, more complex code. I also use these to help me organize personal notes. I dump everything in my brain to text and have the model give it structure.
A large model like GLM4.7 is around 150GB can do all the things ChatGPT or Gemini Pro can do, given web access and a pretty wrapper. This requires big RAM and some patience or a lot of VRAM. There is software designed to run these larger models in RAM faster, namely ik_llama but, at this scale, you're throwing money at AI.

I played around with image creation and there isn't anything there other than a toy for me. I take pictures with a camera.

2w ago

Researchers figured out how to run a 120-billion parameter model across four regular desktop PCs

Jump

afk_strats @lemmy.world 2w ago

I think you're missing the point or not understanding.

Let me see if I can clarify

What you're talking about is just running a model on consumer hardware with a GUI

The article talks about running models on consumer hardware. I am making the point that this is not a new concept. The GUI is optional but, as I mentioned, llama.cpp and other open source tools provide an OpenAI-compatible api just like the product described in the article.

We've been running models for a decade like that.

No. LLMs, as we know them, aren't that old, were a harder to run and required some coding knowledge and environment setup until 3ish years ago, give or take when these more polished tools started coming out.

Llama is just a simplified framework for end users using LLMs.

Ollama matches that description. Llama is a model family from Facebook. Llama.cpp, which is what I was talking about, is an inference and quantization tool suite made for efficient deployment on a variety of hardware including consumer hardware.

The article is essentially describing a map reduce system over a number of machines for model workloads, meaning it's batching the token work, distributing it up amongst a cluster, then combining the results into a coherent response.

Map reduce, in very simplified terms, means spreading out compute work to highly pararelized compute workers. This is, conceptually, how all LLMs are run at scale. You can't map reduce or parallelize LLMs any more than they already are. The article doent imply map reduce other than taking about using multiple computers.

They aren't talking about just running models as you're describing.

They don't talk about how the models are run in the article. But I know a tiny bit about how they're run. LLMs require very simple and consistent math computations on extremely large matrixes of numbers. The bottleneck is almost always data transfer, not compute. Basically, every LLM deployment tool is already tries to use as much parallelism as possible while reducing data transfer as much as possible.

The article talks about gpt-oss120, so were aren't talking about novel approaches to how the data is laid out or how the models are used. We're talking about tranformer models and how they're huge and require a lot of data transfer. So, the preference is try to keep your model on the fastest-transfer part of your machine. On consumer hardware, which was the key point of the article, you are best off keeping your model in your GPU's memory. If you can't, you'll run into bottlenecks with PCIe, RAM and network transfer speed. But consumers don't have GPUs with 63+ GB of VRAM, which is how big GPT-OSS 120b is, so they MUST contend with these speed bottlenecks. This article doesn't address that. That's what I'm talking about.

2w ago

Researchers figured out how to run a 120-billion parameter model across four regular desktop PCs

Jump

afk_strats @lemmy.world 2w ago

This is basically meaningless. You can already run gpt-OSS 120 across consumer grade machines. In fact, I've done it with open source software with a proper open source licence, offline, at my house. It's called llama.cpp and it is one of the most popular projects on GitHub. It's the basis of ollama which Facebook coopted and is the engine for LMStudio, a popular LLM app.

The only thing you need is around 64 gigs of free RAM and you can serve gpt-oss120 as an OpenAI-like api endpoint. VRAM is preferred but llama.cpp can run in system RAM or on top of multiple different GPU addressing technologies. It has a built-in server which allows it to pool resources from multiple machines....

I bet you could even do it over a series of high-ram phones in a network.

So I ask is this novel or is it an advertisement packaged as a press release?

3w ago

MiniMax M2.1 is open source: SOTA for real-world dev workflows and agentic systems

Jump

afk_strats @lemmy.world 3w ago

It's open weight. I haven't been able to find the code or data to reproduce the model. Its licenced as MIT with additional provision that the model nave be displayed prominently on UIs

3w ago

2025, My Year of The Linux Desktop

Jump

afk_strats @lemmy.world 3w ago

Totally. I wouldn't even have the conversation with someone just because. I think what I meant was, for a user like her, mostly web and some office tasks, Linux is perfectly suitable.