Oh yeah, I was referring to billions of params there. And if you want to use a hosted model to play with it, I would recommend DeepSeek, their pricing is great and I’ve found it gets pretty decent results. The way I’d recommend using it would be through crush or a similar tool. It’s a very different experience from using it in a web chat for example and asking it to come up with code.
And yeah, the better the model is at getting stuff right on the first try the less hand holding you need to do. There are also some tricks I found that can help. One thing I get the model to do is to write a plan in markdown for the steps it’s going to do. In particular, you can get it to generate a mermaidjs diagram, then inspect it visually, and then tell it change step x to do blah. Another thing you can do is write the scaffolding by hand, like making the file structure you want, put in function stubs, and then have the LLM start filling in the blanks. It’s a really great way to focus it so it doesn’t try to get creative. My general experience is that they’re good at implementing tasks that are focused and well defined, but if you let them get creative then they can go off track really fast. Another thing I found is that if it doesn’t get the solution mostly right on the first shot, it’s unlikely to converge on a good solution. It will not rethink the problem, but will simply attempt to add kludges to address the specific issues you point out. So, you’re better off to just start from scratch and try to reframe the problem statement.
It’s important to keep in mind that it’s a tool you have to spend some time learning the behaviors of, and how to get the most out of it.
Yeah, I might try doing that if I get a bit of energy. Cause some of this stuff is obvious in retrospect, but it took me a while to realize to do these things.
Oh yeah, I was referring to billions of params there. And if you want to use a hosted model to play with it, I would recommend DeepSeek, their pricing is great and I’ve found it gets pretty decent results. The way I’d recommend using it would be through crush or a similar tool. It’s a very different experience from using it in a web chat for example and asking it to come up with code.
And yeah, the better the model is at getting stuff right on the first try the less hand holding you need to do. There are also some tricks I found that can help. One thing I get the model to do is to write a plan in markdown for the steps it’s going to do. In particular, you can get it to generate a mermaidjs diagram, then inspect it visually, and then tell it change step x to do blah. Another thing you can do is write the scaffolding by hand, like making the file structure you want, put in function stubs, and then have the LLM start filling in the blanks. It’s a really great way to focus it so it doesn’t try to get creative. My general experience is that they’re good at implementing tasks that are focused and well defined, but if you let them get creative then they can go off track really fast. Another thing I found is that if it doesn’t get the solution mostly right on the first shot, it’s unlikely to converge on a good solution. It will not rethink the problem, but will simply attempt to add kludges to address the specific issues you point out. So, you’re better off to just start from scratch and try to reframe the problem statement.
It’s important to keep in mind that it’s a tool you have to spend some time learning the behaviors of, and how to get the most out of it.
In this comment thread you gave great tips! But very few people are likely to find them. Why don’t you organize them and write a Lemmy post?
Yeah, I might try doing that if I get a bit of energy. Cause some of this stuff is obvious in retrospect, but it took me a while to realize to do these things.