

I’ve run small models (a few Gb in size) on my steam deck. It gives reasonably fast responses (faster than a person would type).
I know that they’re far from state-of-the art, but they do work and I know that the Steam Deck is not going to be using much power.
I have
if [ -z "$SSH_AUTH_SOCK" ] ; then eval $(ssh-agent -s) fi
At the end of .bashrc and
AddKeysToAgent yes
In .ssh/config so that it auto-adds keys I unlock.