A potential employer called me multiple times after I turned down a job interview appointment. Should I reconsider?

I'm having a bit of a nervous fit over here, so I could really use the advice. The job offer was nothing special, not a dream job but also not a bullshit job, and I applied out of sheer force of habit. The job does not interest me I'm semi-qualified, as in I have a degree that by default over-qualifies me yet it is in a completely different field. The last mail they sent me a couple of minutes ago said something like "We'd deeply regret if you turned us down", which is something I have never gotten after applying for a job.

The reason why I haven't replied with a firm "No" is that I'm still looking for a job. However, my current situation is that I may get something that pays less than this "boring job", but is morally much more acceptable for me, in a couple of months. It's not a long shot, the training for this other job is just gonna happen in a month, so I don't know until I passed the training.

So, question is, should I just show up to the interview for the job that I don't care about or should I just say no again? I'm leaning towards the latter, but I wonder if I would get in trouble with my unemployment counselor for not accepting a job interview invite.

Edit: I should point out that the job offer is in the public sector. (Sorry for the vagueness, I'm trying not to doxx myself)