I had couple 13B models loaded in, it was ok. But I really wanted a 30B so I got a runpod. I’m using it for api, I did spot pricing and it’s like $0.70/hour

I didn’t know what to do with it at first, but when I found Simply Tavern I kinda got hooked.

Amazon had some promotion in the summer and they had a cheap 3060 so I grabbed that and for Stable Diffusion it was more than enough, so I thought oh… I’ll try out llama as well. After 2 days of dicking around, trying to load a whack of models, I spent a couple bucks and spooled up a runpod instance. It was more affordable then I thought, definitely cheaper than buying another video card.

Ahhhh didn’t know what that was, great. Thanks for the heads up

Bought it right away, though I wish they would charge more. I hate SaaS and would rather support Devs with a higher purchase price.