I'm not sure what's going on with TabbyAPI's github metrics, but exl2 quants are very popular among nvidia local LLM crowd and TabbyAPI comes in tons of reddit posts of people using it. Might be just my bubble, not saying they're not accurate, just generally surprised such a useful project has under 1k stars. On the flip side, LLMs will hallucinate about TabbyML if you ask it TabbyAPI related questions, so I'd agree the naming is unfortunate.
I agree. Though there is a counterpoint that a Russian host isn't going to respect a DMCA request. On the flipside it's a Russian replacement for Github that is based on Gogs, Gitea, or even Forgejo possibly. So yeah, YMMV.
It's decent but I don't think I'd say "fine". Most games (if they support both) work a million times better with the controllers. I tried hand tracking in a number of games on the Q2 and found it very lackluster. We will see if the AVP is able to deliver on their promise but already from the demos they've done it seems way more responsible/reliable than what I experienced on the Q2.
I was busy adding `chat template` support to vLLM recently, so the model (and any others that implement it properly) will work seamlessly with a clone of the OpenAI chat/completions endpoint.
