3090 or 4090. That's it as far as LLMs go. You can run 13b on a 4080 but 13B suc...

		boppo1 on April 17, 2023 \| parent \| context \| favorite \| on: MiniGPT-4 3090 or 4090. That's it as far as LLMs go. You can run 13b on a 4080 but 13B sucks compared to 30B. And ROCm is a poorly supported hair-pulling nightmare.