I am having the same problem

Yes, and when it works, it sometimes takes forever. Seems I'll never learn how to teach my parents' cat to fly!

I am having trouble getting the demo to run. It just errors out

Give this notebook a shot:

https://github.com/arjunguha/BigCode-demos/blob/main/bigcode...

A GPU will help, but I found it passable on a CPU as well.

The demo is up again!

Same here

Might be overloaded – if you have a GPU you can try running it locally by getting the model weights here: https://huggingface.co/bigcode/santacoder

Any idea how much GPU memory you'd need to run this locally?

EDIT: just tried it and it didn't seem to go past ~6gb

It's 1 billion model with Fp16 precision so 4-6 GB max.