And with GPU just do this for the first command docker run -d --gpus=all -v olla... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		politelemon on Oct 6, 2023 \| parent \| context \| favorite \| on: Ollama is now available as an official Docker imag... And with GPU just do this for the first command `docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama` You'll need to have the NVidia container toolkit installed.

alexchantavy on Oct 7, 2023 [–]

Thank you!

For docker-compose, do

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Docs: https://docs.docker.com/compose/gpu-support/

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact