If you use ansible, there is a great feature called ansible-vault that allows you to store all those sensitive bits right in the repo but encrypted and automatically readable by ansible when needed.