Yeah I don't trust any agent to follow document references consistently. I just manually add the relevant files to context every single time.

Though I know some people who have built an mcp that does exactly this: https://www.usable.dev/

It's basically a chat-bot frontend to your markdown files, with both rag and graph db indexes.