My network is dual stack and all my VMs are Debian. No issues here.

anything more concrete?

i was struck by an issue in this area a few years ago and i think it was fixed by tuning gai.conf to prefer v4 because there were some repo servers that had broken v6

Which exact repo was that?

I seem to think that the primary UK mirror (http://ftp.uk.debian.org/) was problematic for v6 for a while? But these days the https://deb.debian.org/ mirror is really heavily pushed, and that one seems to work just fine.

edit: but this was quite a few years ago, from grepping my IRC logs, somewhere around 2016!

there are thousands of debian mirrors, not that far fetched that at least one of them has broken v6