You can disable it entirely apparently. I just checked the settings on Bing and there is a "Copilot response on result page" which can simply be turned off.

That one good thing about Microsoft, they aren't afraid of offering the users settings.