Probably for the same reason that I've seen processes put in place for performance testing a service (internally or on production) - because it puts a significant load on the service and automatic rate limiting, flagging, etc can be triggered.

if you benchmark your competitor, your competitor should have the ability to provide tuning so their numbers look as good as possible, before you run to the press.

Unless your competitor was offering a complimentary service to do that for all customers (tuning) then perhaps it shouldn't be done. I believe the idea of benchmarking is to get a sense of how the service would perform for a customer.

Consider the following scenario: You calling your ISP that you're going to do a benchmark. Then the ISP gives you the best service, at the expense of other customers bandwidth, for the duration of the test. Then after the test, the ISP removes the QoS modification, and reverts to the old behavior. That could end up with a biased, unrealistic result.

No, benchmarking aims to identify the peak performance of a service, not the average customer experience. Real Benchmarking is done for people who are serious about performance. marketing benchmarking is designed to attract naive customers.

As for the latter, that benchmark wouldn't reproduce and would require extensive configuration (which in a sense is just a part of negotiation between the customer and the client). If a company got caught faking benchmarks run by competitors and it could be shown publicly they did, that's pretty much the end of that company's public reputation.